WISDOM

WISDOM (Weighted Information Schema for Distributed Open Metrics) is an open review framework designed from first principles to revolutionise the way we recognise and reward contributions. It began life as a disruptive scholarly publishing model and evolved through active experimentation at Open Heart + Mind to become a generalizable framework for diverse, self-organizing communities. By creating a fair, transparent system for recognizing and rewarding all forms of contribution, WISDOM empowers contributors and connects diverse communities with aligned goals.

At the heart of WISDOM is an inclusive review protocol that earns reviewers immediate rewards for their reviews, which are converted into group-level estimates of the relative value of contributions. Contributors can then be directly rewarded for the value they provide, and respected for their proven expertise and reliability. The entire process is open, decentralized and autonomous, ensuring everyone can participate, while guarding against bias and cheating through a self-reflective meta-review process.

Learn more by watching this video introduction, reading about our experiments at Open Heart + Mind, or by viewing our README.

OHM Gathering

Current political and information systems have proven inadequate to deal with the challenges we face as a global society. Society is becoming increasingly fractured, with communities retreating online and forming echo chambers that isolate rather than unite. In the wake of COVID, it has become clear that we need more spaces for people to connect and share diverse ideas and experiences in a non-judgemental setting.

OHM Gatherings are inclusive spaces where people come together to share art, science, wisdom, goods and services with the community. They also serve as a safe space to prototype new models for society, like the WISDOM framework. Our goal is to develop transparent contribution templates that people around the world can use to replicate OHM gatherings in a decentralised and autonomous fashion. Read more about some of our gatherings here.

Free Our Knowledge

Open science practices have the potential to radically transform the way scientists conduct and communicate their research. Unfortunately, hyper-competition between academics keeps us locked into the old way of doing things (e.g., journals, publishers, sequestering data/code, etc.) and limits the full embrace of digital communication made available by the internet.

Free Our Knowledge aims to overcome this collective action problem by organising critical masses of researchers to act together. Using our open source website, researchers can pledge to take action if and when a critical mass of pledges is reached. By acting together, we hope to accelerate the adoption of open science practices in academia, while protecting those individuals who feel unsafe to act on their own.

After founding and leading this project for four years, I handed it over to a new board in 2023 so I could focus on OHM. See our website for current campaigns and my presentations for conferences where I’ve shared these ideas.

MERITS

In recent years, there has been an explosion of interest in post-publication peer-review, with many models proposing multidimensional article-level ratings as an alternative to unidimensional journal-level metrics (e.g, the journal impact factor). In line with these ideas, a growing number of preprint review platforms solicit reviewers’ ratings of preprints on multiple dimensions (e.g. PREreview, Rapid Reviews Covid-19), but these ratings typically remain siloed within each project, thus limiting their interoperability, searchability, and comparison between sites.

MERITS (Metaresearch Evaluation Repository to Identify Trustworthy Science) is a prototype database for organising and storing article ratings in a common, machine-readable format. We anticipate that this database will: (1) facilitate meta-research into the nature of these ratings and how they relate to real-world outcomes (see below), (2) help various stakeholders in society (e.g., researchers, journalists, the general public) identify peer-rated research and assess it accordingly (e.g., new COVID-related preprints), and (3) foster innovation in scholarly evaluation and publication.

I co-led this project at the 2021 eLife Innovation Sprint, feeding those learnings into what would eventually become the WISDOM framework. You can learn more about MERITS in this project brief and this video introduction.

Meta-research

In parallel to MERITS, I was also developing a meta-research (research on research) paradigm into the types of judgements experts make during peer review. This research paradigm would aim to explore (a) what dimensions/characteristics researchers are interested in when they evaluate research, (b) how these dimensions can best be captured during a structured peer review process, (c) how these dimensions relate to one another, (d) how these dimensions relate to real-world outcomes (e.g., citations, patents), among other questions. My plan was to evolve this line of research alongside the MERITS database and other projects, with the view to develop an open, inclusive, evidence-based, researcher-controlled system of scholarly communication and evaluation. You can see a glimpse of what I was planning in this conference presentation. Through the process of developing this line of research, I realised that I could prototype these ideas more rapidly outside of academia, giving rise to Open Heart + Mind.