Researchers in the social sciences are increasingly confronting the “file drawer problem.” When studies fail to prove a theory, validate a particular intervention or treatment, or produce some other significant effect, they are unlikely to be published or otherwise shared publicly. This leads to a bias in the literature on a given topic, as well as wasted effort when researchers undertake similar studies that others had already tried.
IPL researchers have introduced a simple template to make it easier for researchers to share their null results. Now, they’re proposing ways other key players can support this movement for greater transparency in social science research.
Imagine “a world in which less research funding is wasted, policy decisions are based on the most complete and unbiased evidence possible, and scientific expertise is more widely valued and trusted by the public,” they write in a new paper co-authored with colleagues from Center for Effective Global Action (CEGA) at the University of California, Berkeley, and other members of the Berkeley Initiative for Transparency in the Social Sciences.
Here, CEGA program manager Aleksandar Bogdanoski discusses the proposal to encourage funders, journals, and registries to join researchers in committing to the full reporting of null results. The problem stems from “the absence of norms around good reporting practices, lack of appropriate reporting infrastructure, and the absence of suitable incentives and enforcement mechanisms,” he writes. He and his IPL co-authors present a framework they’ve dubbed “Reporting All Results Efficiently” (RARE):
1. Registries need to develop infrastructure to allow researchers to report hypotheses results and provide interpretation. Such infrastructure would be ideally paired with guidance on best practices for reporting results (e.g. in the form of reporting templates).
2. In their IRB and funding applications, researchers need to describe their plans to share results with the research community and their research participants.
3. Using their mandate of ensuring that research participants benefit from participating (also known as the ethical principle of beneficence), IRBs can serve as the first checkpoint for researchers’ plans.
4. Research funders, who wield a great deal of power in promoting research norms can ask that grantees share past research outcomes as part of their application, while also providing additional support for reporting results (e.g. funding for research assistant time and storage, which per the forthcoming data policy, can be included in proposals to the National Institutes of Health).
5. Finally, journals can use full reporting as a condition for acceptance and incentivize good reporting practices by making an explicit commitment to publish good quality research regardless of the nature of its results.
For more discussion of the RARE framework, read the full article here.
