Reporting Null Results: About the Project

When studies produce null results, they tend to go unpublished. But other researchers can still learn from them and build on them—that is, if they are shared rather than filed away. That’s why IPL scholars created a convenient template to help researchers tell others what they tried and why they think they reached a null finding.

To accumulate knowledge, science requires both transparency and replication. Transparency facilitates the ability of scholars to explore the underlying data and understand the process by which inferences were made from that data. Replication enables scholars to probe whether published results are robust to alternative speciﬁcations and measurements. It also builds on the ﬁndings of prior studies in establishing the conditions under which demonstrated relationships may or may not hold in other contexts.

A commitment to publishing studies with null results—studies where the results are not statistically signiﬁcant at conventional levels—has important implications for both replication and transparency. This type of commitment would provide full visibility for researchers into the entire range of studies that have been carried out, facilitating our collective ability to build on prior work and update our beliefs on the basis of both published and unpublished studies.

Unfortunately, the publication of null results remains uncommon in the social sciences due in part to publication bias, in which journals prize statistically signiﬁcant ﬁndings on a given topic and not those without such relationships. Publication bias disincentivizes scholars from investing time in writing up and sharing null ﬁndings, ultimately preventing valuable information from reaching the scientiﬁc community.

At the Immigration Policy Lab, we believe that posting null results is an important part of the scientiﬁc enterprise. We launched the Reporting Null Results project to mitigate the issue of publication bias within our scholarly community by establishing a standardized and efficient mechanism to disseminate studies with null results.

Ready to contribute to this effort? Here’s what you need to start sharing null results in your own research community. For more, read papers from our team members here and here.