The Reproducibility Crisis in Numbers: 2026 Data Roundup

Only 36 of 100 psychology studies produced a statistically significant result in the same direction when independently repeated. That single number, from the largest coordinated replication project ever run, is the anchor statistic of what researchers now call the reproducibility crisis. This article compiles the core reproducibility crisis statistics — study by study, with sample sizes, methods, and exact figures — so you can cite the underlying data directly rather than a secondhand summary.

If you are looking for the causes behind these numbers and the reforms proposed to fix them, our companion pillar, the reproducibility crisis in research explained, covers p-hacking, publication bias, and preregistration in depth. This article stays focused on the data itself.

The Reproducibility Project: Psychology (2015)

In 2015, the Open Science Collaboration — a large, distributed team of researchers — published the results of a three-year effort to replicate 100 studies drawn from three prominent psychology journals: Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition. Ninety-seven percent of the original studies had reported statistically significant results. When independently replicated using the original materials and much larger, pre-registered samples, only 36% of the replications reproduced a statistically significant effect in the same direction as the original.

Effect sizes told a similar story: among the studies that did replicate successfully, the replication effect size was on average about half the size of the effect reported in the original publication. This combination — a large drop in significant results, plus a systematic shrinkage in effect size among the studies that did replicate — is now treated as a textbook illustration of how published effect sizes can be inflated relative to their true population value, a pattern researchers attribute in part to publication bias favouring larger, “cleaner” findings. The project’s scale is part of why it remains so heavily cited: it was not a handful of researchers cherry-picking studies to challenge, but a coordinated, pre-registered, community-wide effort involving well over one hundred contributing scientists.

Camerer et al. (2018): Social Science Experiments

Psychology is not the only field to have been systematically stress-tested this way. In 2018, Colin Camerer and colleagues published a replication of 21 experimental social-science studies that had appeared in Nature and Science between 2010 and 2015. The replication team used sample sizes roughly five times larger than the originals and pre-registered their analysis plans in advance, in consultation with the original authors — a design choice intended to rule out the “underpowered replication” objection that critics had raised against some earlier, smaller replication efforts.

They found a statistically significant effect in the same direction as the original study for 13 of the 21 studies — approximately 62%. As in the psychology project, the average replicated effect size was around half of the originally reported effect size. A Bayesian analysis in the same paper estimated the underlying “true positive” rate at roughly 67%, depending on which complementary replicability indicator was used (estimates across the paper’s several metrics ranged between 57% and 67%). The fact that a well-powered, pre-registered replication of prestige-journal social science studies still landed well short of 100% is one of the reasons this paper is treated as confirming, rather than contradicting, the pattern first observed in psychology.

Nature’s 2016 Survey of 1,576 Researchers

Rather than replicating specific studies, journalist and researcher Monya Baker’s 2016 Nature survey asked scientists directly about their experiences and beliefs regarding reproducibility. Of the 1,576 researchers who responded across a range of disciplines, more than 70% reported that they had personally tried and failed to reproduce another scientist’s experiments, and more than half said they had failed to reproduce even one of their own past experiments. Roughly 90% of respondents agreed that science was experiencing either a “significant” or a “slight” reproducibility crisis, and only 73% said they believed at least half of the papers published in their own field could be trusted.

Because this is a survey of opinion and self-reported experience rather than a controlled replication project, its numbers should be read as evidence of widespread researcher concern and anecdotal experience, not as a direct measurement of an actual field-wide replication rate. Its value lies less in the precision of any one figure and more in the consistency of the picture: a large majority of practising scientists, across disciplines, report having personally encountered non-reproducible results in their own work, not just read about them in a journal.

Replication Rates by Study, Side by Side

Why Effect Sizes Shrink on Replication

The pattern of replicated effect sizes landing at roughly half the original is not unique to psychology or social science — it shows up consistently enough across replication projects that methodologists have a name for it: regression to the mean combined with publication bias. In brief, when only statistically significant results tend to get published, the published record is skewed toward studies that happened to produce larger-than-typical effects by chance, especially in original studies run with small sample sizes. A well-powered replication, drawing a fresh sample, is more likely to land closer to the true underlying effect — which, on average, is smaller than the inflated original estimate. This is a statistical explanation, not an accusation of misconduct against original authors; it is a structural consequence of how small-sample research combined with a publish-only-significant-results norm behaves in aggregate.

How to Interpret a “Failed” Replication

A replication that does not reach statistical significance is not automatically proof that the original finding was false. Several explanations can produce a “failed” replication: the original effect may have been genuinely overstated due to small original samples and publication bias; the replication itself may be underpowered or may differ from the original in some methodologically relevant way (a different population, setting, or measurement instrument); or the true effect may simply be smaller than originally estimated — consistent with the effect-size shrinkage pattern seen in both the OSC and Camerer projects. Researchers studying replication methodology generally treat a single failed replication as evidence to be weighed, not as a verdict, which is one reason both major projects reported multiple complementary replication metrics rather than a single pass/fail number.

Reproducibility vs Replicability: A Note on Terms

The terms are often used interchangeably in casual writing, but methodologists draw a distinction. Reproducibility typically refers to obtaining the same result from the original data and analysis code — essentially, checking the original researchers’ arithmetic and computational pipeline. Replicability refers to obtaining a consistent result from a new dataset, collected independently under a similar design. The Open Science Collaboration and Camerer et al. projects covered here are both replication studies in this stricter sense — new data was collected — even though the popular term “reproducibility crisis” is applied broadly to cover both concepts across the wider literature. When you cite these figures in your own work, using the more precise term for what was actually done (replication, in both cases) will read as more methodologically careful to an examiner.

What Has Changed Since These Studies

Since 2015, structural reforms aimed at improving replicability have expanded significantly across many fields: preregistration of hypotheses and analysis plans, registered reports (where peer review happens before results are known), open data and open materials requirements from journals and funders, and larger recommended sample sizes for novel effects. These reforms are widely discussed as direct responses to the OSC and Camerer findings, but measuring their aggregate, field-wide effect on actual replication rates requires long-run tracking that is still ongoing — there is no single follow-up statistic yet that confirms replication rates across an entire field have measurably risen as a direct result. If you want a full breakdown of these reform mechanisms, our guide to how to preregister a study using OSF and AsPredicted is a practical starting point, and our broader guide on research methodology tips for reproducibility covers study-design habits that make an individual project more replicable.

Why This Matters If You Are Writing a Thesis

If you are citing prior literature in your own thesis or dissertation, these figures are a useful reminder to check whether a foundational study you are relying on has been independently replicated, and if so, with what result and what effect size. A single original study, however well-cited, carries a meaningfully different evidentiary weight than a finding that has survived a well-powered, pre-registered replication. Where possible, look for replication registries, meta-analyses, or systematic reviews of your key sources rather than relying on the original publication alone — and where you cannot find one, it is reasonable and honest to note that limitation explicitly in your literature review rather than presenting a single unreplicated study as settled fact.

FAQ

Whether you are writing a methods chapter that cites these figures or designing your own study with replicability in mind, keeping your data, code, and drafts organised from the start makes the eventual preregistration and reporting process much easier — Tesify can help you structure that documentation as you write.