Publication Bias and the Funnel Plot: A Complete Guide for Systematic Reviewers (2026)

Every systematic review and meta-analysis rests on a foundational assumption: that the retrieved literature is a representative sample of all conducted research. That assumption is routinely violated. Studies yielding statistically significant or directionally positive results are more likely to be submitted, accepted, and published than studies yielding null or negative findings. The result is a corpus of evidence that systematically inflates effect estimates — a problem collectively termed publication bias, and one that the funnel plot was specifically designed to detect. For any researcher conducting a meta-analysis in 2026, a rigorous assessment of this bias is not optional: it is mandated by the Cochrane Handbook, the PRISMA 2020 statement, and the GRADE framework.

This guide offers a comprehensive, research-grade treatment of publication bias and its detection via funnel plot asymmetry analysis. It covers the theoretical basis of the funnel plot, the standard visual and statistical approaches (Egger’s regression test, Begg’s rank correlation test), the trim-and-fill correction method, contour-enhanced funnel plots, the important distinction between publication bias and other causes of small-study effects, and practical implementation in R (metafor) and RevMan. Where specific methods or data are cited, a primary source is provided.

1. What Is Publication Bias?

Publication bias is a form of reporting bias — a class of distortions that arise when the decision to report, submit, or publish research findings is influenced by the nature of the results themselves. The term was formalised by Sterling (1959), who observed that virtually all published psychological studies reported statistically significant results, a pattern irreconcilable with the statistical power typical of the field at the time.

The mechanism operates at multiple levels. At the study level, investigators may not write up null-result experiments, a phenomenon termed the file-drawer problem (Rosenthal, 1979). At the submission level, authors may self-censor, anticipating journal preferences. At the editorial level, peer reviewers and editors — even when instructed otherwise — demonstrably favour papers that confirm hypotheses over those that refute them. The cumulative effect is a published literature biased toward positive findings.

Publication bias sits within a broader taxonomy of reporting biases that includes:

• Outcome reporting bias: selectively reporting only favourable outcomes from a study that measured multiple outcomes.
• Time-lag bias: positive results are published faster than null results, distorting the literature at any given point in time.
• Language bias: English-language journals disproportionately publish significant results; non-English journals may contain a higher proportion of null findings.
• Citation bias: significant studies accumulate more citations, increasing their visibility in database searches.
• Multiple publication bias: positive trials are more likely to be reported in multiple papers, inflating their apparent weight in a meta-analysis.

Systematic reviewers must address all of these. The funnel plot is primarily a tool for detecting publication bias and other small-study effects in a set of studies included in a meta-analysis; it cannot directly detect outcome reporting bias within individual studies.

2. Why Publication Bias Matters for Evidence Synthesis

The practical stakes of publication bias in systematic reviews are high. When a meta-analysis pools only published studies, the resulting pooled effect estimate will exceed the true effect — potentially by a large margin. The landmark analysis by Egger et al. (1997, BMJ) found funnel plot asymmetry in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews, indicating that bias was widespread even in ostensibly rigorous literature.

The consequences extend beyond academic accuracy. Inflated effect estimates from biased meta-analyses influence clinical guidelines, drug approval processes, and public health policy. Several high-profile reversals in medicine — where early meta-analyses supported interventions later shown by large trials to be ineffective or harmful — have been attributed partly to publication bias operating in the constituent trial literature.

Connection to related integrity problems is worth noting. Publication bias is one driver of the broader reproducibility problem in research: when the literature over-represents positive findings, subsequent researchers attempting to replicate them face a higher-than-expected failure rate. It is also closely related to the ecosystem pressures that foster predatory publishing, where journals publish anything for a fee, and to the review latency documented across disciplines — peer review timelines and desk-rejection rates themselves reflect editorial selection pressures that can amplify bias. And as retraction rates continue to rise, post-hoc correction of the literature becomes an increasingly important complement to prospective bias prevention.

3. The Funnel Plot: Construction and Interpretation

3.1 Axes and Geometry

A funnel plot is a simple scatter plot in which each included study is represented by a single point. The horizontal axis displays the study’s effect estimate — typically a standardised mean difference (SMD), log odds ratio (log OR), or log relative risk (log RR). The vertical axis displays a measure of study precision, most commonly the standard error (SE) of the effect estimate, with the scale inverted so that large, precise studies appear at the top and small, imprecise studies appear at the bottom.

The inverted-SE convention, advocated by Sterne and Egger (2001) and subsequently formalised in the Cochrane Handbook, produces a plot with a natural funnel shape when no bias is present. Three diagonal lines are typically overlaid: the central line represents the pooled effect estimate, and the two outer lines represent the 95% pseudo-confidence interval of the funnel (calculated as pooled effect ± 1.96 × SE), widening downward as SE increases.

Alternative precision metrics — sample size, inverse variance, inverse SE — were more commonly used in early implementations. Sample size on the y-axis remains in use in some contexts, though it can distort the shape of the funnel when studies have unequal group sizes.

3.2 Interpreting Symmetry and Asymmetry

Under the null hypothesis of no publication bias and no heterogeneity, the points should scatter symmetrically around the pooled effect in a rough inverted triangle. Large studies cluster tightly near the apex; small studies scatter more widely at the base — but symmetrically. The funnel is expected to be symmetric because, in an unbiased literature, small studies with large positive effects should be equally balanced by small studies with large negative effects.

Asymmetry manifests as a gap — most commonly in the lower-left quadrant of the plot. This is the region where small studies with null or negative effects would appear; when such studies are absent from the literature (unpublished), the funnel is missing points in this area. The published studies therefore skew upward and to the right of the true effect, producing an asymmetric appearance.

Important nuance: asymmetry should always be interpreted cautiously. Sterne et al.’s 2011 guidance in the BMJ (doi: 10.1136/bmj.d4002) made explicit that funnel plot asymmetry is a marker of small-study effects — the tendency for smaller studies to yield larger effect estimates — of which publication bias is only one possible cause. Reviewing the plot should prompt investigation, not automatic inference of bias.

3.3 The Ten-Study Threshold

The Cochrane Handbook specifies that funnel plots should only be used when a meta-analysis contains at least ten studies. With fewer studies, the plot has insufficient power to distinguish asymmetry from random sampling variation, and visual inspection is unreliable. This threshold applies equally to formal statistical tests of asymmetry. Reviewers working with small evidence bases should rely on narrative assessment of grey literature searches and trial registries rather than graphical approaches.

4. Causes of Funnel Plot Asymmetry Beyond Publication Bias

One of the most important — and most frequently violated — principles in the analysis of funnel plots is recognising that asymmetry does not equal publication bias. Sterne et al. (2011) catalogued the following distinct causes of funnel plot asymmetry:

This taxonomy has significant practical implications. When a reviewer observes asymmetry, the appropriate response is contextual investigation — examining the distribution of study designs, populations, and intervention intensities — rather than mechanically applying the trim-and-fill adjustment. The contour-enhanced funnel plot (Section 8) was developed specifically to help distinguish publication bias from heterogeneity as the cause of asymmetry.

5. Egger’s Regression Test

5.1 Statistical Basis

The Egger test, published by Egger, Davey Smith, Schneider, and Minder in the BMJ (1997, 315:629–634), provides a formal statistical test for funnel plot asymmetry. The procedure regresses the standardised effect estimate (effect size divided by its standard error, equivalent to the study’s Z-score) on the precision of the estimate (1/SE, the inverse of the standard error). Formally:

Z_i = a + b × (1/SE_i) + ε_i

Where Z_i is the standardised effect for study i, 1/SE_i is its precision, a is the intercept, b is the slope (which estimates the effect size under infinite precision), and ε_i is the error term. The regression is weighted by 1/SE² (i.e., by the inverse variance of each study). In the absence of bias, the regression line should pass through the origin (intercept a = 0). A statistically significant intercept — tested by a standard t-test against the null H₀: a = 0 — indicates funnel plot asymmetry.

5.2 Performance and Limitations

Egger’s test has greater statistical power than Begg’s test and is generally recommended as the primary asymmetry test when using effect-size/precision axes. However, it has known limitations. For binary outcomes reported as odds ratios, the test can produce spurious asymmetry when event rates are low, because the effect estimate and its variance are not independent under those conditions. Several authors recommend using the arcsine transformation or risk differences for binary outcomes to mitigate this artefact. The test also lacks power when studies are few: with fewer than ten studies, the nominal type-I error rate is poorly controlled, and the test should not be used.

A rule of thumb from the Cochrane Handbook: use a significance threshold of p < 0.10 (rather than the conventional 0.05) for Egger’s test, because the test is underpowered in many realistic meta-analysis settings and a more liberal threshold reduces the risk of false negatives.

6. Begg’s Rank Correlation Test

Begg and Mazumdar (Biometrics, 1994, 50:1088–1101) proposed a non-parametric alternative that evaluates the correlation between the standardised effect sizes and the ranks of their variances, using Kendall’s tau (τ) as the measure of association. The test examines whether there is a significant rank correlation between effect estimates and their sampling variances — a correlation that would indicate that larger-variance (smaller) studies are reporting systematically different effects from lower-variance (larger) studies.

Begg’s test is more conservative than Egger’s — it produces fewer false positives but also has lower statistical power, particularly in meta-analyses with fewer than 25 studies. For large meta-analyses (75 or more studies), the two tests typically converge. In practice, systematic reviewers most often apply Egger’s test as the primary test and report Begg’s tau as supplementary evidence, noting convergence or divergence between the two.

Both tests should be interpreted within the context established by examining the overall literature review methodology, including whether grey literature, trial registries (ClinicalTrials.gov, WHO ICTRP), and conference abstracts were searched. A non-significant Egger’s test against a background of inadequate grey literature searching should not be interpreted as reassurance against publication bias.

7. The Trim-and-Fill Method

7.1 Algorithm

The trim-and-fill method, introduced by Duval and Tweedie in two companion papers — one in Biometrics (2000, 56:455–463) and one in the Journal of the American Statistical Association (2000, 95:89–98) — provides both a test for asymmetry and an adjusted effect estimate. The algorithm proceeds in three stages:

1. Trim: Studies are ranked by effect size. The most extreme studies on one side of the pooled effect (typically the right, corresponding to the over-represented positive studies) are iteratively removed until the remaining distribution is approximately symmetric. The number of trimmed studies, k₀, estimates the number of missing studies.
2. Centre: The pooled effect is recalculated from the trimmed studies to obtain a more accurate estimate of the true centre.
3. Fill: The trimmed studies are restored and their mirror images — imputed studies placed symmetrically on the opposite side of the centre — are added to the dataset. The final pooled estimate and confidence interval are computed from the augmented dataset.

Three estimators for k₀ are available: L₀, R₀, and Q₀. The R₀ estimator is most widely cited and is the default in metafor’s trimfill() function, though empirical evaluations suggest that L₀ performs comparably in most scenarios.

7.2 Interpreting the Adjusted Estimate

The key output of trim-and-fill is the adjusted pooled effect — the estimate that would be expected if the literature were symmetrically complete. If the adjusted estimate is substantially smaller (in absolute terms) than the unadjusted estimate, this suggests meaningful publication bias may be inflating the original result. If the adjusted and unadjusted estimates are similar, publication bias is unlikely to be a serious concern even if formal tests are nominally significant.

The practical significance of the adjustment should be assessed on the outcome scale, not on the scale of statistical significance. A shift from SMD 0.45 to 0.38 may or may not be clinically meaningful depending on the outcome; a shift from OR 3.1 to 2.9 in a low-event-rate setting may have different implications than the same proportional shift for a common outcome.

7.3 Critical Limitations

Trim-and-fill carries several important caveats that are frequently underemphasised in applied reviews:

• Assumption of symmetry: The method assumes that in the absence of publication bias, the funnel would be symmetric around a single true effect. This assumption fails when substantial heterogeneity is present. Applied to a heterogeneous dataset, trim-and-fill may incorrectly identify heterogeneous variation as missing studies and over-adjust.
• Non-estimability when k₀ = 0: If the algorithm estimates zero missing studies, no adjustment is made. This does not prove the absence of publication bias; it may simply reflect the method’s limited power in small meta-analyses.
• The adjusted estimate is not an unbiased estimator: Duval and Tweedie themselves cautioned that the adjusted estimate should be regarded as a sensitivity analysis, not a definitive correction. It provides a lower bound on the plausible effect, not the true effect.
• Asymmetry caused by heterogeneity inflates k₀: If smaller studies genuinely exhibit larger effects due to population differences — a common scenario in complex social or behavioural interventions — trim-and-fill will remove real studies and produce an artificially attenuated adjusted estimate.

These limitations have led methodologists to recommend reporting trim-and-fill alongside other sensitivity analyses (selection models, p-curve analysis) rather than as a standalone correction.

8. Contour-Enhanced Funnel Plots

A significant advance in funnel plot methodology was the contour-enhanced funnel plot, proposed by Peters, Sutton, Jones, Abrams, and Rushton in the Journal of Clinical Epidemiology (2008, 61:991–996). The innovation lies in superimposing contour lines corresponding to conventional levels of statistical significance (p = 0.10, p = 0.05, p = 0.01) on the standard funnel plot, creating shaded regions that demarcate where studies of various significance levels would appear.

8.1 Reading the Contours

The contour-enhanced plot is centered at zero (the null effect), dividing the plot into regions:

• White (central) region: p > 0.10 — studies here would not reach conventional significance.
• Light grey region: 0.05 < p ≤ 0.10 — studies with borderline significance.
• Mid grey region: 0.01 < p ≤ 0.05 — conventionally significant studies.
• Dark grey region: p ≤ 0.01 — highly significant studies.

8.2 Distinguishing Publication Bias from Heterogeneity

The diagnostic power of the contour-enhanced plot lies in examining where missing studies appear to be absent:

• If the gap falls predominantly in the non-significant (white) region, publication bias is a plausible explanation — the missing studies are precisely those that would not have been published due to non-significant results.
• If the gap falls in the statistically significant regions, publication bias is a less likely explanation, because there is no obvious reason why significant findings would be suppressed. In this case, true heterogeneity or between-study differences in effect modifiers are more plausible causes of the asymmetry.

This distinction has direct implications for what action reviewers should take. If the asymmetry is concentrated in the non-significant region, application of trim-and-fill or selection model adjustments is appropriate. If the asymmetry is in the significant region, the reviewer should investigate substantive moderators rather than applying bias corrections.

9. Software Implementation: R metafor and RevMan

9.1 R: The metafor Package

The metafor package for R (Viechtbauer, 2010; current version 5.0-1 as of April 2026) is the most comprehensive open-source implementation of meta-analysis methods and provides all standard publication bias tools. The following illustrates a complete workflow:

# Install and load metafor
install.packages("metafor")
library(metafor)

# Fit a random-effects model (REML estimator)
res <- rma(yi = effect_size, vi = variance, data = my_data, method = "REML")

# Standard funnel plot (effect size vs. SE)
funnel(res, xlab = "Effect Size (SMD)", ylab = "Standard Error")

# Contour-enhanced funnel plot centered at zero
funnel(res,
       level = c(90, 95, 99),
       shade = c("white", "gray75", "gray55"),
       refline = 0,
       legend = TRUE)

# Egger's regression test
regtest(res)

# Begg's rank correlation test
ranktest(res)

# Trim-and-fill (R0 estimator, default)
tf <- trimfill(res)
summary(tf)
funnel(tf)  # Funnel showing imputed studies (open circles)

The trimfill() output reports k₀ (the estimated number of missing studies), the adjusted pooled estimate, its standard error, and a 95% confidence interval. Imputed studies appear as open (hollow) circles in the funnel plot, clearly distinguished from observed studies.

For contour-enhanced plots, level = c(90, 95, 99) specifies the three significance contours (90%, 95%, 99% CI boundaries, corresponding to p = 0.10, 0.05, 0.01), and refline = 0 centres the plot on the null. The funnel() function documentation and a worked example are available on the metafor project website.

9.2 RevMan

Cochrane’s RevMan software (Review Manager, currently RevMan Web and RevMan 5.x desktop) generates funnel plots automatically as part of the meta-analysis workflow. Funnel plots in RevMan are available under the “Figures” tab once a meta-analysis has been run with ten or more studies. RevMan does not currently implement Egger’s test or Begg’s test natively, nor does it offer contour-enhanced plots; reviewers undertaking formal statistical testing will need to export data to R, Stata, or a dedicated tool such as Comprehensive Meta-Analysis (CMA).

The Cochrane Training platform provides guidance on interpreting RevMan-generated funnel plots within the context of the Cochrane Handbook Chapter 13 (Assessing risk of bias due to missing evidence in a meta-analysis), which now supersedes the older Chapter 10 guidance on reporting biases in Handbook version 5.1.

9.3 Other Software Options

Stata (metabias and metatrim commands), SPSS with the meta-analysis add-on, and the web-based JASP and jamovi packages all implement core publication bias tests. Comprehensive Meta-Analysis (CMA) provides a point-and-click interface covering Egger’s test, Begg’s test, trim-and-fill, and the Orwin fail-safe N in a single tool, making it popular in clinical and health psychology settings.

10. Common Misinterpretations and Hard Limits

Despite being one of the most widely used tools in evidence synthesis, the funnel plot is also one of the most frequently misused. The following list consolidates the most consequential errors documented in methodological literature:

11. Reporting Standards: PRISMA, Cochrane, and GRADE

Three major reporting and evidence-grading frameworks specifically address publication bias assessment:

PRISMA 2020

The updated PRISMA 2020 statement (Page et al., BMJ, 2021, doi: 10.1136/bmj.n71) includes a dedicated checklist item (Item 22) on assessment of reporting biases. Reviewers are expected to describe methods used to assess reporting biases, report results of any tests performed, and discuss implications for conclusions. Funnel plots and asymmetry tests satisfy this requirement when accompanied by appropriate interpretation; they do not satisfy it when presented without interpretation or when applied to meta-analyses with fewer than ten studies. For guidance on constructing the PRISMA flow diagram that documents your literature search process, see the step-by-step guide to drawing a PRISMA 2020 flow diagram.

Cochrane Handbook (Version 6)

Chapter 13 of the current Cochrane Handbook, “Assessing risk of bias due to missing evidence in a meta-analysis,” provides the authoritative Cochrane guidance. Key recommendations include: (a) use contour-enhanced funnel plots to aid interpretation of asymmetry; (b) supplement visual inspection with Egger’s test (preferred) and/or Begg’s test; (c) use trim-and-fill as a sensitivity analysis only; (d) interpret results in the context of the comprehensiveness of the literature search, including trial registers and grey literature.

GRADE

The GRADE framework includes “publication bias” as one of eight domains that can reduce the certainty of a body of evidence. Reviewers rate publication bias as “undetected,” “suspected,” or “strongly suspected” and downgrade certainty accordingly. A strongly suspected publication bias rating typically requires a combination of funnel plot asymmetry, a positive Egger’s test, and contextual evidence of selective reporting (e.g., registered but unreported trials, absence of grey literature).

The growing adoption of open peer review practices and pre-registration requirements across journals is gradually narrowing the conditions under which publication bias can operate undetected — providing a structural complement to the analytical tools described here.

Frequently Asked Questions