Egger Test

Egger Test for Funnel Plot Asymmetry in Meta-Analysis

The Egger test checks whether effect estimates in a meta-analysis shift with study precision, by regressing each study's standardised effect on its precision and testing whether the intercept differs from zero. Egger and colleagues proposed it in the BMJ in 1997, and the Cochrane Handbook calls it the first and best known of the tests for funnel plot asymmetry. In health technology assessment the question has a price attached, because the pooled effect is often a key input to a health economic model, and an effect inflated by small studies carries through into the estimated health gain and the ICER. A significant result shows an association between effect size and study size without identifying its cause, and publication bias is only one possible explanation. The test has low power when studies are few, and with odds ratios its false-positive rate can exceed the nominal level, which led to the Harbord and Peters tests.

The regression behind the test

In a funnel plot each study's effect is plotted against a measure of its size, and an unbiased set of studies forms a symmetrical inverted funnel. To measure asymmetry, Egger and colleagues regressed the standard normal deviate of each study, its effect divided by its standard error, on its precision, the inverse of the standard error.

$$\frac{\hat{\theta}_i}{s_i} = a + b \, \frac{1}{s_i} + \varepsilon_i$$

where $\hat{\theta}_i$ is the effect estimate from study $i$ on a scale where effects are roughly normal (a log odds ratio in the original paper, or a mean difference), $s_i$ is its standard error, $a$ is the intercept, $b$ is the slope and $\varepsilon_i$ is the residual error. The test of no small-study effects is the test that $a$ equals zero.

Small trials sit near the origin on both axes whatever effect they report. Without small-study effects the points scatter around a line through the origin whose slope $b$ gives the effect, and Egger and colleagues explain that negative intercepts indicate more pronounced beneficial effects in smaller studies.

The Stata manual for meta bias shows that this regression is equivalent to a weighted regression of the effect estimates on their standard errors.

$$\hat{\theta}_i = b + a \, s_i + \varepsilon_i^{*}, \qquad w_i = \frac{1}{s_i^2}$$

where $w_i$ is the inverse-variance weight for study $i$, $\varepsilon_i^{*}$ is the residual in the weighted form, and $a$ and $b$ are the coefficients of the first equation with their roles swapped. The Egger intercept is therefore the slope of a line through the funnel plot of effect against standard error, and $b$ is the effect predicted for a study with a standard error of zero.

$$t = \frac{\hat{a}}{\widehat{\operatorname{SE}}(\hat{a})}$$

where $\hat{a}$ is the estimated intercept, $\widehat{\operatorname{SE}}(\hat{a})$ is its standard error and $t$ is compared with a t distribution on $K-2$ degrees of freedom, with $K$ the number of studies.

Versions and thresholds

The original paper ran weighted and unweighted versions of the first regression and reported whichever gave the intercept further from zero. It judged asymmetry at P < 0.1 with 90% confidence intervals, because the small number of trials in most meta-analyses limits statistical power. The Stata documentation records that Sterne and Egger later strongly recommended against the weighted version, as it lacks a theoretical justification. The Cochrane Handbook notes that none of the recommended tests is implemented in RevMan, so other software is needed.

When the test should be used

Sterne and colleagues published recommendations in the BMJ in 2011, which the Cochrane Handbook (version 6.5, chapter 13) summarises. As a rule of thumb, tests for funnel plot asymmetry should not be used with fewer than 10 studies, because power is then usually too low to distinguish chance from real asymmetry, and with substantial between-study heterogeneity the minimum may be substantially more. Tests should not be used when the studies have similar standard errors, results should be read alongside visual inspection of the funnel plot, and as far as possible the testing strategy should be specified in advance.

For mean differences, Sterne and colleagues state that the Egger test may be used and there is no reason to prefer newer tests. They expect power to be greater than with dichotomous outcomes, but judge use with substantially fewer than 10 studies unwise.

Odds ratios and the Harbord and Peters alternatives

Sterne and colleagues explain that the standard error of the log odds ratio is mathematically linked to the size of the odds ratio even without small-study effects, and the Cochrane Handbook states that the original Egger test is not recommended for odds ratios or standardised mean differences for this reason. Harbord, Egger and Sterne reported that existing tests can give false-positive rates above the nominal level with binary outcomes when treatment effects are large, events per trial are few or trials are of similar size.

Harbord's test replaces the effect and its variance with the efficient score and Fisher's information, and its authors found a false-positive rate close to the nominal level, with power similar to the Egger test, when heterogeneity was small. Peters and colleagues regress the log odds ratio on the inverse of the total sample size, which is independent of the effect by construction, and found appropriate type I error rates with comparable power under low heterogeneity. Sterne and colleagues add that when the heterogeneity variance of the log odds ratios exceeds 0.1, only the arcsine test with random effects of Rücker and colleagues had been shown to work reasonably well.

Worked example with ten trials of an enhanced recovery programme

All numbers in this example are illustrative. A cost analysis of an enhanced recovery programme after surgery needs the reduction in hospital stay. Ten trials report mean differences in days, negative values favouring the programme.

1. Pool the trials. The weights sum to 114.19 and the weighted mean differences to −84.99.

$$\hat{\theta}_{FE} = \frac{-84.99}{114.19} = -0.744$$

where $\hat{\theta}_{FE}$ is the inverse-variance fixed-effect pooled mean difference, a reduction of 0.744 days.

2. Set up the regression. With precision as $x$ and the standard normal deviate as $y$, the means are $\bar{x} = 28.845 / 10 = 2.8845$ and $\bar{y} = -24.394 / 10 = -2.4394$, the sum of squared deviations of $x$ is $S_{xx} = 30.982$ and the sum of cross-products is $S_{xy} = -14.629$.

3. Estimate slope and intercept.

$$\hat{b} = \frac{S_{xy}}{S_{xx}} = \frac{-14.629}{30.982} = -0.4722, \qquad \hat{a} = \bar{y}-\hat{b}\,\bar{x} = -2.4394 + 0.4722 \times 2.8845 = -1.077$$

where $\hat{b}$ is the estimated slope and $\hat{a}$ the estimated Egger intercept.

4. Test the intercept. The residual sum of squares is 3.561 on 8 degrees of freedom, giving a residual variance of 3.561 / 8 = 0.4452.

$$\widehat{\operatorname{SE}}(\hat{a}) = \sqrt{0.4452 \times \left( \frac{1}{10} + \frac{2.8845^2}{30.982} \right)} = 0.4051, \qquad t = \frac{-1.077}{0.4051} = -2.66$$

where the bracketed term is the variance factor for an intercept in simple linear regression. On 8 degrees of freedom the two-sided p-value is 0.029, and the 95% confidence interval is −1.077 plus or minus 2.306 × 0.4051, or −2.011 to −0.143. The smaller trials report larger reductions in stay.

5. Compare estimates. The slope of −0.472 days is the reduction predicted for a trial of unlimited size. Pooling only the four most precise trials, whose weights sum to 96.56 and whose weighted mean differences sum to −64.68, gives −64.68 / 96.56 = −0.670 days.

6. Carry them into the cost model. At an illustrative GBP 450 per bed day and a programme cost of GBP 250 per patient, the pooled estimate saves 0.744 × 450 = GBP 334.80 in bed days, a net saving of GBP 84.80 per patient. The large-trial estimate gives 0.670 × 450 = GBP 301.50, a net saving of GBP 51.50, and the extrapolated estimate gives 0.472 × 450 = GBP 212.40, a net cost of GBP 37.60.

7. Interpret. Cochran's Q is 6.71 on 9 degrees of freedom, so no heterogeneity is detected, yet the Egger test finds asymmetry, and the 1997 paper notes that the asymmetry test targets a specific type of heterogeneity. Whether the programme saves money depends on which estimate is believed. If the small trials delivered the programme more intensively, the asymmetry reflects true heterogeneity rather than missing studies, and the pooled effect overstates what wider adoption would achieve. The defensible presentation keeps the pooled result as the base case and reports the other two as scenario analyses.

Using the result in HTA evidence synthesis

The NICE manual (PMG36, section 3.3.4) expects a systematic review to a pre-defined protocol that minimises the risk of biased selection of studies, with potential biases identified, and section 3.2.2 asks that the impact of limitations in the evidence on bias and uncertainty be fully characterised and ideally quantified. An asymmetry test is one input to that judgement, alongside searches of trial registries.

Asymmetry also bears on the synthesis model. Sterne and colleagues note that a random-effects model weights studies more equally than a fixed-effect model, so its estimate is pulled towards the smaller studies when effects are related to standard errors. For intravenous magnesium after myocardial infarction, ISIS-4 took 90% of the weight in a fixed-effect analysis showing no benefit, while small studies dominated a random-effects analysis suggesting benefit, so they recommend comparing the two when asymmetry comes with between-study heterogeneity.

NICE DSU Technical Support Document 3 (section 5.3) describes taking the true effect as that of a study of infinite size, estimated as the intercept in a regression of treatment effect on study variance. Sterne and colleagues caution that the broad applicability of such extrapolation is uncertain, because bias is hard to separate from heterogeneity or chance. The Copas model offers a selection model sensitivity analysis, and meta-regression can test whether a study feature such as setting explains the pattern.

Egger test and the Begg test

The Begg test asks the same question through a rank correlation between standardised effects and their variances. Applied to the eight meta-analyses in the 1997 paper it found significant asymmetry in only one, and the Stata documentation describes it as no longer recommended in the literature.

Common misreadings

The commonest error is to equate a significant Egger test with publication bias. Sterne and colleagues list reporting biases, poor methodological quality in small studies, true heterogeneity, artefactual associations between effects and standard errors, and chance as sources of asymmetry, and state that publication bias can probably be excluded, even with a significant test, if small studies show smaller benefits than large ones or if no studies have significant results. The opposite error is to read a non-significant test as proof of an unbiased evidence base, when the tests typically have low power, so a non-significant result cannot exclude bias. Reporting only the most favourable of several tests breaches the 2011 recommendations, which ask that all results be reported.

Sources

• Dias S, Sutton AJ, Welton NJ, Ades AE. NICE DSU Technical Support Document 3: Heterogeneity: subgroups, meta-regression, bias and bias-adjustment. Decision Support Unit; September 2011, last updated April 2012. Section 5.3. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD3-Heterogeneity.final-report.08.05.12.pdf
• Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634. https://doi.org/10.1136/bmj.315.7109.629
• Harbord RM, Egger M, Sterne JAC. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Statistics in Medicine. 2006;25(20):3443-3457. https://doi.org/10.1002/sim.2380
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 3.2.2 and 3.3.4. https://www.nice.org.uk/process/pmg36/chapter/evidence
• Page MJ, Higgins JPT, Sterne JAC. Chapter 13: Assessing risk of bias due to missing evidence in a meta-analysis. In: Higgins JPT, Thomas J, Chandler J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-13
• Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006;295(6):676-680. https://doi.org/10.1001/jama.295.6.676
• StataCorp. Stata Meta-Analysis Reference Manual: meta bias. College Station, TX: StataCorp LLC; 2025. https://www.stata.com/manuals/metametabias.pdf
• Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. https://doi.org/10.1136/bmj.d4002