Copas Model

Copas Model for Publication Bias in Meta-Analysis

The Copas model, usually called the Copas selection model, treats the trials in a meta-analysis as the visible part of a larger set of studies, some never published, and asks how far the pooled effect would move if small studies with unfavourable results were less likely to appear. Copas introduced it in 1999 and developed it with Shi in 2000 and 2001. In health technology assessment the question has a direct cost, because a pooled relative effect usually becomes a parameter in a health economic model, so an overstated effect becomes overstated health gain and, for a technology that costs more than its comparator, a lower ICER. The model rests on two linked equations and a sensitivity analysis, and its adjusted estimates can be carried into a cost per responder and compared with those of trim and fill and PET-PEESE.

Two linked equations for the result and the decision to publish

The model joins the usual random-effects model for each study's reported effect to a probit selection equation that decides whether the study is published. A correlation parameter links the two; when it is zero, publication is unrelated to the result and the published studies give an approximately unbiased pooled estimate.

$$\hat{\theta}_i = \theta_i + \sigma_i \varepsilon_i, \qquad \theta_i \sim N(\theta, \tau^2), \qquad Z_i = \gamma_0 + \frac{\gamma_1}{s_i} + \delta_i, \qquad \operatorname{corr}(\varepsilon_i, \delta_i) = \rho$$

where $\hat{\theta}_i$ is the effect reported by study $i$ (such as a log odds ratio), $\theta_i$ its true effect, $\theta$ the overall mean effect, $\tau^2$ the between-study heterogeneity variance, $\sigma_i$ the within-study standard error, $\varepsilon_i$ and $\delta_i$ standard normal errors, $Z_i$ a latent propensity to publish, $s_i$ the reported standard error, $\gamma_0$ and $\gamma_1$ the selection parameters and $\rho$ the correlation between sampling error and propensity. Study $i$ is published only when $Z_i > 0$, so the following holds.

$$P(Z_i > 0) = \Phi\left( \gamma_0 + \frac{\gamma_1}{s_i} \right)$$

where $\Phi$ is the standard normal cumulative distribution function. $\Phi(\gamma_0)$ is the probability of publishing a study with an infinitely large standard error, and $\gamma_1$ sets how fast that probability rises as the standard error shrinks. When $\rho$ is positive and larger effects are favourable, a small study with a favourable result is more likely to clear the threshold, which produces funnel plot asymmetry.

Carpenter and colleagues describe the model as derived from the Heckman two-stage regression model, the econometric approach to sample selection, so it is a selection model for studies missing not at random. Standard properties of the normal distribution give the bias of a study once it is known to be published, as follows.

$$E(\hat{\theta}_i \mid Z_i > 0) - \theta = \rho \, \sigma_i \, \lambda(u_i), \qquad u_i = \gamma_0 + \frac{\gamma_1}{s_i}, \qquad \lambda(u) = \frac{\phi(u)}{\Phi(u)}$$

where $u_i$ is the mean propensity to publish study $i$, $\phi$ is the standard normal density and $\lambda$ is the inverse Mills ratio, which grows as the probability of publication falls. The bias rises with the standard error and with the strength of selection, so the smallest published studies overstate the effect most.

Why the selection parameters are varied rather than estimated

Copas and Shi maximise the likelihood of the published estimates conditional on publication. That likelihood can peak on a very flat plateau, because the data say little about unseen studies. Copas's answer, described in the Cochrane Handbook, is a sensitivity analysis in which $\gamma_0$ and $\gamma_1$ are fixed over a grid, and at each point $\theta$, $\tau$ and $\rho$ are estimated by maximum likelihood. Each grid point implies a number of unpublished studies, given by the following.

$$M = \sum_{i=1}^{N} \frac{1 - P(Z_i > 0 \mid s_i)}{P(Z_i > 0 \mid s_i)}$$

where $M$ is the expected number of unpublished studies, $N$ is the number of published studies and $P(Z_i > 0 \mid s_i)$ is the publication probability at the standard error of published study $i$.

The implementation by Carpenter, Rücker and Schwarzer, now the copas function of the R package metasens, plots the adjusted estimate against the probability of publishing the study with the largest standard error, by default from around 0.3 to 1. At each point a test asks whether the remaining funnel plot asymmetry exceeds chance, and degrees of selection with p-values above 0.1 are treated as more plausible. Copas and Shi's test of fit is a likelihood ratio test that, without heterogeneity, is equivalent to the Egger test.

A published analysis of phenobarbital before preterm birth

A published application shows what the model offers and what it cannot settle. Carpenter, Rücker and Schwarzer analysed nine trials of maternal phenobarbital for periventricular haemorrhage in preterm infants. The random-effects odds ratio was 0.49 (0.32 to 0.74) and a regression test for asymmetry gave p = 0.011. The asymmetry was explained only when the probability of publishing the least precise trial was as low as about 0.4; at 0.37 the adjusted odds ratio was 0.83 (0.61 to 1.15), with about 9 studies implied unpublished, and the benefit was no longer significant.

They record that the original review authors, Crowther and Henderson-Smart, had noted that the two largest, higher-quality trials showed no benefit, and they describe the result as allowing for selection bias and/or other small-study effects. The model cannot say which was at work.

From twelve illustrative trials to a cost per responder

A cost-effectiveness model needs the odds ratio of treatment response for a new therapy against standard care. All numbers are illustrative, and positive log odds ratios favour the new therapy. Two large trials (standard error 0.10) average 0.31 and ten small trials (standard error 0.40) average 0.55; each trial is treated as reporting its group average, and heterogeneity is set to zero.

1. Pool without adjustment.

$$\hat{\theta}_{FE} = \frac{200 \times 0.31 + 62.5 \times 0.55}{262.5} = \frac{96.375}{262.5} = 0.3671$$

where $\hat{\theta}_{FE}$ is the unadjusted inverse-variance pooled log odds ratio, an odds ratio of 1.444. The small trials sit 0.24 above the large ones, which is funnel plot asymmetry.

2. Choose a scenario. Suppose a large trial is published with probability 0.95 and a small one with probability 0.35. The standard normal quantiles are 1.645 and −0.385, and the reciprocals of the standard errors are 10 and 2.5, giving the following.

$$\gamma_1 = \frac{1.645 - (-0.385)}{10 - 2.5} = \frac{2.030}{7.5} = 0.2707, \qquad \gamma_0 = -0.385 - \frac{0.2707}{0.40} = -1.062$$

where $\gamma_1$ and $\gamma_0$ are the selection parameters implied by the two probabilities.

3. Calculate the inverse Mills ratios. With densities of 0.1031 and 0.3704 at the two quantiles, the ratios are as follows.

$$\lambda_L = \frac{0.1031}{0.95} = 0.1085, \qquad \lambda_S = \frac{0.3704}{0.35} = 1.0583$$

where $\lambda_L$ and $\lambda_S$ are the inverse Mills ratios for a published large and small trial.

4. Solve for the correlation and the adjusted effect. Setting each group mean equal to its expected value under the model, $\theta + 0.10 \rho \lambda_L = 0.31$ and $\theta + 0.40 \rho \lambda_S = 0.55$, and subtracting gives the following.

$$\rho = \frac{0.55 - 0.31}{0.40 \times 1.0583 - 0.10 \times 0.1085} = \frac{0.24}{0.4124} = 0.582, \qquad \hat{\theta}_{adj} = 0.31 - 0.10 \times 0.582 \times 0.1085 = 0.3037$$

where $\rho$ is the correlation that reproduces the asymmetry and $\hat{\theta}_{adj}$ is the adjusted pooled log odds ratio, an odds ratio of 1.355. This is a moment approximation; the Copas method maximises the conditional likelihood, which also changes the weights, so its figures would differ somewhat. The implied number of unpublished trials is as follows.

$$M = 2 \times \frac{0.05}{0.95} + 10 \times \frac{0.65}{0.35} = 0.105 + 18.571 = 18.68$$

where $M$ is as defined above. About 19 trials, nearly all small, would sit unpublished behind the twelve found.

5. Repeat across scenarios.

With no selection the asymmetry is unexplained. With the correlation re-estimated at each grid point, as in a real Copas analysis, both selection scenarios reproduce the group means exactly and give almost the same odds ratio, because stronger selection is offset by a weaker correlation. The data cannot separate them, which is the flat likelihood in miniature; the choice rests on the test for residual asymmetry and on whether about 57 unpublished trials, rather than about 19, is credible, which trial registries can help to check.

6. Carry the result into the economic model. With an illustrative baseline response probability of 30% (odds 0.4286) and incremental cost of £1,200 per patient, the unadjusted odds ratio gives the following response probability on the new therapy.

$$\frac{0.4286 \times 1.444}{1 + 0.4286 \times 1.444} = \frac{0.6189}{1.6189} = 0.3823$$

where the numerator is the treated odds of response.

Allowing for selection raises the cost per extra responder by about 22% to 23%, which matters if the unadjusted result sits close to what a decision maker would accept.

Copas model, funnel plot tests, trim and fill and PET-PEESE

The Begg test and the Egger test only detect small-study effects. The other methods produce an adjusted estimate, each under its own assumption about the cause of asymmetry.

In 157 meta-analyses with binary outcomes, Schwarzer and colleagues found similar point estimates from the two adjustment methods, but systematically larger standard errors and p-values from trim and fill, and judged the Copas model preferable; Carpenter and colleagues found the Copas model gave a clear interpretation in about 80% of the same meta-analyses. A 2024 simulation of continuous outcomes, posted as a preprint, found Copas and PET-PEESE the least biased of five adjustment methods, noted that likelihood-based methods such as Copas can fail to converge, and concluded that PET-PEESE, being robust to heteroscedasticity, is a preferable technique. The Cochrane Handbook advises regression-based adjustment only with at least 10 studies.

Using Copas results in decision models and HTA

In an economic evaluation the adjusted estimate is best treated as a scenario input, not a new base case, showing how much of a relative effect could rest on missing evidence. In the Cochrane Handbook's reading, estimates that stay stable across assumed selection suggest the unadjusted estimate is unlikely to be driven by non-reporting, while large movements suggest it may well be. A scenario analysis can run the model at plausible degrees of selection. A probabilistic sensitivity analysis centred on the unadjusted estimate does not capture this, because bias moves the centre of the distribution rather than widening it, so parameter uncertainty and selection bias need separate treatment. Outside information can narrow the selection parameters. Mavridis and colleagues built priors from both external data and elicited expert opinion and extended the model to network meta-analysis, and Huang and colleagues used trial registries to estimate all of its parameters.

Assumptions and where the model misleads

The central assumption is that asymmetry comes from selective publication. The Cochrane Handbook calls this a major problem with selection models, since they assume that other causes of small-study effects, such as weaker design in small trials or genuinely different patients, are not operating, and may mislead if that is wrong.

The form of the selection equation matters as much as its parameters. A 2020 simulation of mean differences, posted as a preprint, found that the Copas method corrected adequately when publication followed its own selection model, performed poorly when selection depended directly on the standardised effect, and did not correct enough when between-study heterogeneity was present. Carpenter and colleagues also found that allowing for selection bias can shrink the standard error of the treatment estimate, as with trim and fill, against the intuition that missing evidence should widen uncertainty.

The most common misreading is to treat one grid point's adjusted estimate as the true effect. McShane and colleagues argued that selection methods should be used less to produce a single adjusted estimate and more for sensitivity analysis across forms and severities of publication bias.

Sources

• Almalik O. Adjusting for publication bias in meta-analysis with continuous outcomes: a comparative study. arXiv preprint arXiv:2410.06309. 2024.
• Almalik O, Zhan Z, van den Heuvel ER. Copas' method is sensitive to different mechanisms of publication bias. arXiv preprint arXiv:2007.15955. 2020.
• Carpenter J, Rücker G, Schwarzer G. copas: an R package for fitting the Copas selection model. The R Journal. 2009;1(2):31-36.
• Carpenter JR, Schwarzer G, Rücker G, Künstler R. Empirical evaluation showed that the Copas selection model provided a useful summary in 80% of meta-analyses. Journal of Clinical Epidemiology. 2009;62(6):624-631.
• Copas J, Shi JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics. 2000;1(3):247-262.
• Copas JB, Shi JQ. A sensitivity analysis for publication bias in systematic reviews. Statistical Methods in Medical Research. 2001;10(4):251-265.
• Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455-463.
• Huang A, Komukai S, Friede T, Hattori S. Using clinical trial registries to inform Copas selection model for publication bias in meta-analysis. Research Synthesis Methods. 2021;12(5):658-673. Preprint version: arXiv:2005.14396, 2020.
• Mavridis D, Sutton A, Cipriani A, Salanti G. A fully Bayesian application of the Copas selection model for publication bias extended to network meta-analysis. Statistics in Medicine. 2013;32(1):51-66.
• McShane BB, Böckenholt U, Hansen KT. Adjusting for publication bias in meta-analysis: an evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science. 2016;11(5):730-749.
• Page MJ, Higgins JPT, Sterne JAC. Chapter 13: Assessing risk of bias due to missing evidence in a meta-analysis. In: Higgins JPT, Thomas J, Chandler J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. Section 13.3.4.6.
• Schwarzer G, Carpenter J, Rücker G. Empirical evaluation suggests Copas selection model preferable to trim-and-fill method for selection bias in meta-analysis. Journal of Clinical Epidemiology. 2010;63(3):282-288.
• Schwarzer G, Carpenter JR, Rücker G. metasens: Statistical Methods for Sensitivity Analysis in Meta-Analysis. R package version 1.5-3. 2025. Reference manual, function copas.
• Stanley TD, Doucouliagos H. Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods. 2014;5(1):60-78.