Bayesian Meta-Analysis

Bayesian Meta-Analysis for Decision Models

In health technology assessment, a Bayesian meta-analysis is typically a random-effects model, usually fitted by MCMC simulation, and its output can go straight into a probabilistic cost-effectiveness model. The method matters most where the evidence is thinnest. With only a few trials, the between-study variance is poorly estimated, and the prior placed on it can change both the pooled effect and the range of effects a model should allow for. An illustrative example below analyses four mortality trials under two heterogeneity priors to show how far that choice reaches. Frequentist pooling, including the DerSimonian and Laird estimator, is covered on the aggregate data meta-analysis page.

The hierarchical random-effects model

A Bayesian random-effects meta-analysis is a two-level hierarchical model. Each trial estimates its own true effect, and the true effects are treated as draws from a common distribution. Unknown quantities, including the mean effect and the between-study standard deviation, are given prior distributions, and Bayes theorem combines these with the likelihood to give a joint posterior distribution. For trials reported as a summary effect and its variance, the model takes the following form.

$$y_i \mid \theta_i \sim N\left(\theta_i, s_i^2\right), \qquad \theta_i \mid \mu, \tau \sim N\left(\mu, \tau^2\right), \qquad i = 1, \ldots, k$$

where $y_i$ is the observed effect in trial $i$ (for example a log odds ratio), $s_i^2$ is its within-trial variance, treated as known, $\theta_i$ is the true effect in that trial, $\mu$ is the mean of the distribution of true effects, $\tau$ is the between-study standard deviation and $k$ is the number of trials. Integrating out the trial-level effects gives the joint posterior, shown below.

$$p(\mu, \tau \mid y) \propto p(\mu)\, p(\tau) \prod_{i=1}^{k} N\left(y_i \mid \mu, s_i^2 + \tau^2\right)$$

where $p(\mu)$ and $p(\tau)$ are the priors and the product is the likelihood of the observed effects. NICE DSU Technical Support Document 2 (TSD 2) bases this structure on exchangeability. The trial labels are assumed irrelevant, so the trial-specific effects come from one distribution, and the fixed effect model is the special case with the variance set to zero. TSD 2 works mainly with arm-level likelihoods, such as binomial counts, and also gives the summary-effect version above.

Why decision modellers fit it by simulation

TSD 2 takes a Bayesian approach and supplies WinBUGS code that fits these models by Markov chain Monte Carlo, and it names an advantage for decision making. Sampling from the posterior serves at once as statistical inference and as the Monte Carlo input to probabilistic sensitivity analysis, so the draws that summarise the pooled effect can pass straight into the cost-effectiveness model.

TSD 2 does not make the Bayesian route compulsory. Where all trials are two-arm, it sees no reason why frequentist methods should not be used, provided statistically sound estimators are used and parameter uncertainty, including correlations, is propagated through the decision model.

Priors for the between-study standard deviation

The data usually dominate the vague prior given to the mean effect, but with few trials the data say little about between-study variation, so the prior on $\tau$ shapes its posterior. Turner and colleagues cite a descriptive analysis in which 75% of meta-analyses in Cochrane reviews contained five or fewer studies.

Vague priors and their drawbacks

TSD 2 recommends vague priors, such as $N(0, 100^2)$, for trial baselines and relative effects, and says informative priors for relative effects would need special justification. For the between-trial standard deviation on the log odds scale, it describes a uniform prior such as $\tau \sim U(0, 2)$ as usual practice, warns that what counts as vague depends on the outcome scale, and advises checking that the posterior differs enough from the prior to show that the data have updated it.

The same document lists major disadvantages of routinely using vague priors. Without many large trials, the posterior for $\tau$ is poorly identified and can include implausibly high values. In its discussion, TSD 2 concludes that the vague priors generally recommended produce posteriors biased upwards, especially when the true variance is low and trials are few or small, but this is also a problem when using frequentist estimators. TSD 3 repeats the warning that this posterior is likely to be extremely sensitive to the prior, and points to informative priors based on expert opinion or meta-epidemiological data.

Empirical priors from the Cochrane Database

Empirical informative priors have been derived by fitting hierarchical models to thousands of Cochrane meta-analyses and taking the predictive distribution of the between-study variance in a new meta-analysis. Each set applies to a stated effect scale, so a prior for log odds ratios is not automatically suitable for another measure.

The 2012 paper found heterogeneity variances for all-cause mortality to be on average 17% of those for other outcomes. The 2015 paper defines settings by outcome type and type of comparison, gives a log-normal prior of $LN(-2.56, 1.74^2)$ for a general healthcare setting, and provides R code using numerical integration and importance sampling that gives results equivalent to MCMC.

What NICE asks for

The NICE manual (PMG36) states that NICE prefers the network meta-analysis methods of the DSU evidence synthesis series. Sections 3.4.16 to 3.4.19, in its network meta-analysis guidance, then deal with heterogeneity priors. External information should be used to help estimate between-study heterogeneity, and in networks with few studies, informative priors for the heterogeneity parameter may be preferable. Distributions tailored to particular outcomes and disease areas are recommended, the source of the prior should be noted and justified, and a sensitivity analysis of different candidate priors should be presented. Informative priors for relative effectiveness are not recommended except in very specific circumstances, such as very sparse adverse event data.

The predictive distribution as the model input

A random-effects analysis produces two distributions that are easily confused. The posterior for $\mu$ describes uncertainty about the average effect across trial settings, while the predictive distribution describes the effect expected in a new setting, such as the population of an appraisal. In MCMC the predictive distribution is obtained by drawing one further effect at each iteration, as follows.

$$\theta_{new} \mid \mu, \tau \sim N\left(\mu, \tau^2\right)$$

where $\theta_{new}$ is the true effect in a new trial or population, and $\mu$ and $\tau$ take their sampled values at that iteration. TSD 3 argues that this predictive distribution may in many cases be more relevant to decision making than the distribution of the mean, and Higgins, Thompson and Spiegelhalter describe predictive distributions as potentially the most relevant and complete inference from a random-effects meta-analysis.

On its linear scale the predictive distribution has the same mean as the distribution of the mean effect, so the choice mainly widens the spread of the effect in the cost-effectiveness analysis, and with it decision uncertainty and value of information. TSD 3 cautions that the predictive variance should contain only true variation between populations and that observed heterogeneity is likely to overestimate it. An implausibly large $\tau$ from a vague prior therefore passes straight into the model as excess uncertainty.

Worked example with four mortality trials and two heterogeneity priors

The figures in this example are illustrative. Four placebo-controlled trials of a drug report all-cause mortality as log odds ratios. Trial 1 gives $-0.45$ (standard error 0.20), trial 2 gives $-0.10$ (0.15), trial 3 gives $-0.60$ (0.30) and trial 4 gives 0.05 (0.25). The mean effect receives the vague prior $N(0, 100^2)$ in both analyses. Analysis A gives $\tau$ the vague prior $U(0, 2)$; analysis B uses the prior published by Turner and colleagues in 2015 for all-cause mortality in pharmacological against placebo or control comparisons, given below.

$$\tau^2 \sim LN\left(-3.95, 1.34^2\right)$$

where $LN(m, s^2)$ is a log-normal distribution whose logarithm has mean $m$ and standard deviation $s$. Its median is $e^{-3.95} = 0.0193$ and its central 95% range runs from $e^{-6.576} = 0.0014$ to $e^{-1.324} = 0.266$.

1. The pooled mean at a fixed heterogeneity. Conditional on $\tau$, the posterior mean of $\mu$ is, to close approximation given its diffuse prior, an inverse-variance average with weights $w_i = 1/(s_i^2 + \tau^2)$, the same arithmetic shown on the aggregate data meta-analysis page. At the prior median, $\tau^2 = 0.0193$, this gives a conditional posterior mean of $-0.239$ with standard deviation 0.127.

2. Shrinkage of a trial estimate. The hierarchical model also gives each trial a revised estimate that borrows strength from the others, as follows.

$$E\left(\theta_i \mid \mu, \tau, y\right) = B_i \mu + \left(1 - B_i\right) y_i, \qquad B_i = \frac{s_i^2}{s_i^2 + \tau^2}$$

where $B_i$ is the shrinkage factor for trial $i$. For trial 3, the smallest trial with the largest effect, $s_3^2 = 0.09$ and $B_3 = 0.09/0.1093 = 0.823$, so with $\mu = -0.239$ its estimate moves from $-0.60$ to $B_3 \mu + (1 - B_3) y_3 = 0.823 \times (-0.239) + 0.177 \times (-0.60) = -0.303$.

3. Averaging over the heterogeneity prior. The full posterior averages step 1 over the posterior for $\tau$, which was computed by numerical integration on a fine grid and checked by importance resampling. The frequentist DerSimonian and Laird estimate for the same data is $\tau^2 = 0.026$, for comparison.

4. Feeding a decision model. An external source is assumed to give an annual mortality risk of 0.10 on placebo. The odds ratio is applied on the odds scale.

$$p_1 = \frac{o_0 \times OR}{1 + o_0 \times OR}, \qquad o_0 = \frac{0.10}{0.90} = 0.1111$$

where $o_0$ is the baseline odds of death, $OR$ is the odds ratio and $p_1$ is the annual mortality risk on treatment. The posterior median odds ratio under analysis B gives $p_1 = 0.0805$. Sampling from the predictive distribution, the 95% range of $p_1$ is 0.052 to 0.121 under analysis B and 0.024 to 0.224 under analysis A.

Interpretation. The central estimates barely move, although the probability that $\mu$ is below zero rises from 0.890 under the vague prior to 0.961 under the informative prior. The vague prior allows between-study standard deviations above 1 on the log odds scale, which four trials cannot rule out but which lie far outside the 95% range of the Turner prior for this setting. Under the vague prior the predictive range of mortality risk runs from a large benefit to more than doubled mortality, which would widen the probabilistic results and could inflate the estimated value of further research. Because the result depends on the prior, PMG36 expects the choice to be justified and tested against other candidate priors.

Relation to network meta-analysis

TSD 2 presents pairwise meta-analysis and indirect comparison as special cases of network meta-analysis, and its core models handle all three without distinction. Extending the model to a network adds exchangeable trial-specific effects for each comparison, consistency relations linking the mean effects and, for simplicity, a common between-trial variance. The model, priors and predictive reasoning on this page therefore carry over to a Bayesian network, and PMG36 addresses its advice on informative heterogeneity priors to networks with few included studies.

Where Bayesian synthesis is misread

One misreading is to treat a vague prior on $\tau$ as neutral. With few trials it is informative in practice, tending to inflate heterogeneity, and a report that does not state and test this prior leaves its main source of uncertainty hidden. The opposite error is to borrow an empirical prior from the wrong scale or setting.

A second error is to report the posterior for $\mu$ as if it were the effect expected in the decision population, when the predictive distribution may fit the decision question better. A third is to read a 95% credible interval as a confidence interval. It is a probability statement about the parameter given the data and priors, and only as sound as those priors. Higgins, Thompson and Spiegelhalter note that the Bayesian approach allows naturally for full uncertainty, especially for prediction, but is sensitive to prior judgements, and TSD 2 calls it by no means a panacea. Finally, as with frequentist meta-analysis, exchangeability is an assumption, and no prior can repair trials that differ in effect modifiers or risk of bias.

Sources

• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. Decision Support Unit; 2011, last updated September 2016. Sections 2, 3.5, 5, 6.2, 7.1 and 9.
• Dias S, Sutton AJ, Ades AE, Welton NJ. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Medical Decision Making. 2013;33(5):607-617. doi:10.1177/0272989X12458724
• Dias S, Sutton AJ, Welton NJ, Ades AE. NICE DSU Technical Support Document 3: Heterogeneity: subgroups, meta-regression, bias and bias-adjustment. Decision Support Unit; 2011, last updated April 2012. Executive summary and section 2.
• Higgins JPT, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society: Series A. 2009;172(1):137-159.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. Sections 3.4.11 and 3.4.16 to 3.4.19. https://www.nice.org.uk/process/pmg36/chapter/evidence-2
• Rhodes KM, Turner RM, Higgins JPT. Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data. Journal of Clinical Epidemiology. 2015;68(1):52-60.
• Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Chichester: Wiley; 2004.
• Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JPT. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. International Journal of Epidemiology. 2012;41(3):818-827.
• Turner RM, Jackson D, Wei Y, Thompson SG, Higgins JPT. Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis. Statistics in Medicine. 2015;34(6):984-998.