Random Effects Model

Random Effects Model: Concept Architecture

Orientation and learning roadmap

The page first separates variation in underlying effects from sampling error in their estimates. It then shows the hierarchical model and its weighting in a health evidence synthesis, works through a small numerical example, and explains why a pooled mean, heterogeneity estimate, and prediction for another setting answer different questions.

Why effects can differ across studies

A treatment's effect need not be identical in every eligible study population or care setting. Differences in case mix, delivery, comparator, follow-up, and measurement can contribute to genuine variation, while each study's estimate also varies because it was calculated from a finite sample. A random effects synthesis models both layers rather than treating every observed difference as sampling noise.

Before fitting such a model, check whether the studies are similar enough in question and design for a distribution of underlying effects to be meaningful. Random effects do not repair biased studies, incompatible endpoints, or a poorly specified comparison. The distribution assumed for effects is a modeling choice to examine, not a guarantee that all included contexts are exchangeable.

In a common normal approximation for an effect measured on a consistent analysis scale:

$$y_i\mid\theta_i\sim N(\theta_i,s_i^2),\qquad \theta_i\sim N(\mu,\tau^2).$$

Here $y_i$ is study $i$'s observed effect estimate, $s_i^2$ its within-study sampling variance, $\theta_i$ its underlying effect, $\mu$ the mean of the modeled distribution of effects, and $\tau^2$ the between-study variance. For ratios such as odds ratios or hazard ratios, synthesis is commonly performed on an appropriate transformed scale, such as the log scale, then interpreted back on the original scale as needed.

How the pooled estimate is weighted

In a conventional inverse-variance random effects meta-analysis, the amount of between-study variation enters each study's weight in addition to its sampling variance. When $\tau^2$ is large relative to within-study variances, the weights become more similar than in a common-effect analysis. A smaller study may therefore receive a larger relative share of the pooled weight, which makes assessment of small-study bias especially relevant.

For a chosen estimate $\widehat{\tau}^2$ of between-study variance, an illustrative pooled estimator is:

$$w_i=\frac{1}{s_i^2+\widehat{\tau}^2},\qquad \widehat{\mu}=\frac{\sum_i w_i y_i}{\sum_i w_i}.$$

The formula alone does not specify how $\tau^2$ is estimated or how uncertainty in it is reflected in the interval for $\mu$. Methods such as restricted maximum likelihood and different interval procedures can produce different results, particularly with few studies. State the estimator and interval method rather than reporting only the phrase “random effects.”

A worked two-study calculation

The numbers below illustrate weighting only; two studies are too few to establish a reliable heterogeneity distribution. Suppose effects on a common additive scale are 0.20 and 0.50, with standard errors 0.10 and 0.20, and suppose a between-study variance of 0.01 is given for the exercise. The random effects weights are $1/(0.10^2+0.01)=50$ and $1/(0.20^2+0.01)=20$.

$$\widehat{\mu}=\frac{50(0.20)+20(0.50)}{50+20}=\frac{20}{70}\approx0.286.$$

For comparison, weights that omit the between-study variance are 100 and 25, giving $(100(0.20)+25(0.50))/125=0.26$. The change in the illustrative mean reflects different weighting, not evidence that the underlying effect in either study changed. A real analysis must estimate heterogeneity from the studies and account for substantial uncertainty in that estimate; the assumed 0.01 here should not be presented as an empirical result.

Mean effect, heterogeneity, and prediction

A confidence interval for $\mu$ expresses uncertainty about the mean of the modeled distribution of effects. A prediction interval instead asks about a plausible underlying effect in another comparable study or setting and incorporates between-study variation as well as uncertainty in the mean. It is generally wider and can reveal that a positive mean does not imply a benefit in every setting.

The between-study standard deviation $\tau$ has the same units as the effect measure on the analysis scale, while $\tau^2$ is its variance. The statistic $I^2$ describes a proportion of observed variability attributed to between-study heterogeneity under its definition, but does not replace the magnitude of $\tau^2$ or a clinically meaningful assessment of effect differences. Small numbers of studies can make all of these quantities uncertain.

For health technology assessment, the effect needed by a decision model may apply to a particular population or practice setting rather than to an abstract cross-study mean. Examine effect modifiers, transportability, baseline risks, and whether the pooled relative measure can be applied consistently to the target population. Propagate treatment-effect and heterogeneity uncertainty into decision analysis without assuming every study estimate is a separate independent model input.

How to choose, report, and challenge the model

The choice between a common-effect and random effects synthesis should be justified by the scientific question and plausible heterogeneity, not selected after seeing which result is significant. Explore likely sources of variation through prespecified subgroup analysis or meta-regression when enough data and appropriate covariates exist. A random effects model summarizes variation; it does not explain its causes.

1. Align the studies. Confirm comparators, populations, outcomes, time points, and effect scales before pooling.
2. Inspect individual results. Show study estimates, precision, risk of bias, and clinically relevant differences.
3. Specify the model. Report the effect distribution, heterogeneity estimator, weighting, interval method, and software.
4. Assess heterogeneity and bias. Examine $\tau^2$, prediction where defensible, small-study effects, and plausible effect modifiers.
5. Test decision sensitivity. Compare defensible synthesis choices and carry resulting uncertainty through the health economic model.

The term random effects model also appears in broader multilevel or mixed effects regression, where random intercepts or slopes describe clustering of patients within hospitals, clinicians, or other groups. The hierarchical idea is shared, but the unit of variation, likelihood, and target parameter may differ from a study-level meta-analysis. The structure here is the meta-analysis case; it should not be copied unchanged into every clustered-data problem.

Evidence used for this explanation

Cochrane guidance supports the distinction between within-study error and between-study heterogeneity, the weighting approach, and interpretation of prediction intervals. NICE evidence-synthesis guidance emphasizes justification of model choice and examination of heterogeneity for decision making.

• Cochrane. Handbook for Systematic Reviews of Interventions, Chapter 10: Analysing data and undertaking meta-analyses.
• NICE Decision Support Unit. Introduction to Evidence Synthesis for Decision Making.
• NICE Decision Support Unit. Heterogeneity: Subgroups, Meta-Regression, Bias and Bias-Adjustment.