Concept Architecture
Concept
Theoretically, Between-Study Heterogeneity refers to the variation in true effect sizes across studies included in a meta-analysis beyond that expected from sampling error alone. It reflects genuine differences in study populations, interventions, comparators, outcome definitions, methodologies or other characteristics that influence treatment effects. The concept underpins random-effects meta-analysis, where true effects are assumed to vary across studies rather than being identical.
Mathematically, between-study heterogeneity is represented by the between-study variance parameter, typically denoted ��, within a random-effects model. Heterogeneity is estimated using recognised estimators such as the DerSimonian?Laird, restricted maximum likelihood (REML) or Paule?Mandel methods. Additional statistics, including Cochran's Q, I� and H�, quantify the magnitude and proportion of variability attributable to heterogeneity.
In practice, between-study heterogeneity is assessed routinely during systematic reviews and meta-analyses before selecting an appropriate pooling model. High heterogeneity may justify the use of random-effects models, subgroup analyses or meta-regression to investigate potential sources of variation. In health economics, estimates of between-study heterogeneity directly influence pooled treatment effects used within evidence synthesis and decision-analytic models.
Purpose
Used to quantify genuine variation between studies, inform the selection of meta-analytic models, assess the consistency of evidence and identify whether further investigation of effect modifiers is required.
Mathematical Formulae
Primary Formula
Random-effects model:
?? ~ N(?, ��)
where:
- ?? = true effect for study i
- ? = overall mean effect
- �� = between-study variance
Supporting Formulae
Cochran's heterogeneity statistic:
Q = ?w?(?? ? ??)�
I� statistic:
I� = max{0, ((Q ? df) / Q)} ? 100%
Between-study standard deviation:
� = ���
Related Mathematical Methods
- Random-Effects Meta-Analysis
- Restricted Maximum Likelihood (REML)
- DerSimonian?Laird Estimation
- Paule?Mandel Estimation
- Cochran's Q Test
- I� Statistic
- Meta-Regression
- Bayesian Meta-Analysis
Example
A meta-analysis of 15 randomised controlled trials evaluating a diabetes intervention reports �� = 0.045, Q = 31.8 with 14 degrees of freedom and I� = 56%. These results indicate moderate between-study heterogeneity, supporting the use of a random-effects model and prompting subgroup analyses to investigate differences in patient populations and treatment protocols.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| SUMPRODUCT | =SUMPRODUCT(W2:W16,(E2:E16?F1)^2) | Calculate Cochran's Q statistic |
| SQRT | =SQRT(A2) | Calculate � from �� |
| MAX | =MAX(0,((B2?C2)/B2)*100) | Calculate I� (%) |
| CHISQ.DIST.RT | =CHISQ.DIST.RT(B2,C2) | Calculate the p-value for Cochran's Q test |
VBA (Optional)
Automate estimation of heterogeneity statistics across multiple meta-analyses and generate summary reports comparing fixed-effect and random-effects models.
Sources
- Higgins JPT, Thompson SG. Quantifying Heterogeneity in a Meta-Analysis. Statistics in Medicine. 2002.
- DerSimonian R, Laird N. Meta-Analysis in Clinical Trials. Controlled Clinical Trials. 1986.
- Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions.
- Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis.
- NICE. Health Technology Evaluation Manual.
- ISPOR Good Practice Reports.
Related Concepts (2)
Library
Publications
3
Introduction to Meta-Analysis — Borenstein, Hedges, Higgins & Rothstein, 2nd Edition ed., 2021 (John Wiley & Sons)
A clear, applied introduction to meta-analysis — computing effect sizes, fixed- and random-effects models, heterogeneity, subgroup analysis, meta-regression, and publication bias — written for readers across disciplines.
BookView source →NICE DSU Technical Support Document 4: Inconsistency in Networks of Evidence Based on Randomised Controlled Trials — Dias, Welton, Sutton, Caldwell, Lu & Ades, TSD 4 ed., 2011 (NICE Decision Support Unit (University of Sheffield))
Guidance on assessing and handling inconsistency — conflict between direct and indirect evidence — in network meta-analysis, a key validity check for mixed treatment comparisons.
NICE DSU Technical Support Document 20: Multivariate Meta-Analysis of Summary Data for Combining Treatment Effects on Correlated Outcomes and Evaluating Surrogate Endpoints — Bujkiewicz, Achana, Papanikos, Riley & Abrams, TSD 20 ed., 2019 (NICE Decision Support Unit (University of Sheffield))
Guidance on multivariate and network meta-analysis of correlated outcomes and on the evaluation of surrogate endpoints, extending standard synthesis methods to jointly model multiple related treatment effects.
Frequently Asked Questions (6)
What is between-study heterogeneity?
The genuine variation in true treatment effects across the studies in a meta-analysis, beyond what chance alone would explain.
Source: Higgins et al. 2003
What does between-study heterogeneity say about the studies in a meta-analysis?
Between-study heterogeneity is real variation in the true effect from one study to another, over and above the scatter that chance would produce. Its presence says the studies are not all estimating a single common effect: differences in their patients, doses, settings, or outcome definitions make the treatment genuinely work to different degrees. Recognising this changes how results are combined and warns against reporting one pooled figure as though it applied everywhere. Real differences between studies are what it signals. Borenstein and colleagues (2009) describe this.
Source: Borenstein et al. 2009
How is between-study heterogeneity measured?
Between-study heterogeneity is measured using statistics that quantify variation beyond chance, including the I-squared statistic, which expresses the proportion of total variation due to differences between studies; tau-squared, which estimates the between-study variance; and the Cochran Q test, which tests for the presence of heterogeneity. Visual inspection of forest plots also indicates it. Higher values reflect greater heterogeneity. So between-study heterogeneity is quantified by these statistical measures, which together convey how much the true effects vary across studies, informing whether and how the studies should be combined in meta-analysis.
Source: Higgins et al. 2003
Why does between-study heterogeneity matter?
Between-study heterogeneity matters because substantial variation in true effects across studies affects whether it is appropriate to combine them into a single pooled estimate and how that estimate should be interpreted: high heterogeneity suggests the studies estimate different effects, so a single figure may be misleading, and it lowers confidence in the pooled result. It also prompts investigation of the reasons for the variation, which can be informative. So between-study heterogeneity is important for deciding how to synthesise studies, how to interpret the pooled estimate, and whether to explore the sources of the variation.
Source: Higgins et al. 2003
How is between-study heterogeneity handled?
Between-study heterogeneity is handled by choosing an appropriate model, such as a random-effects model that allows for variation in true effects across studies rather than assuming a single common effect; by investigating its sources through subgroup analyses or meta-regression relating the effect to study characteristics; and, where heterogeneity is very high, by questioning whether pooling is appropriate. Reporting the degree of heterogeneity conveys the consistency of the evidence. So between-study heterogeneity is addressed by modelling it, exploring its causes, and interpreting the pooled estimate accordingly, ensuring the synthesis reflects the variation among the studies.
Source: DerSimonian & Laird 1986
What causes between-study heterogeneity?
Between-study heterogeneity can arise from genuine differences in the effect across studies, such as variation in populations, interventions, doses, comparators, or settings, representing true clinical or methodological diversity; and from differences in study methods, such as design, outcome measurement, or risk of bias. Distinguishing these matters, since genuine variation informs how the effect differs, while methodological differences may explain artefactual heterogeneity. So between-study heterogeneity results from clinical and methodological diversity among the studies, and investigating its causes, for example through meta-regression, helps interpret whether it reflects real variation in the effect or differences in how the studies were conducted.
Source: Higgins et al. 2003
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 1 Dec 2025
Content version: 1.0.0
Canonical Identity
- Term code
- HE-ES-ESM-004
Stable URI · Machine-readable · Resolvable · CC BY 4.0