I-Squared

I-Squared Statistic for Inconsistency Between Trials

I-squared, written $I^2$, is a single-number summary, built into Cochrane's RevMan software, of how far the trials pooled in a meta-analysis disagree beyond the scatter that sampling error would produce. It rescales Cochran's Q into a share between 0% and 100%, so a value of 60% says that most of the observed spread in trial results reflects real differences between the trials. In health technology assessment the pooled relative effect often becomes a parameter in an economic model, and disagreement between trials limits how well one pooled number can stand for the population being appraised. I-squared gives the proportion of the spread that is real, while tau-squared gives its size.

Cochran's Q rescaled as a percentage

Higgins and Thompson set out in 2002 to measure the impact of heterogeneity in a way that did not depend on the number of studies or the treatment effect metric. Their starting point was Cochran's Q, the weighted sum of squared differences between each study's estimate and the fixed-effect pooled estimate.

$$Q = \sum_{i=1}^{k} w_i \left(y_i - \hat{\mu}_{FE}\right)^2, \qquad w_i = \frac{1}{v_i}, \qquad \hat{\mu}_{FE} = \frac{\sum_{i=1}^{k} w_i y_i}{\sum_{i=1}^{k} w_i}$$

where $y_i$ is the effect estimate in study $i$ (for example a log odds ratio), $v_i$ is its within-study variance, $w_i$ is its inverse-variance weight, $\hat{\mu}_{FE}$ is the fixed-effect pooled estimate and $k$ is the number of studies. With no heterogeneity, Q follows a chi-square distribution with $k - 1$ degrees of freedom, so its expected value is $k - 1$. Higgins and Thompson then defined H and I-squared.

$$H^2 = \frac{Q}{k - 1}, \qquad I^2 = \max\left\{0, \frac{Q - (k - 1)}{Q}\right\} \times 100\% = \max\left\{0, 1 - \frac{1}{H^2}\right\} \times 100\%$$

where $H^2$ equals 1 when Q equals its expected value under homogeneity and $I^2$ is the excess of Q over its degrees of freedom as a percentage of Q. Negative values are set to zero. Von Hippel showed that when studies are homogeneous the probability that Q falls below its degrees of freedom is always above 50%, so a reported zero is common and is weak evidence that the trials agree.

The quantity behind the percentage

Von Hippel defines the target of I-squared as the fraction of variance in the study estimates that is due to heterogeneity rather than measurement error. In variance terms it is as follows.

$$\iota^2 = \frac{\tau^2}{\tau^2 + \sigma^2}$$

where $\iota^2$ is the true fraction, $\tau^2$ is the between-study variance of the true effects and $\sigma^2$ is the average within-study variance. The Cochrane Handbook gives the RevMan formula in the same form, with the estimated between-study variance divided by itself plus a typical within-study variance. With the DerSimonian and Laird estimator it equals the Q-based value. Von Hippel stresses that the fraction compares tau-squared with within-study variance, so it is not an absolute measure and can be large even when tau-squared is small.

Why I-squared displaced the heterogeneity test

Higgins and colleagues described the test as poor at detecting true heterogeneity when a meta-analysis includes few studies. In eight amantadine trials for preventing influenza the reductions in odds ranged between 16% and 93%, yet the test gave P = 0.09, while I-squared was 44%. In a Cochrane review of drop-outs on antidepressants, with more than 15,000 participants from 135 trials, the test was significant (P = 0.005) while I-squared was 26%, a little inconsistency that did not change the review's conclusion.

The Cochrane Handbook adds that, given low statistical power, a non-significant test is not evidence of no heterogeneity. Higgins and colleagues noted that I-squared can be compared between meta-analyses with different numbers of studies and different types of outcome data, and Higgins and Thompson recommended presenting H or I-squared in preference to the test.

Reading the value against thresholds

Higgins and colleagues tentatively assigned the adjectives low, moderate and high to values of 25%, 50% and 75%, while warning that a naive categorisation would not suit all circumstances. A given degree of heterogeneity also reads differently when all estimates share one direction of effect.

The Cochrane Handbook gives overlapping bands for meta-analyses of randomised trials. Values of 0% to 40% might not be important, 30% to 60% may represent moderate heterogeneity, 50% to 90% may represent substantial heterogeneity and 75% to 100% is considerable heterogeneity. Its mandatory conduct standard (MECIR C63) says that, given much uncertainty in I-squared and tau-squared when studies are few, simple thresholds should be avoided.

Uncertainty and bias when trials are few

Appraisals often rest on a handful of trials, which leaves I-squared imprecise. Ioannidis, Patsopoulos and Evangelou calculated 95% confidence intervals for I-squared in 1,011 Cochrane meta-analyses with a median of seven studies. Of those with I-squared at or below 25%, 83% had an upper limit of 50% or more, and of those at or above 50%, 67% had a lower limit at or below 25%. Where I-squared was 0%, the upper limit always exceeded 33% and was 50% or higher in 81% of cases. They concluded that 95% confidence intervals should always be given.

The point estimate is also biased. Von Hippel calculated that with seven studies and no true heterogeneity, I-squared overestimates heterogeneity by an average of 12 percentage points, while with seven studies and 80% true heterogeneity its expected value under a random-effects model is only 52%. He recommended confidence intervals to supplement or replace it.

A relative measure that rises with trial size

Because the denominator contains within-study variance, larger trials raise I-squared with the real spread unchanged. Rücker and colleagues simulated meta-analyses with increasing precision under a random-effects model with the true spread held constant and found that estimates of tau-squared stayed the same on average while I-squared rose to nearly 100%. They concluded that the clinical relevance of heterogeneity should be the yardstick for pooling and that tau-squared, rather than I-squared, is the measure for that purpose. Borenstein and colleagues add that an I-squared of 50% cannot show whether true effects range narrowly or widely, so the predicted range should be reported directly.

Worked example with six anticoagulant trials of three possible sizes

All numbers are illustrative. Six randomised trials compare a new anticoagulant with standard care for stroke within one year. Each reports a log odds ratio with within-study variance 0.04 (standard error 0.20), and the six log odds ratios are minus 0.70, minus 0.05, minus 0.40, 0.10, minus 0.55 and minus 0.20.

Step 1 (pooled estimate). Every weight is 1 divided by 0.04, or 25, so the total weight is 150 and the pooled log odds ratio is the simple mean, minus 1.80 divided by 6, or minus 0.30, an odds ratio of $e^{-0.30} = 0.741$.

Step 2 (Q). The deviations from minus 0.30 are minus 0.40, 0.25, minus 0.10, 0.40, minus 0.25 and 0.10. Multiplying each squared deviation by 25 gives the terms below.

$$Q = 4.0 + 1.5625 + 0.25 + 4.0 + 1.5625 + 0.25 = 11.625$$

where Q has 5 degrees of freedom, giving a chi-square P value of 0.040.

Step 3 (H and I-squared). $H^2 = 11.625/5 = 2.325$, so $H = 1.525$.

$$I^2 = \frac{11.625 - 5}{11.625} \times 100\% = 57.0\%$$

where 57.0% is the estimated share of variation due to heterogeneity.

Step 4 (tau-squared and the RevMan form). With equal weights the DerSimonian and Laird denominator is 150 minus 3,750/150, or 125, so $\hat{\tau}^2 = 6.625/125 = 0.053$ and $\hat{\tau} = 0.230$. The RevMan form gives 0.053 divided by 0.093, again 57.0%.

Step 5 (confidence interval). Von Hippel notes that when all studies share one standard error, $(1 - \iota^2)Q$ follows a chi-square distribution with $k - 1$ degrees of freedom under the random-effects model. The 2.5th and 97.5th percentiles on 5 degrees of freedom are 0.831 and 12.833.

$$1 - \frac{12.833}{11.625} = -0.104, \qquad 1 - \frac{0.831}{11.625} = 0.928$$

where the negative lower limit is truncated to zero, so the 95% interval for the heterogeneity fraction is 0% to 93%.

Step 6 (same spread, different trial sizes). Holding the between-study variance at 0.053, the heterogeneity fraction changes with trial size alone.

The same spread of true effects would be called low, moderate or high depending only on trial size.

Step 7 (what the model needs). If true log odds ratios are normal with mean minus 0.30 and standard deviation 0.230, 95% lie within $-0.30 \pm 1.96 \times 0.230$, or minus 0.751 to 0.151, odds ratios of 0.472 to 1.163. At an illustrative baseline stroke risk of 5% (odds 0.0526), these give risks of 3.75% at the mean, 2.42% and 5.77%, or 12.5 fewer strokes per 1,000 patients, 25.8 fewer and 7.7 more.

Interpretation. This range is identical in all three rows of step 6, so the stroke reduction to expect in a new setting depends on tau, and I-squared adds nothing to it. The range ignores uncertainty in the mean and in tau, which the prediction interval under between-study heterogeneity adds.

I-squared in HTA submissions and economic models

The NICE technology appraisal and highly specialised technologies manual (PMG36) asks in section 3.4.10 for statistical pooling to be accompanied by an assessment of heterogeneity, which a random-effects model can take into account only to some extent, with its degree and reasons explored as fully as possible. Section 3.4.15 asks for heterogeneity between pairwise comparisons to be reported in a network meta-analysis. Neither section names a particular statistic.

The NICE Decision Support Unit's technical support document 3 (TSD 3) describes I-squared as scale-free but dependent on the number and size of the included studies, which makes it hard to interpret in a typical meta-analysis. The DSU approach instead compares fixed and random-effects models by residual deviance and the deviance information criterion. The Cochrane Handbook states that the choice between a fixed-effect model and a random-effects model should never rest on a statistical test for heterogeneity.

I-squared keeps some uses. Higgins and colleagues showed that in trials of tamoxifen for early breast cancer, I-squared for recurrence was 50% across 55 trials but 13% once treatment duration was accounted for, the kind of finding subgroup analysis and meta-regression look for. Being scale-free, it can also compare effect measures, as when albumin trials in critically ill patients showed no inconsistency in risk ratios but more in risk differences. For the decision model, TSD 3 obtains the predictive variance of the effect in a new trial by adding the variance of the mean to the between-trials variance, so a probabilistic sensitivity analysis draws on tau, not I-squared.

Boundaries with Q, tau-squared and inconsistency

These neighbouring statistics answer different questions. Cochran's Q tests whether the spread is compatible with chance and grows with the number of studies. H and I-squared rescale Q so that the number of studies matters less, though both depend on precision. Tau-squared is on the scale of the effect measure, so it describes the size of the spread but, as Higgins and colleagues noted, depends on that scale.

Higgins called I-squared a measure of inconsistency, but in network meta-analysis inconsistency means disagreement between direct and indirect evidence, which PMG36 section 3.4.15 treats separately. Variation between patients within a trial belongs to heterogeneous treatment effect, and the review process of which I-squared is one part is described under heterogeneity assessment.

Common misreadings

Most errors come from treating a relative, imprecise statistic as absolute and exact. The commonest are listed below.

1. Reading 0% as homogeneity. With few trials the upper confidence limit is usually 50% or more.
2. Reading a high value as a wide spread of effects. Large, precise trials can give a high value when real differences are clinically trivial.
3. Applying fixed cut-offs. The Cochrane bands overlap, and its mandatory conduct standard says simple thresholds should be avoided.
4. Reporting the point estimate alone. A confidence interval, tau and a prediction interval carry the information a single percentage cannot.

Sources

• Borenstein M, Higgins JPT, Hedges LV, Rothstein HR. Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Research Synthesis Methods. 2017;8(1):5-18. doi:10.1002/jrsm.1230
• Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA (editors). Chapter 10: Analysing data and undertaking meta-analyses (last updated November 2024). In: Higgins JPT, Thomas J, Chandler J, et al (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. Sections 10.10.2 and 10.10.4.
• Dias S, Sutton AJ, Welton NJ, Ades AE. NICE DSU Technical Support Document 3: Heterogeneity: subgroups, meta-regression, bias and bias-adjustment. Decision Support Unit; September 2011, last updated April 2012.
• Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21(11):1539-1558. doi:10.1002/sim.1186
• Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560. doi:10.1136/bmj.327.7414.557
• Ioannidis JPA, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ. 2007;335(7626):914-916. doi:10.1136/bmj.39343.408449.80
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 3.4.10 and 3.4.15.
• Rücker G, Schwarzer G, Carpenter JR, Schumacher M. Undue reliance on I2 in assessing heterogeneity may mislead. BMC Medical Research Methodology. 2008;8:79. doi:10.1186/1471-2288-8-79
• von Hippel PT. The heterogeneity statistic I2 can be biased in small meta-analyses. BMC Medical Research Methodology. 2015;15:35. doi:10.1186/s12874-015-0024-z