Cohen's d

Cohen's d, Hedges' Correction and Re-expression for Economic Models

Cohen's d puts the effect of a treatment on a continuous outcome into standard deviation units, so that trials which measured the same outcome on different instruments can be compared and pooled. In health economics it appears in meta-analyses of patient-reported outcomes, in sample size calculations for trials that also collect economic data, and in judgements about whether an effect is large enough to matter to patients. The basic formula has a small-sample correction that gives Hedges' g, and Cohen's conventions for small, medium and large effects came with his own warning about their use. A standardised effect cannot enter a cost-effectiveness model as it stands, so it has to be re-expressed first, as the illustrative example below does in moving from trial summary data to a QALY gain.

From a raw difference to standard deviation units

A raw difference in means is tied to the units of the instrument that produced it, so six points on a 0 to 100 fatigue scale has no direct counterpart on a 1 to 7 scale. Dividing the difference by a standard deviation removes those units. Cohen defined the index for two populations whose spread is assumed equal, as follows.

$$d = \frac{m_A - m_B}{\sigma}$$

where $d$ is the effect size index, $m_A$ and $m_B$ are the population means in the original units, and $\sigma$ is the common within-population standard deviation. In a study the population values are replaced by sample estimates, and the usual standardiser is the pooled standard deviation of the two groups, calculated as follows.

$$s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}}$$

where $s_p$ is the pooled standard deviation, $n_1$ and $n_2$ are the group sizes, and $s_1$ and $s_2$ are the group standard deviations. The sample estimate is then $d = (\bar{x}_1 - \bar{x}_2)/s_p$, where $\bar{x}_1$ and $\bar{x}_2$ are the observed group means. The sign carries the direction of the effect, so a review has to align instruments first. The Cochrane Handbook notes that the method does not correct for scales that run in opposite directions, and the means from one set of studies must be multiplied by minus one before standardising.

Hedges' g and the small-sample correction

The sample estimate of d tends to overstate the population effect in small samples. In the Cochrane and Lakens usage the corrected estimate is known as Hedges' g (Hedges and Olkin used g for the uncorrected estimate), and it multiplies d by a factor that approaches one as the sample grows. The corrected estimate is calculated as follows.

$$g = d \left(1 - \frac{3}{4N - 9}\right)$$

where $g$ is the corrected estimate, $d$ is the uncorrected sample estimate and $N = n_1 + n_2$ is the total sample size. The difference between the two is small once samples exceed about 20, but the corrected value is no harder to compute and is the one used in Cochrane reviews. The Handbook states that its standardised mean difference is Hedges' adjusted g, with a pooled standard deviation in the denominator. The Review Manager software gives its standard error as follows.

$$SE(g) = \sqrt{\frac{N}{n_1 n_2} + \frac{g^2}{2(N - 3.94)}}$$

where $SE(g)$ is the standard error of the corrected estimate and the other terms are as defined above. Other members of the family change the standardiser. Glass's delta divides by the comparator group's standard deviation alone, on the reasoning that a treatment which changes between-person variation should not alter the yardstick, and within-person designs have their own versions based on the standard deviation of differences or an average of the two measurement occasions. A report should therefore say which version was calculated.

Pooling continuous outcomes across trials

The choice between a mean difference and a standardised mean difference in a meta-analysis turns on whether the trials used the same instrument. When they did, the mean difference keeps the natural units; when they measured the same construct with different instruments, Chapter 10 of the Cochrane Handbook treats the standardised mean difference as the usual choice. Each study's g and standard error then enter an inverse-variance analysis, under a fixed-effect model or a random-effects model.

The approach rests on one assumption that is easy to overlook. Differences in standard deviation between studies are taken to reflect differences in measurement scales, not real differences in how variable the populations are. The Handbook gives the example of early explanatory trials combined with later pragmatic trials, where the pragmatic trials may enrol a wider range of participants, have larger standard deviations, and therefore produce smaller standardised effects for the same true difference in means. Two further rules follow from the same logic. Post-intervention values and change-from-baseline scores should not in principle be combined in one standardised analysis, because the two standard deviations describe different things. When a missing standard deviation is imputed, choosing a higher value biases a standardised result towards no effect.

Reading the size of an effect

Interpretation is where Cohen's d is easiest to misread. In the 1988 second edition of his power analysis text, Cohen set out d = 0.2, 0.5 and 0.8 as conventional values for small, medium and large effects, conventions he later said had been fixed since the 1977 edition. He described a medium effect as one large enough to be visible to the naked eye. In the same passage he stressed that the labels are relative to the field and to the specific content and methods of a study, accepted that offering fixed values carried a risk, and recommended them for use only when no better basis for estimating the effect is available. In a later paper he added that the definitions were made subjectively. The Cochrane Handbook repeats the 0.2, 0.5 and 0.8 rule as one guiding option, and notes that some methodologists regard generic labels as problematic because the importance of a finding to patients depends on context.

For patient-reported outcomes, the more useful yardstick is usually the minimal clinically important difference (the Cochrane Handbook's minimal important difference), the smallest change in score that patients perceive as important. The Cochrane Handbook asks review authors to state it, and describes expressing results in minimal important difference units, dividing each study's mean difference by that value instead of by a standard deviation, as an option where the minimal important difference is well established. A systematic review of 38 studies by Norman and colleagues found that most published estimates of the minimal important difference for health-related quality of life instruments were close to half a standard deviation, with a mean of 0.495 across the 62 estimates. Fractions of a standard deviation of this kind are the basis of a distribution-based MID, but the finding describes past estimates and does not replace anchor-based evidence for a particular instrument and population.

The same conventions enter trial planning. Cohen's power primer shows that detecting d = 0.5 with a two-group comparison at a significance level of 0.05 and statistical power of 0.80 needs 64 participants in each group. Power for a standardised clinical outcome says nothing about the precision of a cost or QALY difference, which has its own variance and would need its own sample size calculation if the trial were designed to estimate a cost or QALY difference with stated precision.

Why a standardised effect cannot enter a cost-effectiveness model directly

A decision model needs effects in units to which costs and preference weights can be attached, in the form of probabilities of events, time in health states, and utility values for those states. A standard deviation unit has neither a cost nor a preference weight, and the same d corresponds to different absolute changes in populations with different spread. Even a d calculated on a utility instrument has to be multiplied back by a standard deviation before it can change a QALY estimate. The Cochrane Handbook describes the main routes for re-expressing a standardised result, set out below.

1. Back to a familiar instrument. The standardised mean difference is multiplied by a standard deviation for one instrument, for example a weighted average from the included studies or a value from a representative observational study, giving a difference in that instrument's units.
2. To an odds ratio and a responder probability. Assuming a logistic distribution with equal spread in both groups, the log odds ratio is approximately 1.81 times the standardised mean difference. Combined with a comparator proportion of responders, this gives a probability that a model can use, though the Handbook warns that the assumption will not hold exactly and the result is an approximation.

Neither route produces utilities. For a cost-utility analysis the natural-unit or responder result still has to be linked to health state utility values. The NICE manual (PMG36) prefers the EQ-5D for adults, and where EQ-5D data are not available it allows them to be estimated with a mapping function from other quality of life measures. The manual treats this as a departure from the reference case and asks for sensitivity analyses on the mapping choice.

Worked example from trial summary data to an illustrative QALY gain

All figures in this example are illustrative. A trial compares a new fatigue programme with usual care, 30 patients in each arm, on a fatigue instrument where higher scores are better. The intervention group mean is 58 with a standard deviation of 12, and the usual care mean is 52 with a standard deviation of 10. Here g stands in for a pooled standardised mean difference from a meta-analysis of trials that used different instruments; with a single trial on one instrument, the mean difference would go straight into the model.

1. Pool the standard deviations.

$$s_p = \sqrt{\frac{29 \times 144 + 29 \times 100}{58}} = \sqrt{\frac{7076}{58}} = \sqrt{122} = 11.05$$

where $s_p$ is the pooled standard deviation in instrument points.

2. Calculate d and correct it.

$$d = \frac{58 - 52}{11.05} = 0.543$$

where 58 and 52 are the group means and 11.05 is the pooled standard deviation from step 1. Applying the correction gives the following.

$$g = 0.543 \times \left(1 - \frac{3}{231}\right) = 0.543 \times 0.987 = 0.536$$

where $g$ is Hedges' g and 231 is four times the total sample of 60, minus 9. With 60 participants the correction changes the estimate by little more than 1%.

3. Attach uncertainty.

$$SE(g) = \sqrt{\frac{60}{900} + \frac{0.536^2}{112.12}} = \sqrt{0.0667 + 0.0026} = 0.263$$

where 112.12 is twice the total sample of 60 less 3.94. The 95% confidence interval is 0.536 ± 1.96 × 0.263, or 0.021 to 1.051. The point estimate sits near Cohen's medium label, but the interval runs from almost no effect to a large one.

4. Re-express on a familiar instrument. Rounding g to 0.54 and taking a standard deviation of 20 points for a widely used 0 to 100 fatigue instrument, the effect is 0.54 × 20 = 10.8 points. Against a minimal important difference of 10 points on that instrument, the point estimate just exceeds the threshold, while the lower confidence limit corresponds to 0.021 × 20 = 0.42 points, far below it.

5. Convert to a responder probability.

$$\ln OR = 1.814 \times 0.54 = 0.980$$

where $\ln OR$ is the log odds ratio for responding. The odds ratio is $e^{0.980}$, about 2.664. If 30% of usual care patients respond, their odds are 0.30 / 0.70 = 0.429, the intervention odds are 0.429 × 2.664 = 1.143, and the intervention response probability is 1.143 / 2.143 = 0.533.

6. Attach utilities. Suppose EQ-5D data give utility values of 0.78 for responders and 0.70 for non-responders. The extra proportion responding is 0.233 (0.533 minus 0.30), so the expected utility gain is 0.233 × 0.08 = 0.0186. Assuming response status and the utility difference hold for a full year, this is about 0.019 QALYs. Carrying the confidence limits for g through the same steps gives a range of about 0.0006 to 0.035 QALYs.

A d of about 0.5 becomes a QALY gain only through external inputs, namely a standard deviation for the target instrument, a comparator response rate and utility values. Each carries its own uncertainty, and a probabilistic analysis of the QALY gain would need to sample g with its standard error as well as those inputs.

Cohen's d alongside neighbouring measures

Standardised mean difference is the family name used here, with Cohen's d as its basic member. Cochrane's own "standardised mean difference" is Hedges' g. Measures that standardise by something other than the spread of scores at one time point answer different questions.

Where Cohen's d is misread

A common misreading is to treat 0.2, 0.5 and 0.8 as statements of clinical or economic importance. Cohen offered them as a fallback for power analysis when nothing better was known, and a small standardised effect on an outcome that drives costs or survival can matter more than a large one on a symptom score. A second error is to compare standardised effects across populations with different spread, as if a larger d always meant a larger benefit; a narrow, explanatory trial population inflates d relative to a broad pragmatic one with the same mean difference.

Reporting errors include change scores and final values mixed in one standardised analysis, d and g used interchangeably without saying which was calculated, and a standardised response mean presented as if it were comparable with d. In economic work one error is to carry a pooled standardised effect into a model as if it were a utility or probability, when it has to be re-expressed with the standard deviation, response threshold and mapping choices stated and tested in sensitivity analysis.

Sources

• Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates; 1988. Section 2.2.
• Cohen J. A power primer. Psychological Bulletin. 1992;112(1):155-159.
• Deeks JJ, Higgins JPT, on behalf of the Cochrane Statistical Methods Group. Statistical algorithms in Review Manager. Cochrane; May 2022. https://documentation.cochrane.org/revman-kb/files/210600101/210600103/1/1654774257333/Statistical_methods_in_revman.pdf
• Higgins JPT, Li T, Deeks JJ (editors). Chapter 6: Choosing effect measures and computing estimates of effect. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. Sections 6.5.1.2 and 6.5.2.7.
• Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA (editors). Chapter 10: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. Sections 10.5 and 10.6.
• Schünemann HJ, Vist GE, Higgins JPT, Santesso N, Deeks JJ, Glasziou P, Akl EA, Guyatt GH. Chapter 15: Interpreting results and drawing conclusions. In: Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. Section 15.5.
• Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology. 2013;4:863.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. Section 4.3.
• Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Medical Care. 2003;41(5):582-592.