Hedges' g

Hedges' g Formula, Small-Sample Correction and Use in Meta-Analysis

Hedges' g puts the results of studies that measured the same outcome on different scales into one common unit, so that a meta-analysis can pool them. Health technology assessment meets it whenever the evidence for a treatment comes as symptom scores, function scales or patient-reported outcomes that differ from trial to trial. The statistic is the uncorrected standardised mean difference multiplied by a factor that removes the upward bias found in small studies, and it is the version of the standardised mean difference that Cochrane reviews report by default. Its formula, variance and interpretation matter for economic work because a pooled g cannot enter a cost-effectiveness model until it has been turned back into natural units or another effect measure.

Name, spelling and where the measure comes from

Larry Hedges set out the distribution theory for standardised effect sizes in 1981, showing that the sample standardised mean difference overestimates the population value in small samples and deriving a correction. The measure is written Hedges' g, with a lower-case g, and is also searched as Hedges g. The Cochrane Handbook calls the measure Hedges' adjusted g and notes that the standardised mean difference in Cochrane reviews is this statistic, while Cohen's d is the same ratio without the correction.

The Hedges' g formula step by step

The calculation has three parts, a pooled standard deviation, an uncorrected standardised difference and a correction factor. Borenstein and colleagues set out the standard version of each step.

$$s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}}, \qquad d = \frac{\bar{x}_1 - \bar{x}_2}{s_p}$$

where $\bar{x}_1$ and $\bar{x}_2$ are the group means, $s_1$ and $s_2$ the group standard deviations, $n_1$ and $n_2$ the group sizes, $s_p$ the pooled standard deviation and $d$ the uncorrected standardised mean difference. The correction factor depends only on the degrees of freedom.

$$J = 1 - \frac{3}{4\,df - 1}, \qquad df = n_1 + n_2 - 2, \qquad g = J \times d$$

where $J$ is the small-sample correction factor, $df$ is the degrees of freedom and $g$ is Hedges' g. Substituting for $df$ gives the common form $J = 1 - 3/(4(n_1 + n_2) - 9)$. This is an accurate approximation to an exact factor based on gamma functions, and the two agree to three decimal places from about 20 degrees of freedom.

Variance and confidence interval for Hedges' g

Meta-analysis weights each study by the inverse of its variance, so the variance of g matters as much as g itself. Borenstein and colleagues give the large-sample variance of d and scale it by the square of the correction factor.

$$V_d = \frac{n_1 + n_2}{n_1 n_2} + \frac{d^2}{2(n_1 + n_2)}, \qquad V_g = J^2 \times V_d, \qquad SE_g = \sqrt{V_g}$$

where $V_d$ is the variance of d, $V_g$ the variance of g and $SE_g$ its standard error. A 95% confidence interval is $g \pm 1.96 \times SE_g$. The Cochrane Handbook gives a slightly different approximation for the standard error, and the two give near-identical results except in very small studies.

Worked example with a small trial

The figures in this example are illustrative. A small trial of a rehabilitation programme reports a mean function score of 62 (standard deviation 9) in 10 treated patients and 54 (standard deviation 10) in 12 control patients, with higher scores better.

$$s_p = \sqrt{\frac{9 \times 81 + 11 \times 100}{20}} = \sqrt{91.45} = 9.563, \qquad d = \frac{62 - 54}{9.563} = 0.837$$

where 81 and 100 are the squared group standard deviations and 20 is the degrees of freedom. The correction factor and Hedges' g follow.

$$J = 1 - \frac{3}{4 \times 20 - 1} = 1 - \frac{3}{79} = 0.962, \qquad g = 0.962 \times 0.837 = 0.805$$

where the correction lowers the estimate by about 4%. The variance and interval come next.

$$V_d = \frac{22}{120} + \frac{0.837^2}{44} = 0.199, \qquad V_g = 0.962^2 \times 0.199 = 0.184, \qquad SE_g = 0.429$$

where 22 is the total sample size and 120 is the product of the two group sizes. The 95% confidence interval is 0.805 ± 1.96 × 0.429, or about minus 0.04 to 1.65, so this trial alone is compatible with no effect even though its point estimate is large. With 50 patients in each arm the correction would be $J = 1 - 3/391 = 0.992$, a reduction of under 1%, which is why the correction matters mainly for small trials.

Interpreting the size of g

A value of g is a difference in means measured in pooled standard deviations. Cohen's rough benchmarks, which the Cochrane Handbook repeats as a rule of thumb, treat 0.2 as a small effect, 0.5 as moderate and 0.8 as large, but Cohen offered them for use only when nothing better is known. In health care a clinically important difference on the instrument itself is a better guide, and the same g can be trivial on one scale and important on another.

Two features of g affect what it means. It depends on the spread of the population studied as well as the treatment effect, so a trial with a homogeneous population gives a larger g for the same raw difference than a trial with a varied one. It also assumes the standard deviations in different studies measure the same between-person variation, which fails when trials use change scores in some cases and final values in others.

Turning a pooled g into model inputs

A cost-effectiveness model needs effects in units it can cost and value, not in standard deviations. The Cochrane Handbook describes re-expressing a pooled standardised mean difference in the units of a familiar instrument by multiplying it by a typical standard deviation for that instrument, ideally taken from a representative population. With an illustrative standard deviation of 20 points on a 0 to 100 scale, a pooled g of 0.805 corresponds to about 16 points.

For a binary model input, the Handbook also describes Chinn's conversion, which treats a standardised mean difference as a log odds ratio scaled by $\pi/\sqrt{3}$.

$$\ln(\text{OR}) = \frac{\pi}{\sqrt{3}} \times g \approx 1.814 \times g$$

where OR is the implied odds ratio. For g = 0.805 this gives a log odds ratio of 1.460 and an odds ratio of about 4.3. The conversion assumes an underlying logistic distribution of the continuous outcome, and it is an approximation whose error should be reflected in the model's uncertainty analysis. Moving from a function score to utilities is a further step that needs a mapping algorithm and brings its own error.

Hedges' g, Cohen's d and other effect sizes

Hedges' g is one of several standardised effect size measures for continuous outcomes. Hedges' g and Cohen's d differ only by the factor J, so they agree closely in trials of moderate size and diverge in small ones. Glass's delta divides by the control group's standard deviation alone, which suits treatments expected to change the spread of outcomes. A mean difference in natural units is preferred whenever all studies use the same instrument, because it needs no standardisation and is easier to interpret.

Common errors with Hedges' g

Reporting d and calling it g, or applying the correction twice, are frequent mistakes that make pooled results inconsistent. Mixing studies whose outcomes run in opposite directions without first reversing the sign of some of them reverses part of the evidence. Treating Cohen's benchmarks as fixed thresholds, or comparing g across populations with very different spread, misreads the scale. Pooling g from studies that report change scores with studies that report final values is a further error, because their standard deviations measure different things.

Sources

• Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics. 1981;6(2):107-128.
• Hedges LV, Olkin I. Statistical Methods for Meta-Analysis. Academic Press; 1985.
• Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. Wiley; 2009. Chapter 4.
• Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane. 2024. Chapters 6, 10 and 15.
• Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum; 1988.
• Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Statistics in Medicine. 2000;19(22):3127-3131.