Hedges' g from Cohen's d with the approximate correction factor J

Signature

m = n_1 + n_2 - 2; J = 1 - 3 / (4 * m - 1); g = J * d

Function

Hedges' g small-sample correction, sampling variance and bias of the standardised mean difference

Computational function

• Computational function: exact Hedges' correction factor by the log-gamma function with g and its variance

  Computes the exact correction factor of Hedges, the ratio of gamma functions in his equation 6e, through the logarithm of the gamma function so that it does not overflow in large samples, and returns with it Hedges' g, the approximate factor of HE-FM-HEDG-001 and its error, the variance and standard error of g of HE-FM-HEDG-002 with the exact factor, and the exact variance of the corrected estimate from Hedges' Theorem 2 evaluated with g in place of delta. The site calculator has no gamma function, so the Math / Code line runs the approximate chain of HE-FM-HEDG-001 and HE-FM-HEDG-002, and the exact steps are in the Excel, R and Python code. The inputs are d from HE-FM-CD-001 and the two group sizes.

  Inputs and outputs: d is Cohen's d from HE-FM-CD-001 in standard deviation units, and n_1 and n_2 are the group sizes in people, with a total of at least 4 so that m is at least 2, and of at least 5 for v_theorem2, which needs m of at least 3. The calculator outputs m, J and g are as in HE-FM-HEDG-001, and V_d, V_g and SE_g as in HE-FM-HEDG-002. The code also returns J_exact, the exact factor c(m); approx_error, the approximate less the exact factor; g_exact, v_g_exact and se_g_exact, Hedges' g with its variance and standard error under the exact factor; and v_theorem2, the exact variance of the corrected estimate at delta equal to g_exact, c(m) squared times m/(m minus 2) times (1/n tilde plus g squared) less g squared, with n tilde equal to n_1 n_2/(n_1 + n_2), in squared standard deviation units.

  Assumption: Normal outcomes with a common standard deviation in both independent groups and a pooled standard deviation, as in Hedges' Theorems 1 and 2. Putting g in place of the unknown delta in the exact variance is a plug-in choice made here, as Hedges does for his large-sample variance.

  Worked example (Article rehabilitation trial): With d of 0.8366 and 10 and 12 patients the calculator chain gives J of about 0.962025, g of about 0.8048 and SE_g of about 0.4294. The code gives J_exact of about 0.961945, so approx_error is about 0.000081, g_exact about 0.8048, se_g_exact about 0.4294 and v_theorem2 about 0.2067 against V_g of about 0.1844 (computed here). d = 0.8366; n_1 = 10; n_2 = 12; m = 20; J = 0.962025; g = 0.8048; V_d = 0.1992; V_g = 0.1844; SE_g = 0.4294

  Worked example (Hedges' Table I at 20 degrees of freedom): With d of 1 and 11 patients per arm, m is 20 and J_exact is 0.96194, the value in Hedges' Table I, against the approximate 0.962025. d = 1; n_1 = 11; n_2 = 11; m = 20; J = 0.962025; g = 0.9620; V_d = 0.2045; V_g = 0.1893; SE_g = 0.4351

  Excel: =EXP(GAMMALN(DegFreedom/2)-GAMMALN((DegFreedom-1)/2))/SQRT(DegFreedom/2) in a cell named ExactJ; =CorrectionJ-ExactJ in ApproxErrorJ; =ExactJ*CohensD in HedgesGExact; =ExactJ^2*VarianceD in VarianceGExact; =SQRT(VarianceGExact) in StdErrGExact; =ExactJ^2*DegFreedom/(DegFreedom-2)*((SizeIntervention+SizeComparator)/(SizeIntervention*SizeComparator)+HedgesGExact^2)-HedgesGExact^2 in VarianceTheorem2. DegFreedom, CorrectionJ, CohensD, VarianceD, SizeIntervention and SizeComparator are the named cells of HE-IM-HEDG-001 and HE-IM-HEDG-002. GAMMALN is available in all Excel versions.

  R: hedges_g_exact <- function(d, n1, n2) { m <- n1+n2-2; J_exact <- exp(lgamma(m/2)-lgamma((m-1)/2))/sqrt(m/2); J_approx <- 1-3/(4*m-1); nt <- n1*n2/(n1+n2); v_d <- (n1+n2)/(n1*n2)+d^2/(2*(n1+n2)); g <- J_exact*d; list(m=m, J_exact=J_exact, J_approx=J_approx, approx_error=J_approx-J_exact, g_exact=g, v_g_exact=J_exact^2*v_d, se_g_exact=sqrt(J_exact^2*v_d), v_theorem2=J_exact^2*m/(m-2)*(1/nt+g^2)-g^2) } Returns a named list; hedges_g_exact(0.8366, 10, 12)$J_exact is about 0.9619445.

  Python: def hedges_g_exact(d, n1, n2): m = n1+n2-2; J_exact = math.exp(math.lgamma(m/2)-math.lgamma((m-1)/2))/math.sqrt(m/2); J_approx = 1-3/(4*m-1); nt = n1*n2/(n1+n2); v_d = (n1+n2)/(n1*n2)+d**2/(2*(n1+n2)); g = J_exact*d; return {"m": m, "J_exact": J_exact, "J_approx": J_approx, "approx_error": J_approx-J_exact, "g_exact": g, "v_g_exact": J_exact**2*v_d, "se_g_exact": math.sqrt(J_exact**2*v_d), "v_theorem2": J_exact**2*m/(m-2)*(1/nt+g**2)-g**2} Requires the math module; returns a dictionary with the same values as the R function.

  Test (Hedges' Table I value at 20 degrees of freedom): With SizeIntervention and SizeComparator both 11, the model's ExactJ cell rounds to 0.96194, the value printed in Hedges' Table I. Expected result TRUE. FALSE shows the total sample size used in place of the degrees of freedom, which gives 0.96545, the Table I value for 22, or m/2 used in the denominator without its square root, about 0.3042. Excel check =ROUND(ExactJ,5)=0.96194

  Test (Hedges' maximum approximation error at two degrees of freedom): With both group sizes 2, the model's ExactJ cell rounds to 0.56419, the Table I value, and ApproxErrorJ rounds to Hedges' maximum error of 0.007. Expected result TRUE. FALSE shows the two GAMMALN arguments swapped, which gives about 1.7725, or the error taken as exact less approximate, minus 0.007. Excel check =AND(ROUND(ExactJ,5)=0.56419,ROUND(ApproxErrorJ,3)=0.007)

  Common error (Gamma function overflow): Computing the ratio with the gamma function itself fails in large trials, because the gamma function exceeds the largest double-precision number above an argument of about 171.6. In Python math.gamma raises an overflow error and in R gamma returns Inf, so the factor is Inf at 344 degrees of freedom, a total of 346 patients, and NaN from 345 (checked here), while the log-gamma form returns about 0.997818. Excel's GAMMALN avoids the problem in the same way.

  Source: Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics. 1981;6(2):107-128. Equation 6e (page 111) defining c(m), Theorem 2 (page 116) giving the exact variance of the corrected estimator, and Table I (page 115) tabulating c(m) for each m up to 50.

  m = n_1 + n_2 - 2; J = 1 - 3 / (4 * m - 1); g = J * d; V_d = (n_1 + n_2) / (n_1 * n_2) + d^2 / (2 * (n_1 + n_2)); V_g = J^2 * V_d; SE_g = sqrt(V_g)

Try this function

Implementations

• Excel

  Hedges' g in three cells from Cohen's d and the two group sizes

  With Cohen's d in the cell named CohensD (the output cell of HE-IM-CD-001) and the group sizes in cells named SizeIntervention and SizeComparator, the three formulas return the degrees of freedom in a cell named DegFreedom, the correction factor in CorrectionJ and Hedges' g in HedgesG.

  =SizeIntervention+SizeComparator-2; =1-3/(4*DegFreedom-1); =CorrectionJ*CohensD

Assumptions

• Normal outcomes with a common standard deviation behind the Hedges' correction

  Hedges derived the bias of d and its correction for outcomes that are normally distributed in each group with the same standard deviation in both, the model of his Theorem 1, under which d scaled by the square root of n_1 n_2 over (n_1 + n_2) follows a noncentral t distribution. Outside that model the factor still shrinks d towards zero, but the result is not shown to be unbiased.

• Degrees of freedom of the standard deviation actually used in the Hedges' correction

  Hedges sets m to the comparator group size less 1 when d is standardised by the control group standard deviation, as in Glass's estimator, and to the total sample size less 2 when the pooled standard deviation is used. This formula takes the pooled standard deviation of HE-FM-CD-001, so m is n_1 plus n_2 less 2.

• Algebraic approximation in place of the exact gamma-function factor for Hedges' g

  The algebraic factor is slightly larger than the exact one. Hedges reports a maximum error of 0.007 at m equal to 2, an error within 0.00033 for m of 10 or more and an error no larger than 0.000015 for m above 50. The exact factor, which the site calculator cannot evaluate, is computed in HE-CF-HEDG-001.

Worked examples

• Article rehabilitation trial with 10 and 12 patients corrected to Hedges' g

  Illustrative trial from the article, with means of 62 and 54, standard deviations of 9 and 10 and 10 and 12 patients, which give a d of about 0.8366 by HE-FM-CD-001. With 20 degrees of freedom J is 1 minus 3 over 79, about 0.962025, and g is about 0.8048, shown as 0.962 and 0.805 in the article.

  n_1 = 10; n_2 = 12; d = 0.8366; m = 20; J = 0.962025; g = 0.8048

• Fifty patients per arm shrink the Hedges' correction below 1%

  Computed here for illustration with the same d of 0.8366 and 50 patients in each arm. J is about 0.992327, the article's 0.992, and g is about 0.8302, a reduction of under 1%.

  n_1 = 50; n_2 = 50; d = 0.8366; m = 98; J = 0.992327; g = 0.8302

• No difference in means gives a Hedges' g of zero

  With a d of 0 and 30 patients per arm, J is about 0.987013 and g is 0, a limiting case. The correction rescales d and never changes its sign.

  n_1 = 30; n_2 = 30; d = 0; m = 58; J = 0.987013; g = 0

Common errors

• Total sample size written in place of the degrees of freedom in Hedges' J

  Writing 1 minus 3 over (4N minus 1), with N the total sample size, mixes the two forms of the factor. For the article's trial it gives a J of about 0.9655 instead of 0.9620 and a g of about 0.8078 instead of 0.8048, computed here for illustration, so part of the bias stays in the estimate.

• Hedges' correction applied to an estimate that is already g

  Multiplying a reported Hedges' g by J again, often because a paper labels g as d, gives about 0.7743 instead of 0.8048 for the article's trial (0.962025 squared times 0.8366, computed here for illustration). The article lists applying the correction twice among the frequent mistakes that make pooled results inconsistent.

• Pooled degrees of freedom used with a comparator-group standard deviation

  When the difference is standardised by the comparator group's standard deviation alone, Hedges sets m to the comparator group size less 1. Using the pooled value of 20 for the article's trial, whose comparator arm has 12 patients, gives a J of about 0.9620 instead of about 0.9302 for 11 degrees of freedom and leaves almost half of the bias in place, an expected overstatement of about 3.4% instead of about 7.5% with the exact factor, computed here for illustration.

Sources

• Hedges on the exact small-sample correction factor and its algebraic approximation

  Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics. 1981;6(2):107-128. Theorem 1 (page 111), which for normal outcomes with a common variance gives the expectation of the uncorrected estimate as delta over c(m), with m equal to the control group size less 1 or the total of both groups less 2 depending on whether a control group or pooled standard deviation is used, and defines c(m) as Gamma(m/2) over the square root of m/2 times Gamma((m minus 1)/2) (equation 6e). Section 4, An unbiased estimator based on the standardized mean difference (pages 112 to 114), which defines the unbiased estimator as the uncorrected one times c(m) (equation 7) and gives the approximation c(m) about 1 minus 3/(4m minus 1), with a maximum error of 0.007 when m is 2, accuracy within 0.00033 when m is at least 10 and an error not above 0.000015 when m exceeds 50. Table I (page 115) tabulates c(m) to five decimals for each m up to 50. In Hedges' notation g is the uncorrected estimator and the corrected one is g with superscript U.

  View source →

• What Works Clearinghouse handbook on Hedges' g with the correction omega

  What Works Clearinghouse. What Works Clearinghouse Procedures and Standards Handbook, Version 5.0 (WWC 2022008). US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance; August 2022. Appendix E, General effect size formulas (pages 162 and 163), equations E.1 to E.4, which compute Hedges' g as omega times the mean difference over the pooled standard deviation with omega equal to 1 minus 3/(4df minus 1) and df equal to N minus 2 for individual-level assignment, citing Hedges 1981, and state that the factor is 0.99 for a study with 100 students in total.

  View source →

• Review Manager algorithms on Hedges' adjusted g with the factor 1 minus 3/(4N minus 9)

  Deeks JJ, Higgins JPT, on behalf of the Cochrane Statistical Methods Group. Statistical algorithms in Review Manager. Cochrane; May 2022. Section Standardized difference in means (standardized mean difference) (page 3), which states that the version implemented in RevMan is Hedges' adjusted g, very similar to Cohen's d but with an adjustment for small sample bias, and gives the SMD as the difference in means over the pooled standard deviation times 1 minus 3/(4N minus 9), with N the sum of the two group sizes.

  View source →

• Lakens on Hedges's g as Cohen's d times the small-sample factor

  Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology. 2013;4:863. Section Cohen's d in between-subjects designs, equation 4, which gives Hedges's g as Cohen's d times 1 minus 3/(4(n1 + n2) minus 9), and states that the difference between the two is very small, especially in sample sizes above 20.

  View source →

Canonical Identity