Signature
S = C - D; tau = S / (K * (K - 1) / 2); Var_S = K * (K - 1) * (2 * K + 5) / 18; Z = S / sqrt(Var_S)
| Inputs | Definition | Unit |
|---|---|---|
C | Number of pairs in which the study with the larger variance has the larger standardised effect | count of pairs |
D | Number of pairs in which the study with the larger variance has the smaller standardised effect | count of pairs |
K | Number of studies in the meta-analysis, giving K(K-1)/2 pairs | count, at least 2 |
S | Kendall's score, concordant minus discordant pairs of studies | count of pairs, negative when larger variances go with smaller standardised effects |
|---|---|---|
tau | Kendall's rank correlation between the standardised effects and the variances | none, from -1 to 1 |
Var_S | Variance of S under independence when no standardised effects or variances are tied | squared count of pairs |
Z | Kendall's score divided by its null standard deviation, without continuity correction | standard normal deviate |
Function
Begg and Mazumdar adjusted rank correlation test function for small-study effects
Maps the effect estimates of K studies in a meta-analysis, on an approximately normal scale such as log hazard ratios, and their within-study variances to the Begg test. Each estimate is standardised against the inverse-variance pooled estimate, and Kendall's rank correlation between the standardised effects and the variances is tested against zero. A significant correlation signals funnel plot asymmetry, a small-study effect, but not its cause: non-reporting of small studies, flawed small trials, genuinely different patients in small trials, artefact and chance can all produce it. The records follow the notation of the Begg Test article. The pooled estimate itself is the common-effect estimate HE-FM-ADMA-001.
Computational function
Computational function: Begg test from study effect estimates and standard errors
Takes the effect estimates of K studies on an approximately normal scale and their standard errors, and returns Kendall's tau, the normal statistic and its two-sided p-value. It computes the inverse-variance weights and pooled estimate (HE-FM-ADMA-001), standardises each estimate (HE-FM-BEGG-002), compares every pair of studies once by the signs of their differences in standardised effect and in variance, and applies HE-FM-BEGG-001 to the counts. The inputs therefore differ from the formula's variables: the formula needs the pair counts, and the function builds them from the raw estimates.
Inputs and outputs:
effect: Effect estimate of each study on an approximately normal scale, such as a log hazard ratio; required. Unit: scale of the effect measure.;se: Standard error of each estimate, in the same order; required, above zero. Unit: same as effect.;tau: Kendall's rank correlation between standardised effects and variances. Unit: none.;Z: Normal statistic without continuity correction. Unit: standard normal deviate.;p: Two-sided p-value from the standard normal distribution. Unit: probability.Assumption: No tied standardised effects or tied variances, so the null variance needs no tie correction; a tied pair counts as neither concordant nor discordant, and with ties the variance is only approximate. With fewer than 10 studies, or studies of similar size, the Cochrane Handbook advises against tests for funnel plot asymmetry; it does not name the rank correlation test, so applying this to the Begg test is an inference.
Worked example (Ten illustrative trials behind a model's hazard ratio): The article's ten log hazard ratios and standard errors give a pooled log hazard ratio of -0.1735, 11 concordant and 34 discordant pairs, tau of about -0.511, Z of about -2.06 and a two-sided p-value of about 0.040.
effect = [-0.12,-0.20,-0.08,-0.18,-0.30,-0.10,-0.42,-0.35,-0.55,-0.40]; se = [0.06,0.08,0.10,0.12,0.15,0.18,0.20,0.24,0.28,0.32]; tau = -0.5111; Z = -2.0572; p = 0.0397Excel:
=SUMPRODUCT(SIGN((StdEffects-TRANSPOSE(StdEffects))*(SEs-TRANSPOSE(SEs))))/2With the estimates in a range named Effects, the standard errors in SEs, the weights=1/SEs^2in Weights, the pooled estimate=SUMPRODUCT(Effects,Weights)/SUM(Weights)in PooledEffect and the standardised effects=(Effects-PooledEffect)/SQRT(SEs^2-1/SUM(Weights))in StdEffects, this formula returns Kendall's score S with each pair counted once (the full matrix counts each pair twice). In versions before Excel 365 the range formulas are entered as array formulas.R:
begg_test <- function(effect, se) { v <- se^2; W <- sum(1/v); pooled <- sum(effect/v)/W; t <- (effect-pooled)/sqrt(v-1/W); K <- length(v); S <- sum(sign(outer(t, t, "-")*outer(v, v, "-")))/2; z <- S/sqrt(K*(K-1)*(2*K+5)/18); c(tau = S/(K*(K-1)/2), z = z, p = 2*pnorm(-abs(z))) }Returns tau -0.511, z -2.06 and p 0.040 for the article's trials; with t and v computed as in the function,cor.test(t, v, method = "kendall")gives the same tau with the exact p-value of about 0.047.Python:
def begg_test(effect, se): import math, itertools; v = [s**2 for s in se]; W = sum(1/x for x in v); pooled = sum(e/x for e, x in zip(effect, v))/W; t = [(e-pooled)/math.sqrt(x-1/W) for e, x in zip(effect, v)]; K = len(v); S = sum(((t[i]-t[j])*(v[i]-v[j]) > 0)-((t[i]-t[j])*(v[i]-v[j]) < 0) for i, j in itertools.combinations(range(K), 2)); z = S/math.sqrt(K*(K-1)*(2*K+5)/18); return S/(K*(K-1)/2), z, math.erfc(abs(z)/math.sqrt(2))Returns (-0.5111, -2.0572, 0.0397) for the article's trials.Test (Begg score from pairwise signs equals the direct pair count): The score from the matrix of pairwise signs, in a cell named BeggS, equals concordant minus discordant pairs counted one pair at a time, -23 for the article's trials. Expected result: TRUE. FALSE shows the halving of the full matrix left out, which gives -46 (computed here for illustration). Excel check:
=BeggS=Concordant-DiscordantTest (Begg score within its attainable range): The absolute score cannot exceed the number of pairs, 45 for ten studies. Expected result: TRUE. Excel check:
=ABS(BeggS)<=Studies*(Studies-1)/2Common error (Leaving out the variance correction in the standardisation): Dividing each deviation by the study's own standard error instead of the square root of its variance less one over the sum of weights changes the ranks in the article's example to 12 concordant and 33 discordant pairs, so S is -21 and Z about -1.88, a p-value of about 0.060 instead of 0.040 (computed here for illustration).
Source: StataCorp. Stata Meta-Analysis Reference Manual: meta bias. College Station, TX: StataCorp LLC; 2025. Methods and formulas, Begg's rank correlation test: the standardised effect sizes, the inverse-variance pooled estimate, the variance of the difference between each estimate and the pooled estimate, and the test as Kendall's rank correlation test of independence between the standardised effects and their variances.
W = sum_(j=1)^K [1 / v_j]; theta_IV = sum_(j=1)^K [theta_j / v_j] / W; t_j = (theta_j - theta_IV) / sqrt(v_j - 1 / W); S = sum_(j<k) [sign((t_j - t_k) * (v_j - v_k))]; tau = S / (K * (K - 1) / 2); Z = S / sqrt(K * (K - 1) * (2 * K + 5) / 18)
Try this function
Implementations
Excel
Begg normal statistic from named pair counts
Excel divides concordant minus discordant pairs, from the named cells, by the number of pairs to give tau in a cell named KendallTau, and by the square root of the null variance for the named number of studies to give the statistic in a cell named BeggZ. =2*(1-NORM.S.DIST(ABS(BeggZ),TRUE)) then gives the two-sided p-value; the continuity-corrected version uses ABS(Concordant-Discordant)-1 as the numerator.
=(Concordant-Discordant)/(Studies*(Studies-1)/2); =(Concordant-Discordant)/SQRT(Studies*(Studies-1)*(2*Studies+5)/18)
Assumptions
No tied ranks in the Begg test
No two standardised effects and no two variances are equal, so the null variance K(K-1)(2K+5)/18 applies without the tie corrections in Stata's exact formula. Studies of identical precision create ties, and the Cochrane Handbook advises against tests for funnel plot asymmetry when studies have similar standard errors.
Enough studies of differing size for the Begg test
The normal approximation and the power of the test are poor with few studies. The Cochrane Handbook reports the rule of thumb that tests for funnel plot asymmetry be used only when a meta-analysis includes at least 10 studies, and not when the studies have similar standard errors, with the result read alongside the funnel plot.
Worked examples
Begg statistic for the article's ten illustrative trials
The article's ten illustrative trials behind a model's hazard ratio give 11 concordant and 34 discordant pairs out of 45, so S is -23, tau is about -0.511, Var(S) is 125 and Z is about -2.06, a two-sided p-value of about 0.040. With the continuity correction the absolute statistic is 22 divided by 11.18, about 1.97 (p about 0.049), and the exact permutation p-value is about 0.047.
C = 11; D = 34; K = 10; S = -23; tau = -0.5111; Var_S = 125; Z = -2.0572
Perfect ordering of four studies in the Begg test
With four studies in perfect order, 6 concordant pairs and none discordant, tau is 1 and the uncorrected Z is about 2.04, above 1.96. The exact two-sided p-value for a perfect ordering with no ties is 2 divided by 4 factorial, about 0.083.
C = 6; D = 0; K = 4; S = 6; tau = 1; Var_S = 8.6667; Z = 2.0381
Common errors
Reading the uncorrected normal Begg statistic with four studies
With four studies in perfect order the uncorrected Z of about 2.04 suggests significance at 5%, but the exact two-sided p-value is about 0.083, so even perfect concordance cannot reach significance on the exact test with four studies. With five it is about 0.017.
Treating a non-significant Begg test as evidence of no bias
Begg and Mazumdar warned that bias cannot be ruled out when the test is not significant. In simulations by Sterne, Gavaghan and Egger with ten trials, a 20% control event rate and no treatment effect, moderate bias was detected by the rank correlation test in 14.7% of meta-analyses and severe bias in 31.1%, judging evidence at P < 0.1.
Reporting a Begg p-value without stating its version
For the article's ten trials the uncorrected normal p-value is about 0.040, the continuity-corrected one about 0.049 and the exact permutation p-value about 0.047. All lie close to 0.05, so a result reported without its version cannot be checked or compared.
Sources
Stata definition of Kendall's score, tau and null variance used by the Begg test
StataCorp. Stata Base Reference Manual: spearman (Spearman's and Kendall's correlations). College Station, TX: StataCorp LLC; 2025. Methods and formulas, Kendall's tau: concordant and discordant pairs, the score S as C minus D, tau as S over n(n-1)/2, the exact null variance of S with its tie terms, and the normal approximation with a continuity correction using abs(S) minus 1. The meta bias entry refers to these formulas for the Begg test.
Begg and Mazumdar rank correlation test and its power
Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4):1088-1101. Abstract: an adjusted rank correlation test proposed as a direct statistical analogue of the funnel graph, fairly powerful with 75 studies but only moderately powerful with 25, with the warning that bias cannot be ruled out if the test is not significant.
Power of the rank correlation test against the regression test
Sterne JAC, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology. 2000;53(11):1119-1129. Abstract: the rank correlation test was less powerful than the regression method; with ten trials, a 20% control event rate and no treatment effect it detected moderate bias in 14.7% and severe bias in 31.1% of simulations, at P < 0.1.
Cochrane conditions for testing funnel plot asymmetry
Page MJ, Higgins JPT, Sterne JAC. Chapter 13: Assessing risk of bias due to missing evidence in a meta-analysis. In: Higgins JPT, Thomas J, Chandler J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. Section 13.3.4.4: tests for funnel plot asymmetry used only with at least 10 studies, not when studies are of similar size, and interpreted alongside visual inspection of the funnel plot.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0