Kendall Tau

Kendall Tau for Rankings in Health Economics

Kendall tau, usually written Kendall's tau or $\tau$, asks whether two variables put the same units in the same order, by looking at every pair of units and checking whether both variables rank that pair the same way round. In health economics the units are often health states, treatments or studies, so the coefficient answers questions such as whether a ranking task orders EQ-5D states as time trade-off values do, or whether two ranking metrics in a network meta-analysis give the same treatment hierarchy. Its simplest version reads directly as a difference between two probabilities, which makes it easier to explain to an appraisal committee than a Spearman or Pearson coefficient. The same pair counts sit inside the Begg test for small-study effects and inside Harrell's C for survival models. Pearson's and Spearman's coefficients, with a cost and QALY example comparing all three, are on the Correlation Coefficient page.

Concordant and discordant pairs of units

The coefficient works on pairs of units, not on single observations. The Stata reference manual calls two observations $i$ and $j$ concordant when $(x_i - x_j)(y_i - y_j) > 0$ and discordant when that product is negative, and defines Kendall's score as the number of concordant pairs minus the number of discordant pairs. Dividing the score by the number of possible pairs gives Kendall's original coefficient, now called tau-a.

$$S = C - D, \qquad \tau_a = \frac{S}{n(n-1)/2}$$

where $S$ is Kendall's score, $C$ and $D$ are the numbers of concordant and discordant pairs, $n$ is the number of units and $n(n-1)/2$ is the number of distinct pairs among them.

Newson defines the population value of tau-a as the expected product of the signs of the differences in $X$ and in $Y$ for two units drawn independently from the same population. That makes it the probability of concordance minus the probability of discordance, and he notes that values of 1, minus 1 and 0 signify a perfect positive relationship, a perfect negative relationship and no overall ordinal relationship.

$$\tau_a = \Pr(\text{concordant}) - \Pr(\text{discordant})$$

where both probabilities refer to a randomly chosen pair of units. With no ties the two probabilities sum to one, so the probability that a random pair is ordered the same way by both variables is $(1 + \tau_a)/2$. Craig and colleagues used this reading for health states, where seven concordant pairs and one discordant pair out of eight give a tau of 7/8 minus 1/8, or 0.75.

Because only the order within each pair enters the count, any transformation that preserves order leaves tau unchanged. Logging costs, rescaling a utility index or replacing values by ranks gives the same coefficient, which is why tau suits ordinal scaling data and skewed costs alike.

Ties in valuation data and the tau-b and tau-c variants

Ties are common in health economic data, for example identical time trade-off values for two states, or patients sharing a grade on the modified Rankin scale. In the eight-country EQ-5D study by Craig and colleagues, time trade-off responses were more likely to be tied than rank responses in every country. A tied pair is neither concordant nor discordant but still sits in the tau-a denominator, so tau-a cannot reach 1 when ties are present. The NIST Dataplot manual states that tau-b is preferred to tau-a when there are ties.

$$\tau_b = \frac{C - D}{\sqrt{(C + D + T_x)(C + D + T_y)}}$$

where $T_x$ is the number of pairs tied on the first variable only and $T_y$ the number tied on the second variable only. The SciPy documentation adds that a pair tied on both variables is counted in neither.

Dataplot describes tau-c, which SciPy calls Stuart's tau-c, for two variables that take a limited number of values on different scales.

$$\tau_c = \frac{2(C - D)}{n^2 (m - 1)/m}$$

where $m$ is the smaller of the numbers of distinct values of the two variables and $n$ is the number of units. The SciPy documentation states that tau-b and tau-c differ only in how they are scaled to lie between minus 1 and 1, that their p-values are identical, and that both reduce to tau-a without ties.

Software defaults differ, so a report should name the variant. Stata's ktau command displays tau-a unless told otherwise, SciPy returns tau-b by default, and the default Dataplot command follows Conover's formulation, in which each tie adds 0.5 to both the concordant and the discordant counts. The same data can therefore produce three different published values.

Testing for association and putting an interval around tau

The usual test asks whether the variables are independent. The Stata manual gives the exact null variance of the score, which in the absence of ties reduces to a short expression, and applies a normal approximation with a continuity correction.

$$\operatorname{Var}(S) = \frac{n(n-1)(2n+5)}{18}, \qquad z = \frac{|S| - 1}{\sqrt{\operatorname{Var}(S)}}$$

where $\operatorname{Var}(S)$ is the variance of the score under independence with no tied values and $z$ is compared with the standard normal distribution. With ties, the full formula adds terms for the number and size of the tied groups. The manual describes the normal approximation as surprisingly accurate for samples as small as 8, at least for null p-values with continuous variables, and notes that Kendall and Gibbons tabulate exact p-values for samples below 10.

A p-value addresses only the null hypothesis of independence. Newson argues that a confidence interval for tau-a is more informative, and that the most common single error is to run a rank test on a small sample, find a large p-value and treat it as proof of no association. His example is 17 scripts with a tau-a of 0.35 (95% CI minus 0.11 to 0.69, P = 0.17), an interval that still allows strong agreement. Rankings of a few treatments or a small expert panel carry the same risk.

How tau compares with Spearman and Pearson coefficients

Tau and Spearman's coefficient both use only order, but they sit on different scales. The Stata manual quotes Conover that Spearman's rho tends to be larger than Kendall's tau in absolute value, while as tests of significance the two give nearly identical results in most cases. Craig and colleagues found the same pattern in EQ-5D data, with a mean Kendall's tau between time trade-off and rank responses of 0.65 in the United Kingdom sample against a mean Spearman's rho of 0.79.

For two variables with a bivariate normal distribution, Newson gives Greiner's relation between tau-a and Pearson's coefficient.

$$\rho = \sin\left(\frac{\pi \tau_a}{2}\right)$$

where $\rho$ is the Pearson correlation of the bivariate normal distribution and $\tau_a$ the population tau-a. Values of tau-a of 1/3 and 1/2 correspond to Pearson correlations of 1/2 and about 0.71, so the Pearson value is the larger in magnitude. Newson adds that tau-a is much less affected than Pearson's coefficient by extreme observations and nonlinearity, and that Spearman's coefficient cannot be read as a difference between probabilities. Size labels calibrated on Pearson or Spearman values therefore cannot be carried over to tau.

Worked example with eight health states ranked by a patient panel

All numbers are illustrative. A model uses a value set to score eight health states, and an analyst checks whether a patient panel's ordinal ranking orders them the same way. The value set gives no ties, while the panel places states S4 and S5 equal.

1. Count the pairs. With 8 states there are 8 × 7 / 2 = 28 pairs.

2. Classify each pair. The panel reverses S2 and S3 and reverses S6 and S7, so $D$ = 2. The panel ties S4 and S5, which the value set separates, so $T_y$ = 1 and $T_x$ = 0. The remaining 28 − 2 − 1 = 25 pairs are concordant, and $S$ = 25 − 2 = 23.

3. Tau-a. $\tau_a$ = 23 / 28 = 0.821.

4. Tau-b. The two bracketed terms are 25 + 2 + 0 = 27 and 25 + 2 + 1 = 28, their product is 756 and its square root is 27.50, so $\tau_b$ = 23 / 27.50 = 0.836.

5. Pairs that both sources separate. Of the 27 pairs untied on both, 25/27 = 0.926 are concordant and 2/27 = 0.074 discordant, a difference of 0.852. Three defensible summaries (0.821, 0.836 and 0.852) come from one set of rankings.

6. Test of independence. One tied group of two in the panel ranking (Stata tie term 2 × 1 × 9, with the cross terms zero because the value set has no ties) gives Var($S$) = (8 × 7 × 21 − 2 × 1 × 9) / 18 = (1,176 − 18) / 18 = 64.33, a standard deviation of 8.021. With the continuity correction, $z$ = (23 − 1) / 8.021 = 2.74 and the two-sided p-value is about 0.006. Without the correction, $z$ = 23 / 8.021 = 2.87 and the p-value is about 0.004, the value SciPy reports.

7. Spearman's coefficient for comparison. The Pearson correlation of the ranks, with S4 and S5 sharing rank 4.5, is 0.946, above both tau values as Conover's remark predicts.

8. Interpretation. The panel and the value set order 25 of the 28 pairs the same way, and the association is unlikely under independence. The two reversals count equally in tau, although the utility gap is 0.06 between S2 and S3 and 0.13 between S6 and S7. Tau also says nothing about whether the panel would assign the same utilities, only the same order.

Uses in valuation, evidence synthesis and survival modelling

The pair-counting idea reaches decision inputs by several routes. Each use below asks a different question of the same counts.

In health state valuation, Craig and colleagues analysed 179,431 rank, visual analogue scale and time trade-off responses from 11,483 people in eight countries for 52 EQ-5D states, estimating Kendall's tau and Spearman's rho for each respondent. Rank responses showed a strong linear correlation with time trade-off responses (Pearson's rho 0.88 to 0.99), yet rank-based values implied greater decrements in health for mild states than cardinal values did. A strong correlation thus left a real difference in the values that would enter QALY calculations.

In network meta-analysis, Chiocchia and colleagues re-analysed 232 networks of four or more interventions, published between 1999 and 2015, and compared treatment hierarchies from relative treatment effects, the probability of producing the best value and SUCRA. The median Kendall's tau was 1 (quartiles 0.95 to 1) between SUCRA and relative treatment effects and 0.8 (quartiles 0.67 to 0.91) between the best-value probability and SUCRA. They note that tau and Spearman's rho give the same weight to every item, whereas the top of a treatment ranking matters more, so they also used Yilmaz's tau-AP, a top-weighted variant of Kendall's tau.

For small-study effects, the Stata meta-analysis manual describes the Begg test as Kendall's rank correlation test of independence between standardised effect sizes and their variances. In survival modelling, Newson defines Somers' D from tau-a as a conditional difference between the probabilities of concordance and discordance, and gives Harrell's C as (D + 1)/2, where D is Somers' D of the predictor with respect to the possibly censored survival time, which links tau to the Harrell C-index used to judge how well a prognostic model orders patients.

Common misreadings of Kendall's tau

Most errors come from reading tau with habits formed on other coefficients. Six misreadings recur in valuation and evidence synthesis work.

1. Reading tau on a Pearson scale. Tau is usually smaller in magnitude than Spearman's coefficient for the same data, and smaller than Pearson's under bivariate normality, so cut-offs designed for those coefficients make a given tau look weaker than it is.
2. Leaving the variant unnamed. Tau-a, tau-b, tau-c and Conover's version differ when ties are present, and software defaults differ.
3. Treating agreement in order as agreement in value. Two methods can order states identically while assigning different utilities, which is a question for the intraclass correlation or, for categories, the kappa statistic.
4. Reading a large p-value as no association. With few pairs, Newson's example shows an interval wide enough to include strong agreement.
5. Ignoring clustering. Many rankings from one respondent are not independent pairs. Craig and colleagues computed tau within each respondent and reported country means, and Newson's somersd package offers a cluster option.
6. Assuming every rank matters equally. Tau weights a swap at the top of a hierarchy the same as one at the bottom, which Chiocchia and colleagues addressed with a top-weighted variant.

Sources

• Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4):1088-1101. https://doi.org/10.2307/2533446
• Chiocchia V, Nikolakopoulou A, Papakonstantinou T, Egger M, Salanti G. Agreement between ranking metrics in network meta-analysis: an empirical study. BMJ Open. 2020;10(8):e037744. https://doi.org/10.1136/bmjopen-2020-037744
• Craig BM, Busschbach JJV, Salomon JA. Keep it simple: ranking health states yields values similar to cardinal measurement approaches. Journal of Clinical Epidemiology. 2009;62(3):296-305. https://doi.org/10.1016/j.jclinepi.2008.07.002
• National Institute of Standards and Technology. Dataplot Reference Manual, entry KENDALLS TAU (last updated 30 April 2026). Gaithersburg, MD: NIST Statistical Engineering Division. https://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/kendell.htm
• Newson R. Parameters behind "nonparametric" statistics: Kendall's tau, Somers' D and median differences. Stata Journal. 2002;2(1):45-64. https://doi.org/10.1177/1536867X0200200103
• SciPy Developers. scipy.stats.kendalltau. SciPy v1.18.0 Manual. Accessed October 2026. https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kendalltau.html
• StataCorp. Stata Base Reference Manual, entry spearman (Spearman's and Kendall's correlations). College Station, TX: StataCorp LLC; July 2026 online edition. https://www.stata.com/manuals/rspearman.pdf
• StataCorp. Stata Meta-Analysis Reference Manual, entry meta bias (tests for small-study effects in meta-analysis). College Station, TX: StataCorp LLC; July 2026 online edition. https://www.stata.com/manuals/metametabias.pdf