Harrell C-Index

Harrell C-Index for Survival and Prognostic Models

In health economics, Harrell's C-index is a statistic commonly quoted when a survival or prognostic model is judged on how well it orders patients by risk, for example a Cox model that an economic evaluation borrows to stratify a cohort or to predict who progresses first. The index looks only at pairs of patients whose order of events is known despite incomplete follow-up and counts how often the model gave the higher predicted risk to the patient who had the event sooner. A value of 0.5 means the ordering is no better than chance and 1 means every usable pair is ordered correctly. Because the index depends only on ranks, a model can score well on it and still predict survival times, and therefore life years and QALYs, that are far too long or too short. The binary-outcome version of the same idea is covered under area under curve.

Usable pairs and the concordance count

The C-index belongs to survival analysis, where the outcome is a time to event and some patients are still event-free when follow-up stops. Royston and Altman define Harrell's c-index as the proportion of all patient pairs in which the predictions and outcomes are concordant, adding that because of censoring not all pairs are evaluable, and they attribute the index to Harrell and colleagues' 1982 paper on evaluating the yield of medical tests.

A pair counts only when the order of the two event times is known. If the patient with the shorter observed time had the event, the order is known, whatever happened to the other patient. If the shorter observed time ended in censoring, the other patient may or may not have outlived the censored one, so the pair is set aside. The survival package vignette by Therneau and Atkinson gives the same rule, ignores pairs tied on event time, and treats a patient censored at the same time as another patient's event as the longer survivor. Among usable pairs it counts those that are concordant, discordant or tied on the prediction.

$$C = \frac{n_c + \tfrac{1}{2} n_t}{n_c + n_d + n_t}$$

where $C$ is the concordance index, $n_c$ is the number of usable pairs in which the patient with the earlier event has the higher predicted risk, $n_d$ is the number in which that patient has the lower predicted risk, and $n_t$ is the number tied on predicted risk.

Uno and colleagues describe Harrell's statistic as a rank-correlation measure motivated by Kendall's tau for censored survival data, which links the index to Kendall tau. The vignette defines Somers' D on the same counts and shows that the concordance equals $(D+1)/2$.

$$D_{xy} = \frac{n_c - n_d}{n_c + n_d + n_t} = 2C-1$$

where $D_{xy}$ is Somers' rank correlation between predicted risk and outcome. The vignette also notes that for a binary outcome the concordance equals the area under the ROC curve.

What the index estimates when follow-up is incomplete

Setting aside unusable pairs is simple but has a cost. Uno and colleagues showed that when patients have different follow-up times, Harrell's statistic converges to an association measure that involves the study censoring distribution, so two studies of the same model can target different values. Steyerberg and colleagues likewise note that ignoring pairs that cannot be ordered gives a statistic that depends on the censoring pattern, and Royston and Altman state that c is known to be biased away from the null due to right censoring.

Uno's censoring-adjusted estimator

Uno and colleagues proposed weighting each usable pair by the inverse square of the estimated probability of remaining uncensored, with follow-up restricted to a prespecified time. They describe the result as consistent for a concordance measure that is free of censoring.

$$\hat{C}_\tau = \frac{\sum_{i}\sum_{j} \Delta_i \, \hat{G}(X_i)^{-2} \, I(X_i < X_j, X_i < \tau) \, I(\eta_i > \eta_j)}{\sum_{i}\sum_{j} \Delta_i \, \hat{G}(X_i)^{-2} \, I(X_i < X_j, X_i < \tau)}$$

where $X_i$ is the observed time of patient $i$, $\Delta_i$ equals 1 for an event and 0 for censoring, $\eta_i$ is the risk score, $\hat{G}(\cdot)$ is the Kaplan-Meier estimator of the probability of remaining uncensored, $\tau$ is a time at which some patients are still under observation, and $I(\cdot)$ equals 1 when its condition holds. Dropping the weights and the limit $\tau$ gives Harrell's statistic as Uno and colleagues write it, without the half credit for tied predictions. The method assumes that censoring is independent of event times and covariates, the independent censoring assumption.

Other variants and how much the choice matters

Steyerberg and colleagues note that Gönen and Heller proposed a variant that is independent of censoring but holds only within a Cox proportional hazards model, and that time-dependent C statistics have also been proposed. The vignette objects that Gönen and Heller's estimate assumes the model is exactly correct, when validation asks how well an imperfect model performs.

In practice the estimators often agree. In Uno and colleagues' Framingham example the conventional C-index was 0.75, with a 95% confidence interval of 0.73 to 0.77, and their new estimates were very similar for horizons of 8, 10 and 12 years. In the vignette's example most likely to show a difference, a Cox model fitted to data on non-alcoholic fatty liver disease, Harrell's weighting gives 0.823 and Uno's 0.805. The vignette names two conditions for larger gaps, heavy censoring while events are still occurring and non-proportional hazards.

Worked example with six patients and one censored follow-up

All numbers in this example are illustrative. A Cox model gives six patients with advanced disease a risk score, with higher scores meaning a worse prognosis.

1. Find the usable pairs. Six patients form 6 × 5 / 2 = 15 pairs. Patient B was censored at month 7, before the observed times of C, D, E and F, so those four pairs cannot be ordered, and the pair E and F is dropped for the same reason. That leaves 10 usable pairs.

2. Score each pair. Patient A, the earliest event, has the highest score and is concordant with all five others. Patient C (1.8) is concordant with D (0.9) and F (0.5) and tied with E (1.8). Patient D (0.9) is concordant with F (0.5) but discordant with E, who outlived D despite a higher score. The totals are 8 concordant, 1 discordant and 1 tied.

$$C = \frac{8 + 0.5 \times 1}{8 + 1 + 1} = \frac{8.5}{10} = 0.85$$

where the numerator gives full credit for concordant pairs and half credit for the tie. Somers' D is 7 / 10 = 0.70, which equals 2 × 0.85 minus 1.

3. Change what censoring hides. Suppose patient B had instead been followed up and had the event at month 8. B (1.6) is then concordant with D (0.9) and F (0.5) but discordant with C (1.8) and E (1.8), both of whom lived longer, and the pair A and B stays concordant.

$$C = \frac{10 + 0.5 \times 1}{10 + 3 + 1} = \frac{10.5}{14} = 0.75$$

where 14 usable pairs now give 10 concordant, 3 discordant and 1 tied. The model has not changed, yet the index fell by 0.10 because the pairs that censoring had removed were among the hardest to order.

4. Follow the scores into an economic model. A second version of the model predicts every patient's survival to be exactly twice as long. The ordering is identical, so its C-index is the same 0.85. If the first version implies a mean survival of 14 months, at an illustrative utility of 0.7 the undiscounted QALYs per patient are 14 / 12 × 0.7 = 0.817, against 28 / 12 × 0.7 = 1.633 for the second. The C-index cannot tell the two versions apart.

Where the C-index enters economic evaluation

Economic models often take event probabilities from published prognostic models or risk equations, and Uno and colleagues note that C-statistics are routinely used in the medical literature to judge such risk scores. Royston and Altman separate two uses. For risk stratification, such as setting trial entry criteria or adjusting outcomes for disease severity when comparing the performance of health care institutions, they regard discrimination as the key indicator, with calibration still playing a role, and they note that prognostic models are not good at individual predictions of outcome. A risk adjustment use therefore leans on the C-index, while a model that supplies transition probabilities needs its predicted risks checked as well.

For a Cox model the C-index needs only the prognostic index, but Royston and Altman note that validating Cox models is not straightforward because event probabilities are estimated relative to an unspecified baseline function. A cost-effectiveness model needs absolute survival over a lifetime horizon, so a high C-index says nothing about the baseline survival or its extrapolation.

The NICE Decision Support Unit's Technical Support Document 14 concludes that parametric survival models are likely to be the preferred way to bring survival data into health economic models in most cases. It judges a model suitable when it fits the observed data and its extrapolated portion is clinically and biologically plausible, a portion likely to have a very large influence on estimated mean survival. The C-index measures neither property. When a curve is fitted to each arm of a trial for a partitioned survival model, every patient in an arm receives the same prediction, so pairs within an arm are tied and the index says almost nothing about which curve to choose.

Choices that change the reported value

A published C-index rests on decisions that are easy to miss when the value is carried into a model.

1. Direction of the score. The vignette explains that a higher Cox linear predictor implies shorter survival, while in an accelerated failure time model a higher linear predictor implies longer survival. A hand calculation from Cox predictions without reversing the order swaps the concordant and discordant counts and returns one minus the correct value, below 0.5 for any useful model.
2. Stratification. Where survival curves for different strata may cross, the vignette compares pairs only within each stratum and adds up the counts.
3. Time horizon. The vignette argues that a model should be evaluated over the time range in which it will be used, and that reaching too far into the future may return an overly pessimistic C.
4. Estimator. Harrell's, Uno's and Gönen and Heller's estimators answer slightly different questions, so values are comparable only when the method and horizon match.
5. Optimism. Harrell, Lee and Mark state that predictive discrimination and calibration should be validated using bootstrapping or cross-validation before predictions are used in a new data series. In Royston and Altman's breast cancer example, Harrell's c-index was 0.673 in the derivation data and 0.658 in the validation data.

Misreadings of the C-index

The index answers one narrow question about ranking. Most errors come from asking more of it.

A high C-index is not evidence of accurate survival predictions. Steyerberg and colleagues note that, as a rank order statistic, the c statistic is insensitive to systematic errors in calibration such as differences in average outcome. The Brier score and calibration plots address that gap, and the worked example shows how far apart two equally concordant models can be in QALYs.

The C-index describes pairs, not patients. A value of 0.85 means that 85% of usable pairs are ordered correctly, counting ties as half.

Values below 0.5 are possible, and values just above it are weak. The vignette notes that concordance can fall below one half and describes values of 0.5 to 0.55 as not very impressive, because some pairs are obvious to order. Royston and Altman call a c-index below 0.7 modest in absolute terms, while noting that even a weakly stratifying model may be better than no model.

Small changes can hide significant predictors. In the Framingham example, adding HDL cholesterol, statistically highly significant for cardiovascular risk over 10 years, raised Uno's 10-year C-statistic by only 0.01, with a confidence interval of minus 0.00 to 0.02. The vignette also shows that adding a variable can lower the concordance.

Values from studies with different follow-up are not directly comparable. Because Harrell's estimate depends on the censoring pattern, two cohorts can report different values for reasons unrelated to model quality. A model validation report for an economic evaluation should state the estimator, the horizon and the amount of censoring alongside the number.

Sources

• Harrell FE Jr, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982;247(18):2543-2546. https://pubmed.ncbi.nlm.nih.gov/7069920/
• Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine. 1996;15(4):361-387. https://pubmed.ncbi.nlm.nih.gov/8668867/
• Latimer N. NICE DSU Technical Support Document 14: Survival analysis for economic evaluations alongside clinical trials, extrapolation with patient-level data. Sheffield: Decision Support Unit; June 2011, last updated March 2013. Executive summary and sections 2 and 3. https://www.sheffield.ac.uk/nice-dsu/tsds/survival-analysis
• Royston P, Altman DG. External validation of a Cox prognostic model: principles and methods. BMC Medical Research Methodology. 2013;13:33. https://pmc.ncbi.nlm.nih.gov/articles/PMC3667097/
• Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://pmc.ncbi.nlm.nih.gov/articles/PMC3575184/
• Therneau T, Atkinson E. Concordance. Vignette for the survival package for R, version dated 9 September 2026. https://cran.r-project.org/web/packages/survival/vignettes/concordance.pdf
• Uno H, Cai T, Pencina MJ, D'Agostino RB, Wei LJ. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine. 2011;30(10):1105-1117. https://pmc.ncbi.nlm.nih.gov/articles/PMC3079915/