Non-Parametric Estimation

Non-Parametric Estimation of Survival Curves and Mean Costs

Non-parametric estimation recovers a whole distribution, such as the survival curve of a trial arm or the spread of patient costs, directly from the observed sample, instead of picking a family such as the Weibull and estimating its two or three parameters. In health economics it supplies the Kaplan-Meier curve against which survival extrapolations are judged, the area under the observed curve that gives a restricted mean, and the resampled data sets behind bootstrap intervals for costs and incremental cost-effectiveness ratios. Its value is fidelity to the data within follow-up, and its limit is that it says nothing beyond the last observation. Economic models therefore usually rely on a parametric curve fitted to the trial data for the years after follow-up, with the non-parametric estimate as the benchmark for the observed period.

What makes an estimator non-parametric

A parametric estimator assumes the data come from a distribution indexed by a short list of numbers and estimates those numbers, which is the approach of parametric estimation. A non-parametric estimator treats the unknown as a function and lets the data fix its shape. Aalen, writing on counting processes, put the condition plainly, calling his model nonparametric when the unknown rate function in the intensity is allowed to vary arbitrarily apart from regularity conditions.

The simplest case, with no censoring, is the empirical distribution function, which places equal weight on each observed value.

$$\hat{F}_n(x) = \frac{1}{n} \sum_{i=1}^{n} I(X_i \le x)$$

where $\hat{F}_n(x)$ is the estimated probability that a value is at most $x$, $n$ is the number of patients, $X_i$ is the value observed for patient $i$ and $I(\cdot)$ equals 1 when the condition holds and 0 otherwise.

Any summary of the population distribution can then be estimated by computing the same summary on $\hat{F}_n$. The mean of the empirical distribution is the ordinary sample mean, so the arithmetic mean cost per patient in a trial arm is itself a non-parametric estimate. Barber and Thompson stress that health care decisions need arithmetic mean costs, because these reflect the total cost of treating all patients with the disease, however skewed the cost distribution is.

Censoring and the product-limit estimator

Trial follow-up ends before every patient has died or progressed, so the empirical distribution of observed times is biased once some times are censoring times. Kaplan and Meier set out to estimate the proportion of a population whose lifetimes would exceed a given time without making any assumption about the form of that function. Their product-limit estimate is, in their words, the distribution unrestricted as to form that maximises the likelihood of the observations, and it reduces to the observed proportion of survivors when no losses occur before that time.

$$\hat{S}(t) = \prod_{j:\, t_j \le t} \left( 1 - \frac{d_j}{n_j} \right)$$

where $\hat{S}(t)$ is the estimated probability of remaining event free beyond time $t$, $t_j$ are the distinct times at which events occur, $d_j$ is the number of events at $t_j$ and $n_j$ is the number of patients still at risk just before $t_j$.

The same risk sets give the Nelson-Aalen estimate of the cumulative hazard, $\hat{H}(t) = \sum_{j:\, t_j \le t} d_j / n_j$, with the symbols defined as above. Aalen describes his estimator as a generalisation of the empirical cumulative hazard rate of survival analysis, related to the product-limit estimator. The two estimators have their own pages, the Kaplan-Meier estimator and the Nelson-Aalen estimator, and survival-specific practice such as group comparison, hybrid tails and reconstructed curves is covered under non-parametric survival.

The freedom from a distributional form is bought with another assumption. Kaplan and Meier assumed that lifetime is independent of the potential loss time and noted that in practice this deserves careful scrutiny. NICE Decision Support Unit Technical Support Document 14 (TSD 14) likewise states that the survival methods it discusses are suitable only if censoring is uninformative, so independent censoring is the condition on which the whole estimate rests.

Restricted means from the step curve

A cost-effectiveness model needs mean time in each health state, and mean survival is the area under the survival curve. TSD 14 notes that when some patients are still alive at the end of follow-up, only a mean restricted to that time can be estimated directly from the Kaplan-Meier curve.

$$\hat{\mu}(\tau) = \int_{0}^{\tau} \hat{S}(t)\, dt$$

where $\hat{\mu}(\tau)$ is the estimated restricted mean survival time, $\tau$ is the restriction time, usually the end of follow-up, and $\hat{S}(t)$ is the Kaplan-Meier estimate. Because $\hat{S}(t)$ is a step function, the integral is a sum of rectangles.

Costs, censored costs and the bootstrap

The same principle reaches cost data in two ways. Lin and colleagues showed that when survival times are censored, the naive average of observed costs, whether over all patients or uncensored cases only, can be severely biased. One of their two estimators splits follow-up into short intervals and weights interval costs by Kaplan-Meier probabilities.

$$\hat{\mu}_C = \sum_{k=1}^{K} \hat{S}(a_k)\, \hat{C}_k$$

where $\hat{\mu}_C$ is the estimated mean total cost, $K$ is the number of intervals, $a_k$ is the start of interval $k$, $\hat{S}(a_k)$ is the Kaplan-Meier probability of being alive at $a_k$ and $\hat{C}_k$ is the estimated mean cost during interval $k$ among patients alive at its start. Lin and colleagues report that the estimator is consistent if censoring occurs only at interval boundaries.

The second route is bootstrapping, which treats the empirical distribution as the population and resamples patients from it. Briggs, Wonderling and Mooney described non-parametric bootstrap confidence intervals for the ICER whose advantage is that they do not depend on parametric assumptions about its sampling distribution. They found percentile intervals promising but warned that bootstrap estimates of bias and standard error may be unstable. Barber and Thompson recommend bootstrap methods for comparing arithmetic mean costs in moderately sized samples of highly skewed data.

Worked example with eight patients and two censored follow-ups

All numbers are illustrative. A small single-arm study follows eight patients for up to 12 months. Deaths occur at 2, 5, 5, 9 and 11 months, one patient is lost to follow-up at 3 months, one at 7 months, and one is alive at the 12-month cut-off.

1. Product-limit estimate. At 2 months 8 are at risk and 1 dies, so the estimate is 7/8 = 0.8750. At 5 months 6 are at risk and 2 die, giving 0.8750 × 4/6 = 0.5833. At 9 months 3 are at risk and 1 dies, giving 0.5833 × 2/3 = 0.3889. At 11 months 2 are at risk and 1 dies, giving 0.3889 × 1/2 = 0.1944, which holds to 12 months.

2. Naive alternatives. Counting both lost patients as survivors gives 3/8 = 0.375 alive at 12 months. Dropping them gives 1 survivor among the 6 patients whose status is known, or 0.167. The product-limit value of 0.194 lies between these two biased figures.

3. The estimate as reweighted data. Each patient starts with weight 1/8 = 0.125. When the patient lost at 3 months leaves, that weight is shared equally among the 6 patients still under observation, raising each to 0.125 + 0.125/6 = 0.1458. When the patient lost at 7 months leaves, that weight is shared among the remaining 3, raising each to 0.1458 + 0.1458/3 = 0.1944. Survival at 12 months is then 1 − 0.1250 − 0.2917 − 0.3889 = 0.1944, the product-limit value. Without censoring the weights stay at 1/8 and the result is the empirical distribution function.

4. Cumulative hazard. The Nelson-Aalen estimate at 12 months is 1/8 + 2/6 + 1/3 + 1/2 = 1.2917, and exp(−1.2917) = 0.275. In a sample this small the two estimators disagree noticeably at the tail.

5. Restricted mean. The area under the step curve to 12 months is the sum of five rectangles. They are 1 × 2 = 2.000, 0.8750 × 3 = 2.625, 0.5833 × 4 = 2.333, 0.3889 × 2 = 0.778 and 0.1944 × 1 = 0.194, which add to 7.931 months.

Choices that move the estimate

Although no distribution is chosen, the analyst still makes decisions that change the result. The time origin, the event definition and the restriction time all alter a restricted mean. The tail of the curve rests on few patients at risk, and in the example the single death at 11 months halves the estimate.

Common misreadings

Most errors with non-parametric estimates come from confusing them with rank tests or from forgetting the conditions they rest on. Three recur in trial-based economic work.

1. Treating a rank test as non-parametric estimation of costs. Thompson and Barber explain that the Mann-Whitney U test compares whole distributions in shape and location, not arithmetic means. In a low back pain trial they describe, a mean cost difference of GBP 148 had a t test based 95% confidence interval of minus GBP 146 to GBP 442 (P = 0.32), while a Mann-Whitney test on the same data gave P = 0.02.
2. Reading the curve as assumption free. Independent censoring and a representative sample are still required.
3. Averaging observed costs in censored data. The naive mean is biased, as Lin and colleagues showed.

Sources

• Aalen O. Nonparametric inference for a family of counting processes. Annals of Statistics. 1978;6(4):701-726. https://doi.org/10.1214/aos/1176344247
• Barber JA, Thompson SG. Analysis of cost data in randomized trials: an application of the non-parametric bootstrap. Statistics in Medicine. 2000;19(23):3219-3236. https://pubmed.ncbi.nlm.nih.gov/11113956/
• Briggs AH, Wonderling DE, Mooney CZ. Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Economics. 1997;6(4):327-340. https://pubmed.ncbi.nlm.nih.gov/9285227/
• Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association. 1958;53(282):457-481. https://doi.org/10.1080/01621459.1958.10501452
• Latimer N. NICE DSU Technical Support Document 14: Survival analysis for economic evaluations alongside clinical trials, extrapolation with patient-level data. Decision Support Unit; June 2011, last updated March 2013. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD14-Survival-analysis.updated-March-2013.v2.pdf
• Lin DY, Feuer EJ, Etzioni R, Wax Y. Estimating medical costs from incomplete follow-up data. Biometrics. 1997;53(2):419-434. https://doi.org/10.2307/2533947
• Thompson SG, Barber JA. How should cost data in pragmatic randomised trials be analysed? BMJ. 2000;320(7243):1197-1200. https://doi.org/10.1136/bmj.320.7243.1197