Cook's Distance

Cook's Distance in Health Cost and Outcome Regressions

Cook's distance gives each record in a fitted regression one number, showing how far the fitted model would move if that record were left out. In health economics this matters because cost data are skewed and a few very expensive patients can shift a mean cost or a cost-effectiveness result. A flagged patient calls for investigation before any deletion, and when the record is valid the usual advice is to report the analysis with and without that patient.

Unusual outcomes, unusual predictors and influence

Three properties of an observation are often confused. An outlier has an outcome far from its predicted value, so its residual is large. An observation with unusual predictor values, such as a patient with far more chronic conditions than anyone else, has a large hat value. Influence is a third idea, the extent to which the fitted model itself changes when the observation is removed.

The three need not coincide. Penn State's regression course shows an outlying outcome with ordinary predictor values, and an extreme predictor value that follows the trend, each barely moving the fitted line, while a point that is both changes the estimates substantially. NIST's regression diagnostics guidance adds that points with unusual predictor values tend to have small residuals, so residual checks alone often miss them. Cook's distance combines the two features in one deletion-based measure.

Two equivalent ways to compute the statistic

The statistic is named after R. Dennis Cook, whose 1977 paper in Technometrics introduced it. The definition compares the fitted values from the full ordinary least squares fit with those from the same model refitted without observation $i$. Squared differences are added across every observation and then scaled, as follows.

$$D_i = \frac{\sum_{j=1}^{n} \left(\hat{y}_j - \hat{y}_{j(i)}\right)^2}{p \, \mathrm{MSE}}$$

where $D_i$ is Cook's distance for observation $i$, $n$ is the number of observations, $\hat{y}_j$ is the fitted value for observation $j$ from the full model, $\hat{y}_{j(i)}$ is the fitted value for observation $j$ after observation $i$ has been deleted and the model refitted, $p$ is the number of estimated coefficients including the intercept and $\mathrm{MSE}$ is the residual mean square of the full model. Since fitted values are the design matrix times the coefficients, the same quantity is also a scaled distance between the full and the deletion coefficient vectors, as the Stata documentation describes it.

No refitting is needed in practice. For a linear model fitted by least squares, the result can be written with quantities from the full fit alone, the form printed by NIST, Penn State and Stata.

$$D_i = \frac{e_i^2}{p \, \mathrm{MSE}} \cdot \frac{h_{ii}}{(1 - h_{ii})^2}$$

where $e_i = y_i - \hat{y}_i$ is the ordinary residual of observation $i$, $y_i$ is its observed outcome and $h_{ii}$ is its hat value, the $i$th diagonal element of the hat matrix $X(X^{\top}X)^{-1}X^{\top}$ with $X$ the design matrix, lying between 0 and 1. The second factor grows steeply as $h_{ii}$ approaches 1, so a patient with a small residual can still have a large distance.

Why skewed cost data make influence a health economics question

Cost data in economic evaluation are usually skewed, with a minority of patients incurring very large costs. Thompson and Barber, writing in the BMJ on pragmatic trial costs, argue that despite this skew the arithmetic mean is the informative measure, because a total budget for treating all patients is the arithmetic mean multiplied by the number treated. Any single patient who moves the mean therefore moves the budget estimate as well.

A review of statistical methods for healthcare costs by Mihaylova and colleagues regards apparent cost outliers as part of the true distribution of costs. It recommends checking that answers are not unduly influenced by a few very large outlying costs, and cautions that log transformation and lognormal models may be non-robust to outliers. Cook's distance is a direct way to make that check for a regression.

The same reasoning extends to cost-effectiveness. Net-benefit regression, set out by Hoch, Briggs and Willan, uses the net benefit framework to place the cost-effectiveness problem within a standard regression, which opens it to established econometric techniques. When such a model is fitted by least squares, Cook's distance can show whether one participant's costs or outcomes are moving the fitted model, including the estimated treatment effect.

Worked example with six patients and one complex case

The data in this example are illustrative. Six patients in a disease registry have between 0 and 10 chronic conditions, and annual healthcare cost in GBP thousand is regressed on the number of conditions by ordinary least squares. The sixth patient has 10 conditions, far more than anyone else, and costs GBP 9,000.

Step 1. Fit the full model. The mean number of conditions is $\bar{x} = 20/6 = 3.333$ and the mean cost is $\bar{y} = 27/6 = 4.5$. The sums of squares and cross-products about the means are $S_{xx} = 63.33$ and $S_{xy} = 44$, so the slope and intercept are as follows.

$$\hat{\beta}_1 = \frac{44}{63.33} = 0.6947, \qquad \hat{\beta}_0 = 4.5 - 0.6947 \times 3.333 = 2.184$$

where $\hat{\beta}_1$ is the estimated extra annual cost per additional condition in GBP thousand and $\hat{\beta}_0$ is the estimated cost for a patient with no conditions. Each additional condition adds about GBP 695 a year. The residual sum of squares is 0.9316 on $n - p = 4$ degrees of freedom, so $\mathrm{MSE} = 0.9316/4 = 0.2329$.

Step 2. Compute residuals, hat values and distances. For a model with one predictor the hat value is $h_{ii} = 1/n + (x_i - \bar{x})^2/S_{xx}$. The results for all six patients are shown below.

For patient 6 the hat value and distance are as follows.

$$h_{66} = \frac{1}{6} + \frac{(10 - 3.333)^2}{63.33} = 0.1667 + 0.7018 = 0.8684$$

where $h_{66}$ is the hat value of patient 6, 10 is that patient's number of conditions and 63.33 is $S_{xx}$.

$$D_6 = \frac{(-0.1316)^2}{2 \times 0.2329} \times \frac{0.8684}{(0.1316)^2} = \frac{0.8684}{2 \times 0.2329} = 1.864$$

where 0.1316 is both the absolute residual and $(1 - h_{66})$ for this patient, so the two cancel. Patient 6 has a small residual yet by far the largest distance. Patient 4 has the largest residual, 0.732, but a distance of only 0.280 because that patient's number of conditions is close to the mean.

Step 3. Check by refitting without patient 6. The remaining five patients give $\bar{x} = 2$, $\bar{y} = 3.6$, $S_{xx} = 10$ and $S_{xy} = 8$, so the slope becomes 0.8 and the intercept $\hat{\beta}_{0(6)} = 3.6 - 0.8 \times 2 = 2.0$. The reduced model predicts $\hat{y}_{6(6)} = 2.0 + 0.8 \times 10 = 10.0$ for patient 6, against an observed 9, so the full model had been pulled most of the way towards that patient. The squared changes in the six fitted values sum to 0.8684, and dividing by 0.4658 again gives 1.864.

Step 4. Interpret the result. Removing patient 6 raises the estimated cost of each additional condition from GBP 695 to GBP 800 a year and lowers the sample mean cost from GBP 4,500 to GBP 3,600. Patient 6 is also the only evidence about costs above four conditions, so without that patient any prediction for complex patients would rest on extrapolation of a line fitted to simpler cases. The estimate depends on one record, which is not the same as that record being wrong.

Reading the size of the value

Several cut-offs are in circulation, and they do not agree. Each is a screening rule for deciding which records to examine, not a test that a record is invalid.

• Penn State's course treats a value above 0.5 as worth further investigation, a value above 1 as quite likely influential, and a value that stands well apart from the rest as almost certainly influential.
• Stata's regress documentation, following Bollen and Jackman, suggests examining values above 4 divided by $n$, which for the example is 0.667.
• NIST and Penn State relate the value to the F distribution with $p$ and $n - p$ degrees of freedom. A value near or above the 50th percentile indicates substantial influence, and one below about the 10th or 20th percentile indicates little. In the example, 1.864 lies at about the 73rd percentile of F with 2 and 4 degrees of freedom.

Every rule flags patient 6 alone. In large data sets 4 divided by $n$ becomes tiny and flags many records, while a threshold of 1 may flag none. Within wider model diagnostics, the better test is whether the decision-relevant estimate moves when the records that stand apart are omitted.

What an analyst should do with an influential patient

A flagged record calls for investigation, with deletion only one outcome. Penn State's recommended strategy begins with errors. A data entry error should be corrected, a record from outside the intended study population or invalidated by a procedural error may be removed, and no point should be deleted merely because it does not fit the preferred model.

The next question is whether the model is wrong for the data. An omitted predictor, a missing interaction or an unmodelled nonlinear relation can make valid patients look influential; in the example, a cost curve that steepens with many conditions might fit patient 6 without strain. Any change of model should follow the scientific question, not a wish to make the diagnostic disappear.

When the patient is valid and the answer depends on that patient, Penn State recommends analysing the data with and without the point and reporting both, which in an economic evaluation is a sensitivity analysis on influential records. Dropping a genuinely expensive patient because the patient changes a cost-effectiveness conclusion would change the population the estimate describes.

Beyond ordinary least squares

The formulas above apply to ordinary least squares. Cost models are often fitted instead as a generalised linear model, commonly with a log link and a gamma specification, where the deletion idea has to be approximated. R's stats documentation states that its values for such models are approximations following Williams that can be poor when some cases have large influence, so a direct refit without the flagged patients is a sensible check.

Grouped data differ. Patients clustered within hospitals, as in a hierarchical model, can be influential as a group, and deleting one patient asks a different question from deleting a whole hospital. Single-case deletion has a related blind spot, since with two similar extreme patients, removing either alone leaves the other to hold the fit in place.

Cook's distance beside related diagnostics

Each regression diagnostic answers a narrower question, so Cook's distance is best read with the others. The table shows the division.

NIST describes DFBETA as similar to Cook's distance except that it assesses influence on each individual parameter rather than on the parameters as a whole. In a cost regression where only the treatment coefficient matters for the decision, DFBETAS for that coefficient is often the more relevant check, and Cook's distance serves as the overall screen.

Limitations and common misreadings

A large Cook's distance shows that the fitted model is sensitive to one record under its current specification. It does not show that the record is an error or should be excluded. Small distances for every patient do not establish a correct model either, since a wrong functional form, unmodelled clustering or selection bias can leave each distance modest.

The statistic is tied to the fitted model and sample, so computing it on different rows from the reported analysis gives misleading values. The rules of thumb have no calibrated error rates. They pick records for scrutiny, and the decision about each record rests on its provenance and its effect on the estimate that informs the decision.

Sources

• Cook RD. Detection of influential observation in linear regression. Technometrics. 1977;19(1):15-18. https://doi.org/10.1080/00401706.1977.10489493
• Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Economics. 2002;11(5):415-430. https://doi.org/10.1002/hec.678
• Mihaylova B, Briggs A, O'Hagan A, Thompson SG. Review of statistical methods for analysing healthcare resources and costs. Health Economics. 2011;20(8):897-916. https://doi.org/10.1002/hec.1653
• NIST. Dataplot Reference Manual, Regression Diagnostics. National Institute of Standards and Technology; last updated 2023. https://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/regrdiag.htm
• Penn State Eberly College of Science. STAT 501 Regression Methods, Lesson 11 Influential Points, sections 11.5 and 11.7. Accessed 10 October 2026. https://online.stat.psu.edu/stat501/Lesson11
• R Core Team. influence.measures, Regression Deletion Diagnostics. R package stats, version 4.6.1; 2026. https://search.r-project.org/R/refmans/stats/html/influence.measures.html
• StataCorp. regress postestimation, Postestimation tools for regress. Stata Base Reference Manual, Release 19. College Station, TX: Stata Press; 2025. https://www.stata.com/manuals/rregresspostestimation.pdf
• Thompson SG, Barber JA. How should cost data in pragmatic randomised trials be analysed? BMJ. 2000;320(7243):1197-1200. https://doi.org/10.1136/bmj.320.7243.1197