Cook's Distance

A regression diagnostic identifying observations with disproportionately large influence on estimated coefficients, based on how much fitted values would change if removed.

Source: Cook 1977

Cook's distance identifies observations that exert an outsized influence on a regression's estimated coefficients, by measuring how much the fitted model would shift if a given point were removed. A large value marks a case that, on its own, pulls the results noticeably, whether because it is extreme in its predictors, its outcome, or both. Flagging such points prompts a check of whether they are errors, genuine outliers, or unduly shaping the conclusions. Spotting overly influential observations is its purpose. Kirkwood and Sterne (2003) describe this diagnostic.

Source: Kirkwood & Sterne 2003

Cook's distance for an observation is calculated from the change in all the fitted values that results from omitting that observation, scaled by the number of parameters and the error variance; equivalently, it combines the observation's residual with how far its predictor values lie from the others, so that a point with both a large residual and unusual predictor values receives a large Cook's distance. So Cook's distance is calculated as a summary of how much the fitted values shift when a case is removed, integrating the size of its residual and how extreme its predictor values are into one measure, which allows the overall influence of each observation on the regression to be quantified and compared.

Source: Cook 1977

Cook's distance is interpreted by its size relative to the other observations and to rough thresholds, with larger values indicating greater influence; observations with notably high Cook's distance are examined as potentially influential. Various rule-of-thumb cut-offs exist, but these are guides rather than strict rules. So Cook's distance is interpreted by flagging observations whose values are large compared with the rest, marking them as influential cases warranting scrutiny, and rather than automatically removing such points, the analyst investigates why they are influential, since a high Cook's distance signals that a single observation is strongly affecting the regression results.

Source: Cook 1977

Cook's distance is useful because a single influential observation can distort a regression, pulling the estimated coefficients and fitted values, and identifying such points allows the analyst to check whether the results depend unduly on a few cases. This supports robust and honest modelling. So Cook's distance is useful for detecting observations that exert excessive influence on a regression, prompting investigation of whether they are errors, unusual but valid cases, or points that unduly drive the conclusions, which helps ensure that a fitted model reflects the general pattern in the data rather than being dominated by a small number of influential observations.

Source: Cook 1977

Observations flagged by Cook's distance should be investigated rather than automatically deleted: the analyst checks whether they result from data errors, represent unusual but legitimate cases, or reveal model inadequacy, and considers the sensitivity of conclusions to their inclusion. Removal is justified only for genuine errors or with clear rationale. So influential observations identified by Cook's distance are examined to understand why they are influential and how much they affect the results, with the response depending on the cause, since discarding valid data merely because it is influential can bias the analysis, whereas retaining an erroneous point can distort it.

Source: Cook & Weisberg 1982