Variation

Variation in Health Systems and Health Economics

In health economics and health services research, variation describes how far a measured quantity such as admissions per head, cost per episode or survival differs between the units being compared, whether those units are patients, clinicians, hospitals, areas or months. Any statement about variation has to name what varies and across which units, because the same data can show wide differences between patients and almost none between hospitals. Health systems study it because a national average hides where money is spent, and because part of the spread may reflect care that could be changed. Only the share left after chance, measurement, need and informed preference have been accounted for is a candidate for policy.

Shewhart, Wennberg and two traditions of studying variation

Two older literatures shape how health economists read variation. Industrial quality control asks whether differences over time are routine noise or a sign of change, and small-area analysis asks whether differences between places reflect need or the way care is supplied.

Shewhart (1931) asked how much variation in manufactured product should be left to chance, and NHS Improvement's Making Data Count guide (2019) applies his split into expected (common cause) and unusual (special cause) variation to NHS performance data. The method is covered under common cause variation and statistical process control.

Wennberg and Gittelsohn (1973) used a population-based data system in Vermont to show wide variations in resource input, use of services and expenditure among neighbouring communities, which they read as evidence of considerable uncertainty about the effectiveness of health services. Wennberg (2002) later called variations unwarranted when they cannot be explained by type or severity of illness or by patient preferences, and sorted them into effective, preference-sensitive and supply-sensitive care. That branch, with its welfare analysis, is the subject of clinical practice variation.

The unit of comparison decides the question

Cost per patient within one hospital, mean cost across hospitals and monthly cost within a hospital need different methods. A national mean can conceal a long right tail of individual spending or a systematic gap between groups. The table sets out the common comparisons and the main reason each can mislead.

Statistical summaries of spread, such as the variance, the standard deviation and the interquartile range, describe how much values differ but not whether the difference is fair or avoidable. That judgement needs a standard, such as clinical need or the evidence on effectiveness.

Where an observed difference comes from

An observed difference between two hospitals or areas usually has several causes at once, and the policy response depends on which ones dominate. Chance comes first, because event counts fluctuate even when underlying rates are identical. Diehr and colleagues (1990) simulated the null hypothesis that small areas share one underlying surgery rate and found the expected variability surprisingly large, and worse for low-incidence procedures, smaller populations and repeat admissions.

Spiegelhalter (2005) described the opposite problem in NHS trust indicators. When an indicator shows substantially more variability than chance alone would produce, ignoring this over-dispersion can classify many institutions as abnormal. He proposed a random effects model, which the Healthcare Commission adopted for its star ratings.

Differences in patients come next. Rates and costs are standardised for age and sex and adjusted for case mix before providers are compared. Sutherland and Levesque (2020) point out that underadjustment overstates the room for improvement while overadjustment can adjust away modifiable factors that should be tackled, which is a live issue for any risk adjustment model. Some apparent variation is a coding artefact.

What remains reflects informed preference, clinical judgement and the supply of services, which clinical practice variation treats in depth.

Splitting variation within and between providers

Health service data are clustered, with patients nested in practices, hospitals or areas, so total variation can be split into a part within providers and a part between them. In a simple model the observed mean for a provider varies around the overall mean for two reasons, because true provider means differ and because each observed mean is estimated from a finite number of patients.

$$\operatorname{Var}(\bar{y}_j) = \sigma^2_u + \frac{\sigma^2_e}{n}, \qquad \rho = \frac{\sigma^2_u}{\sigma^2_u + \sigma^2_e}, \qquad \lambda = \frac{\sigma^2_u}{\sigma^2_u + \sigma^2_e / n}$$

where $\bar{y}_j$ is the observed mean outcome or cost for provider $j$, $\sigma^2_u$ is the variance of true provider means, $\sigma^2_e$ is the variance between patients within a provider, $n$ is the number of patients per provider, $\rho$ is the intraclass correlation (the share of total patient-level variance that lies between providers) and $\lambda$ is the reliability of a provider mean, the factor by which an observed deviation is multiplied when shrunk towards the overall mean.

In the hypothetical example of Merlo and colleagues (2005), an intraclass correlation of 0.08 meant that 8% of the total individual differences in systolic blood pressure lay at the neighbourhood level, which might reflect contextual factors or who lives where. The same split underlies hierarchical models of hospital costs and outcomes.

Worked example with cost per episode at four hospitals

All figures in this example are illustrative. Four hospitals each treat 100 patients for the same elective procedure. Their mean costs per episode are GBP 2,800, GBP 3,000, GBP 3,100 and GBP 3,500, and the standard deviation of cost between patients within each hospital is GBP 1,500.

Step 1 describes the observed spread. The overall mean is (2,800 + 3,000 + 3,100 + 3,500) / 4 = GBP 3,100. The deviations are minus 300, minus 100, 0 and 400, whose squares sum to 90,000 + 10,000 + 0 + 160,000 = 260,000. Dividing by three (one fewer than the number of hospitals) gives a variance of hospital means of 86,667, a standard deviation of about GBP 294.

Step 2 removes the chance part. Each hospital mean rests on 100 patients, so its sampling variance is 1,500 × 1,500 / 100 = 22,500, a standard error of GBP 150. Subtracting 22,500 from 86,667 leaves 64,167 as the estimated variance of true hospital means, a standard deviation of about GBP 253.

Step 3 finds the share between hospitals. Total patient-level variance is 64,167 + 2,250,000 = 2,314,167, so the intraclass correlation is 64,167 / 2,314,167 = 0.028. Under 3% of the variation in cost per episode lies between hospitals.

Step 4 shrinks the outlier. The reliability of a hospital mean is 64,167 / (64,167 + 22,500) = 0.740. The highest-cost hospital's observed excess of GBP 400 shrinks to 0.740 × 400 = GBP 296 per episode, or 296 × 100 = GBP 29,600 across its 100 patients, before any adjustment for case mix.

Step 5 interprets the result. A small intraclass correlation does not make the gap unimportant, because an excess that recurs each year is one a commissioner can act on, while variation between patients mostly reflects their condition. The GBP 296 is an unadjusted estimate of the hospital effect, and it can rise or fall once severity, coding and prices have been examined.

Variation, heterogeneity and uncertainty in economic evaluation

The ISPOR and SMDM modelling task force (Briggs and colleagues, 2012) separates stochastic uncertainty and heterogeneity from parameter uncertainty and structural uncertainty. Parameter uncertainty feeds probabilistic sensitivity analysis and shrinks with more data, which do not make patients more alike. The NICE manual (PMG36, section 4.9) asks for subgroup analysis where clinical and cost effectiveness may differ between patient groups, and views post hoc data dredging sceptically. Patient-level differences are treated under between-subject variation and differences in the effect of treatment under heterogeneous treatment effect.

How variation differs from nearby ideas

Variation in this sense is a descriptive property of data and becomes a quality or equity question only when it is compared with a standard. Clinical practice variation narrows it to clinical decisions about similar patients, with warranted and unwarranted variation as its two kinds. Health inequity adds the judgement that differences in health between groups are unfair. Quality of care supplies one of the standards against which provider variation is judged.

The word has a separate meaning in welfare economics. Compensating variation and equivalent variation are money measures of a change in welfare after a price or policy change, and they have no connection with the statistical spread discussed here.

Common misreadings

Most errors with variation come from treating the raw spread as if it already carried a verdict. The standard against which it is judged, and the share that chance explains, are usually missing.

1. Ranking providers without allowing for chance. League tables built on small numbers exaggerate differences, and the extreme ranks are the least reliable.
2. Treating all variation as waste. Sutherland and Levesque (2020) describe a clinician responding to a suddenly deteriorating patient as desirable and appropriate variation, and care that follows informed patient choice should differ between patients.
3. Adjusting until nothing is left. Controlling for a modifiable cause of poor care hides the variation a commissioner most needs to see.
4. Reacting to every movement in a monthly indicator. Making Data Count notes that common cause variation is reduced only by changing the process, while special cause variation needs its specific cause found and addressed.

Sources

• Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value in Health. 2012;15(6):835-842. https://doi.org/10.1016/j.jval.2012.04.014
• Diehr P, Cain K, Connell F, Volinn E. What is too much variation? The null hypothesis in small-area analysis. Health Services Research. 1990;24(6):741-771. https://pubmed.ncbi.nlm.nih.gov/2312306/
• Merlo J, Chaix B, Yang M, Lynch J, Råstam L. A brief conceptual tutorial of multilevel analysis in social epidemiology: linking the statistical concept of clustering to the idea of contextual phenomenon. Journal of Epidemiology and Community Health. 2005;59(6):443-449. https://doi.org/10.1136/jech.2004.023473
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Section 4.9. https://www.nice.org.uk/process/pmg36
• NHS Improvement. Making data count: getting started. London: NHS Improvement; 2019. https://www.england.nhs.uk/wp-content/uploads/2019/12/making-data-count-getting-started-2019.pdf
• Shewhart WA. Economic control of quality of manufactured product. New York: D. Van Nostrand Company; 1931. Chapters I and II.
• Spiegelhalter DJ. Handling over-dispersion of performance indicators. Quality and Safety in Health Care. 2005;14(5):347-351. https://doi.org/10.1136/qshc.2005.013755
• Sutherland K, Levesque JF. Unwarranted clinical variation in health care: definitions and proposal of an analytic framework. Journal of Evaluation in Clinical Practice. 2020;26(3):687-696. https://doi.org/10.1111/jep.13181
• Wennberg J, Gittelsohn A. Small area variations in health care delivery. Science. 1973;182(4117):1102-1108. https://doi.org/10.1126/science.182.4117.1102
• Wennberg JE. Unwarranted variations in healthcare delivery: implications for academic medical centres. BMJ. 2002;325(7370):961-964. https://doi.org/10.1136/bmj.325.7370.961