Statistical Process Control

Statistical Process Control in Health Service Monitoring

Statistical process control (SPC) is the way many health services watch a measure month by month, such as mean length of stay or the rate of hospital-acquired infection, and decide whether a movement is the ordinary noise of an unchanged system or evidence that something has changed. Charts with limits worked out from the measure's own past behaviour, and rules for unusual patterns, make that decision. Its economic case is that both possible mistakes use resources, since chasing noise takes staff time while a missed deterioration lets excess cost and harm build up month after month. NHS guidance promotes SPC charts for reports at every level up to the board in place of red, amber and green ratings, and a systematic review of its use in health care describes SPC as a key approach to quality improvement.

Two kinds of variation and the price of a false alarm

Walter Shewhart framed the approach in economic terms in his 1931 book, and his split between chance and assignable causes is covered under common cause variation. Where the limits sit is a decision about risk. The NIST/SEMATECH statistics handbook explains that the placement of the limits sets the risk of searching for an assignable cause when none exists, a risk management chooses. With limits three standard deviations from the centre line and a normally distributed statistic, a point falls outside by chance with probability 0.0027, so a stable process gives a false alarm on average once in about 370 points.

Building a monitoring system for a health service measure

An SPC system is more than a single chart, and its design rests on a handful of decisions made before any point is plotted. The chart families and their formulae are set out under control chart.

1. Fix the measure. The numerator, denominator and inclusion rules must stay the same throughout, or a change in coding will appear as a change in care.
2. Establish a baseline. NHS Improvement's Making Data Count guidance states that the most dependable charts are based on 15 to 20 time-series points and suggests a run chart, with a median line and no limits, for shorter series. The NIST handbook adds that monitoring can begin only if the process was in control when the baseline data were collected, and that otherwise the usual Phase 1 work has to be completed first.
3. Choose the chart and agree the rules in advance. The chart types and the four Making Data Count signal rules, with its advice to redraw limits only for a new system, are set out on the control chart page.

Charts that detect small shifts sooner

A Shewhart chart judges each point on its own, which makes it quick to flag a large jump and slow to notice a small sustained shift. The NIST handbook gives the average run length (ARL), the mean number of points plotted before a signal. For a three-sigma Shewhart chart it is 370.37 points when nothing has changed and 43.96 points after a shift of one standard deviation, so on monthly data a modest but lasting deterioration would, on average, go unflagged by the outside-limit rule for more than three and a half years. Run rules shorten that delay, but they also add false alarms.

The cumulative sum (CUSUM) chart accumulates small deviations from the in-control mean, so a run of slightly high values builds towards a signal. The handbook states that CUSUM charts are more efficient than Shewhart charts for shifts of two standard deviations or less, and its tabular form for detecting an increase is

$$S_t = \max\left(0,\ S_{t-1} + x_t - \mu_0 - k\right), \qquad S_0 = 0$$

where $S_t$ is the upper cumulative sum after period $t$, $x_t$ is the value observed in period $t$, $\mu_0$ is the in-control mean, $k$ is a reference value, and the chart signals when $S_t$ exceeds a decision limit $h$. A mirror-image sum detects decreases. The handbook's rule of thumb sets $k$ at half the shift to be detected and $h$ at around 4 or 5 standard deviations, and its table gives an ARL of 8.38 for a one standard deviation shift with $h$ of 4.

The exponentially weighted moving average (EWMA) chart works similarly, averaging past data with weights that shrink with age, set by a weighting factor that is usually between 0.2 and 0.3 according to the NIST handbook. In surgery the case mix changes over time, so a raw CUSUM can signal because more high-risk patients were referred. Steiner and colleagues addressed this with a risk-adjusted CUSUM that scores each patient's outcome against that patient's pre-operative risk, which makes the chart suited to settings with a varying mix of patients.

Worked example with an illustrative rise in length of stay

All numbers in this example are illustrative. A surgical ward discharges about 100 patients a month, and a 24-month baseline gives a stable mean length of stay of 5.0 days with a standard deviation of the monthly mean of 0.4 days. The Shewhart limits are therefore 5.0 plus or minus 3 × 0.4 = 1.2 days, which places them at 3.8 and 6.2 days. After a change in discharge arrangements the true mean rises by one standard deviation, to 5.4 days, and the ward runs a CUSUM alongside its Shewhart chart with $k$ = 0.5 × 0.4 = 0.2 days and $h$ = 4 × 0.4 = 1.6 days.

1. Check the post-change mean. The nine values sum to 48.6, so their mean is 48.6 / 9 = 5.4 days, one standard deviation above the baseline.

2. Shewhart outside-limit rule. The highest value is 5.9, below the upper limit of 6.2, so this rule gives no signal in nine months.

3. Shewhart run rules. Month 2 falls below the mean and breaks the run, and months 3 to 9 give seven consecutive points above the mean. A seven-point run rule signals at month 9 and a six-point rule at month 8. The outer third of the band lies above 5.0 + 0.8 = 5.8 days, and only month 6 falls in it, so the two-of-three rule does not fire.

4. CUSUM. The running sum reaches 1.0 + 0.7 = 1.7 at month 6, above the decision limit of 1.6, so the CUSUM signals at month 6.

5. Value the delay. The shift adds 0.4 × 100 = 40 bed-days a month. At an illustrative GBP 400 per bed-day this is 40 × 400 = GBP 16,000 a month of capacity. Signalling at month 6 instead of month 9 gives three months' notice, worth 3 × 16,000 = GBP 48,000 of capacity if the cause is found and removed promptly. Relying on the outside-limit rule alone, with its average run length of about 44 points, would put about 44 × 16,000 = GBP 704,000 of capacity at stake before the average signal.

6. Interpretation. The CUSUM warns earliest here, though a fair comparison of designs also needs their false-alarm rates when nothing has changed. The signal still only says that the mean has probably moved. The bed-day figure is an average cost, so the GBP values describe the opportunity cost of beds that could have been used for other patients, not cash that would be saved.

Comparing providers and handling over-dispersion

SPC ideas also apply across providers at a single point in time. Spiegelhalter recommended funnel plots for institutional comparisons, in which each provider's estimate is plotted against a measure of its precision and control limits form a funnel around the target, in close analogy to Shewhart charts, which avoids spurious ranking into league tables. Such comparisons feed benchmarking and indicators such as the standardised mortality ratio.

Spiegelhalter's funnel plots of NHS trust indicators, which often vary more than chance alone would predict, also show how the method chosen for this over-dispersion changes the banding a trust receives.

What SPC can and cannot tell an economic evaluation

SPC is mostly used for timely local decisions. Benneyan, Lloyd and Plsek note that traditional statistical methods account for natural variation but require measurements to be aggregated over time, which can delay decisions, whereas SPC combines time series analysis with graphical presentation that lay decision makers can follow. Making Data Count warns that a green rating can give false assurance while performance deteriorates.

An economic evaluation of a quality improvement programme needs an effect estimate, and here SPC is a weak tool. Fretheim and Tomic point out that judging an intervention from a chart requires a stable baseline, that comparing mean values before and after ignores secular trends, and that autocorrelation left unadjusted gives confidence intervals that are too narrow. In their constructed readmission example the u chart showed no special cause variation after the programme, while segmented regression found that a pre-programme rise of 0.87 percentage points a month reversed to a fall of 0.77. They recommend interrupted time series analysis whenever SPC data are used for impact assessment, which estimates the change in level and slope that can be costed.

The monitoring itself has a cost. Thor and colleagues, reviewing 57 studies published between 1990 and 2004, found that data collection and analysis can be time consuming and costly, and an economic case for SPC within continuous quality improvement should count analyst and clinician time alongside the expected value of earlier detection.

Limitations and common misreadings

Most errors with SPC come from treating the chart as more than a description of past variation. The four below each lead to wasted investigation or a missed change.

1. Reading stability as good performance. A process can be in control while consistently missing a clinical standard.
2. Treating a signal as a cause. The explanation may lie in case mix, coding or a concurrent change, as well as in the intervention under study.
3. Ignoring dependence and skew. Three-sigma limits assume roughly independent and symmetric data, and autocorrelated or highly skewed series give false-alarm rates that differ from the nominal figure.
4. Watching many charts with loose limits. The Thor review quotes a study warning that monitoring many units on individual charts, especially with two-sigma warning limits used as control limits, increases the false-alarm rate.

Sources

• Benneyan JC, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Quality and Safety in Health Care 2003;12(6):458-464. doi:10.1136/qhc.12.6.458
• Fretheim A, Tomic O. Statistical process control and interrupted time series: a golden opportunity for impact evaluation in quality improvement. BMJ Quality and Safety 2015;24(12):748-752. doi:10.1136/bmjqs-2014-003756
• NHS Improvement. Making data count: getting started. London: NHS Improvement; 2019. https://www.england.nhs.uk/wp-content/uploads/2019/12/making-data-count-getting-started-2019.pdf
• NIST/SEMATECH. e-Handbook of statistical methods, sections 6.3.1 (What are control charts?), 6.3.2.3 (CUSUM control charts), 6.3.2.3.1 (CUSUM average run length) and 6.3.2.4 (EWMA control charts). National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/pmc/section3/pmc3.htm (accessed 28 October 2026).
• Shewhart WA. Economic control of quality of manufactured product. New York: D. Van Nostrand Company; 1931.
• Spiegelhalter DJ. Funnel plots for comparing institutional performance. Statistics in Medicine 2005;24(8):1185-1202. doi:10.1002/sim.1970
• Spiegelhalter DJ. Handling over-dispersion of performance indicators. Quality and Safety in Health Care 2005;14(5):347-351. doi:10.1136/qshc.2005.013755
• Steiner SH, Cook RJ, Farewell VT, Treasure T. Monitoring surgical performance using risk-adjusted cumulative sum charts. Biostatistics 2000;1(4):441-452. doi:10.1093/biostatistics/1.4.441
• Thor J, Lundberg J, Ask J, Olsson J, Carli C, Pukk Härenstam K, Brommels M. Application of statistical process control in healthcare improvement: systematic review. Quality and Safety in Health Care 2007;16(5):387-399. doi:10.1136/qshc.2006.022194