Benchmarking

Benchmarking Health Care Providers to Find and Adopt Better Practice

Benchmarking in health care sets a hospital, practice or service against peers, a standard or its own past results, then uses the gap to learn how better performers achieve their results and to change practice. Health economists meet it as a tool for productivity and efficiency review, as the logic behind paying providers by reference to the costs of others, and as a quality improvement intervention whose effects and costs can be evaluated.

From industrial cost comparison to quality improvement

A review of the benchmarking literature by Ettorchi-Tardy and colleagues dates industrial benchmarking to the early 1930s and traces its modern, competitive form to Xerox, which in 1979 found that Japanese competitors were selling their products at Xerox's own production cost and began what it called competitive benchmarking. The method spread through industry in the 1980s and was later reframed as a form of continuous quality improvement. The review dates structured benchmarking in health care to the mid-1990s, when it emerged in the United States and the United Kingdom to compare hospital outcomes and rationalise their funding.

The authors stress that benchmarking is often reduced to comparing indicators, whereas the full method also includes analysing the processes behind high performance, visits between organisations and acting on what is learned. Comparing indicators is one step in the cycle, necessary but not sufficient.

Why comparison matters where prices are missing

Most providers of publicly funded care face no market test of their costs, so a payer cannot easily tell whether a high cost reflects sicker patients or waste. Shleifer's theory of yardstick competition addresses this. A regulated monopoly has little incentive to cut costs, but if the price it receives depends on the costs of comparable firms, each firm in equilibrium chooses a socially efficient level of cost reduction. Shleifer notes that the mechanism extends to firms with observable differences and that Medicare's prospective payment of hospitals by diagnosis-related groups is very similar to the scheme he describes. Benchmarking therefore sits behind activity-based payment as well as improvement work, and the quality of the comparison sets the fairness of the payment.

Types of benchmarking in health care

Ettorchi-Tardy and colleagues describe how the method has moved from quantitative comparison of outcomes towards qualitative study of processes. The types below differ in who is compared and what is compared.

A second distinction runs across these types. Performance benchmarking compares results, such as costs, mortality or waiting times, and locates a gap. Process benchmarking examines how the work is done, such as staffing, pathways and scheduling, to find a mechanism that explains the gap. The review notes that external comparison of clinical practice with other sectors is hard because clinical indicators are specific to medicine, while non-clinical processes such as billing and stock control can be compared with other industries.

The benchmarking cycle

The review places benchmarking within Deming's cycle of quality improvement and reproduces the nine-step model of Pitarelli and Monnier. Its authors advise preparing the process before analysis starts, deciding what to compare, against whom and how the data will be collected.

1. Select the object. Choose the service or activity to be improved.
2. Identify partners. Choose the reference points for comparison.
3. Collect internal data. Gather and organise the organisation's own data.
4. Measure the gap. Compare against external data.
5. Set targets. Agree future performance objectives.
6. Communicate results. Share the findings with those who will act.
7. Develop action plans. Decide what will change.
8. Act. Implement the changes through project management.
9. Monitor progress. Track the indicators over time.

The reference point chosen at step 2 can be a peer average, the best observed performer, an agreed standard or the organisation's own history. Each answers a different question, and a peer group chosen after seeing the results can make almost any provider look acceptable.

Data, adjustment and uncertainty

Benchmarking in health care rests on indicators of structure, process and outcome, and its conclusions are only as good as their definitions. Two providers can report the same label, such as a 30-day readmission rate, while counting different patients, exclusions or time windows. Every indicator needs a stated numerator, denominator, inclusion rules and data source before results are compared.

Outcomes and costs depend on case mix, so raw rates penalise providers that treat sicker patients. Risk adjustment estimates the expected number of events for each provider's patients, and the ratio of observed to expected events, as in the standardised mortality ratio, gives a like-for-like comparison. Adjustment should not remove differences that reflect unequal access or poor care, since those are what the comparison is meant to find.

Rankings and league tables imply more precision than the data hold, because small providers swing widely by chance. Funnel plots instead show each provider's result against its volume, with control limits that narrow as volume rises, and are often used to flag outliers in feedback to hospitals. Kuhrij and colleagues, using the Dutch Acute Stroke Audit, found that a funnel plot of median door-to-needle time flagged 14 of the 58 hospitals that looked average on a binary delayed-treatment measure, so the choice of indicator also decides who is flagged. The same reasoning underlies statistical process control, which separates common-cause from special-cause variation over time.

Benchmarking productivity in the Carter review

The review of English non-specialist acute hospitals led by Lord Carter, published in February 2016, is a large-scale example of benchmarking for efficiency. It used metrics and benchmarks across clinical staff, medicines, diagnostics, procurement, back-office functions and estates, and concluded that there was significant unwarranted variation in every main resource area and that no one hospital was good at everything. It estimated the opportunity at 5 billion GBP, at least 9% of the 55.6 billion GBP spent by these hospitals.

To compare output across trusts the review built the weighted activity unit (WAU), where one WAU is a quantity of clinical activity equivalent to the cost of the average elective inpatient stay, 3,500 GBP. Its illustration showed one trust spending 3,830 GBP per WAU against a national average of 3,500 GBP and another trust spending 3,160 GBP. The review recommended that NHS Improvement develop a model hospital and its metrics as one source of data, benchmarks and good practice, and it warned that no single metric is perfect. Cost per unit of output is a ratio benchmark; frontier methods such as data envelopment analysis and stochastic frontier analysis instead estimate technical efficiency against an estimated best-practice frontier, and the efficiency index page compares the two.

What the evidence says about effects

Willmington and colleagues systematically reviewed studies that assessed the relationship between benchmarking and quality of care over time and found 17. All reported a positive association with improvement, in processes (10 studies), patient outcomes (13) or both (7). In 12 studies at least one complementary intervention accompanied the comparison, such as meetings between participants, improvement plans or financial incentives, and 10 combined several. Every study came from a high-income country.

The evidence is weaker than the unanimous direction suggests. The authors note that almost all studies were observational and open to bias, that gains in the five studies followed for 2 years or less may reflect the novelty of the initiative rather than lasting change, and that a causal link between benchmarking and quality remains hard to establish. Where the comparison is fed back to clinicians, the trial evidence on audit and feedback applies. In the 2025 version of the Cochrane review, low-confidence meta-regression evidence linked larger effects to feedback that compared performance with top peers or a benchmark rather than the average, and it cites a trial in which an achievable benchmark, the mean of the top 10% of peers, improved diabetes care more than feedback of the peer mean. For an economic evaluation, benchmarking is best treated as an intervention with its own data, staff and visit costs, judged against the change in practice it produces.

Worked example with the same rate and a different signal

The figures in this example are illustrative. Two hospitals each have a raw 30-day readmission rate of 7.0%. In one year, Hospital A has 84 readmissions among 1,200 discharges, and its case-mix model predicts 72. Hospital B has 336 readmissions among 4,800 discharges, with 288 expected. Both have the same ratio of observed to expected events.

$$\frac{O_A}{E_A} = \frac{84}{72} = 1.17, \qquad \frac{O_B}{E_B} = \frac{336}{288} = 1.17$$

where $O$ is observed readmissions and $E$ is expected readmissions given case mix. If observed counts vary by chance as a Poisson count with mean $E$, the standard error of the ratio is about one divided by $\sqrt{E}$, and approximate 95% limits around a ratio of 1 are as follows.

$$L = 1 \pm \frac{1.96}{\sqrt{E}}$$

where $L$ is the lower or upper control limit and 1.96 is the standard normal value for 95% coverage. For Hospital A, the square root of 72 is 8.49 and 1.96 divided by 8.49 is 0.23, giving limits of 0.77 to 1.23. For Hospital B, the square root of 288 is 16.97 and 1.96 divided by 16.97 is 0.12, giving limits of 0.88 to 1.12.

Hospital A's ratio of 1.17 lies inside its limits, so its excess is consistent with chance. Hospital B's identical ratio lies outside its narrower limits and is a signal worth investigating, starting with coding, the case-mix model and discharge processes before any judgement on quality. Hospital B has 336 minus 288, or 48, more readmissions than expected. At an illustrative 2,500 GBP per readmission, closing the gap would avoid at most 48 multiplied by 2,500, or 120,000 GBP a year in readmission costs. A programme costing more than that would need health gains from fewer readmissions to justify it.

Boundaries with related concepts

Benchmarking is the whole cycle of comparison, learning and change. A single act of setting performance against a reference is a benchmark comparison, and setting the reference itself is benchmark development; a benchmark price is a spending target used in payment, not a quality comparator. Audit and feedback returns performance data to clinicians, often using a benchmark as the comparator. Clinical audit measures practice against an explicit standard rather than against peers. Frontier efficiency analysis is a statistical way to choose the benchmark, not a substitute for the learning steps.

Where benchmarking misleads

Most failures come from treating a number as a diagnosis. A gap from a benchmark shows where to look, and the cause still has to be found.

• A gap is not a cause. An outlier can reflect case mix, coding, chance or genuine differences in care, and only investigation tells them apart.
• The best performer may not be a feasible target. Context, scale and patient mix may not transfer.
• Copying a visible practice can fail. Process benchmarking aims at the mechanism behind a result, which may not travel with the practice.
• Measured targets invite gaming. When payment or reputation depends on a benchmark, providers may select patients, change coding or neglect what is not measured.
• Activity per unit of cost is not productivity. Cost per unit of output ignores quality and outcomes unless they are built into the measure.
• Composite scores can hide trade-offs. A strong result on one indicator can offset a serious weakness in another, so component results should stay visible.
• Positive studies are not proof of effect. The evidence base is small, observational and drawn from high-income settings.

Sources

• Ettorchi-Tardy A, Levif M, Michel P. Benchmarking: a method for continuous quality improvement in health. Healthcare Policy 2012;7(4):e101-e119. https://pmc.ncbi.nlm.nih.gov/articles/PMC3359088/
• Willmington C, Belardi P, Murante AM, Vainieri M. The contribution of benchmarking to quality improvement in healthcare. A systematic literature review. BMC Health Services Research 2022;22:139. doi:10.1186/s12913-022-07467-8. https://pmc.ncbi.nlm.nih.gov/articles/PMC8812166/
• Shleifer A. A theory of yardstick competition. RAND Journal of Economics 1985;16(3):319-327. https://econpapers.repec.org/RePEc:rje:randje:v:16:y:1985:i:autumn:p:319-327
• Lord Carter of Coles. Operational productivity and performance in English NHS acute hospitals: unwarranted variations. Department of Health, February 2016. https://www.gov.uk/government/publications/productivity-in-nhs-hospitals
• Ivers N, Yogasingam S, Lacroix M, Brown KA, Antony J, Soobiah C, et al. Audit and feedback: effects on professional practice. Cochrane Database of Systematic Reviews 2025;3:CD000259. doi:10.1002/14651858.CD000259.pub4. https://pmc.ncbi.nlm.nih.gov/articles/PMC11934852/
• Kuhrij LS, van Zwet EW, van den Berg-Vos RM, Nederkoorn PJ, Marang-van de Mheen PJ. Enhancing feedback on performance measures: the difference in outlier detection using a binary versus continuous outcome funnel plot and implications for quality improvement. BMJ Quality and Safety 2021;30(1):38-45. doi:10.1136/bmjqs-2019-009929. https://pmc.ncbi.nlm.nih.gov/articles/PMC7788228/