Observed-to-Expected Ratio

Observed-to-Expected Ratio in Hospital Performance and Payment

An observed-to-expected ratio tells a health system whether a hospital or surgeon recorded more or fewer adverse events than its own patients would predict if they had been treated at the average standard of a reference population. The numerator counts what happened, and the denominator is a model-based count of what average care would have produced, usually the sum of each patient's predicted risk. Health economists meet the ratio whenever outcomes or payment adjustments are compared across providers, because a crude rate mostly reflects who comes through the door. The same arithmetic sits behind NHS mortality indicators and the infection ratios used to cut Medicare payments. Small expected counts make chance look like performance, so the confidence limits and the risk model matter as much as the point estimate.

Building the expected count from a risk model

The expected count answers a counterfactual question. The Quality Indicator Empirical Methods of the Agency for Healthcare Research and Quality (AHRQ) describe the comparison as the rate of adverse events a hospital would show if it provided the average level of care observed in the reference population to the patients it actually treated. A model, typically a logistic regression for a binary outcome, is fitted to the reference population and then applied to each patient at the provider.

$$\text{O/E}_h = \frac{O_h}{E_h}, \qquad E_h = \sum_{i=1}^{n_h} \hat{p}_i$$

where $\text{O/E}_h$ is the observed-to-expected ratio for provider $h$, $O_h$ is the number of patients at provider $h$ who had the event, $E_h$ is the expected number of events, $n_h$ is the number of patients at provider $h$ and $\hat{p}_i$ is the predicted probability of the event for patient $i$ from the reference model.

The older form uses stratum rates instead of a regression. Jarman and colleagues, studying English hospital death rates, defined the indirectly standardised mortality ratio as actual over expected deaths multiplied by 100, applying national death rates for each age, sex and primary diagnosis stratum to the hospital's admissions.

Because the model is fitted to the reference population, the reference population's own ratio is 1 by construction. Van Calster and colleagues note the same for prediction models, where at internal validation the average of predicted risks will match the event rate. A provider's ratio is therefore always relative to the standard of care in the data used to fit the model, and when the model is refitted each year a whole system can improve while the average ratio stays at 1.

Reading the ratio for adverse and favourable outcomes

A ratio of 1 means the provider had exactly as many events as its case mix predicts. For an adverse outcome such as death or readmission, AHRQ states that a ratio below 1 gives reason to think the hospital or area is performing better than average. The direction reverses for a favourable outcome, such as discharge home or a gain in a patient-reported score, where a ratio above 1 is good news, so the outcome must always be named alongside the ratio. AHRQ also converts the ratio back into a rate on a familiar scale.

$$R_h = \frac{O_h}{E_h} \times \bar{y}_{\text{ref}}$$

where $R_h$ is the risk-adjusted rate for provider $h$, $O_h$ and $E_h$ are the observed and expected counts defined above and $\bar{y}_{\text{ref}}$ is the observed event rate in the reference population. AHRQ describes this rate as the indirectly standardised ratio multiplied by the reference population rate, so it adds no information beyond the ratio.

Uses in health systems and economic evaluation

Mortality reporting is the best known use, and the standardised mortality ratio page covers the mortality indicators in depth. Payment programmes use the same structure for other events. In the Medicare Hospital-Acquired Condition Reduction Program, the Centers for Medicare and Medicaid Services (CMS) scores five healthcare-associated infection measures with the standardised infection ratio (SIR) of observed to predicted infections that the Centers for Disease Control and Prevention calculates. The fiscal year 2027 fact sheet states that hospitals with a Total HAC Score in the worst-performing quartile receive a 1% payment reduction on their Medicare fee-for-service payments. An expected count in this setting is a price signal as well as a statistic.

Costs follow the same logic. A cost index divides a provider's actual costs by the cost expected for its activity at national average unit costs, as described under case-mix adjustment, and an efficiency index builds on that comparison.

Decision modellers use the ratio to check a model against data. Van Calster and colleagues describe mean calibration, or calibration-in-the-large, as comparing the average predicted risk with the overall event rate, and note that a mismatch between predicted and observed risks is common at external validation. In model validation, the ratio of observed to model-predicted events in an independent dataset shows whether a model over- or underpredicts, while adjusting inputs until the two agree is calibration.

Chance variation and confidence limits

An observed count varies by chance even when care is exactly average. Breslow and Day treat the observed number of deaths as approximately Poisson distributed with mean and variance both equal to the expected number under the null hypothesis. The width of the confidence interval is driven by the number of observed events.

Exact Poisson limits need iteration, but Byar's approximation is, in Breslow and Day's assessment, accurate even for quite small numbers of deaths. The approximation gives limits for the observed count, which are then divided by the expected count.

$$O_L = O\left(1 - \frac{1}{9O} - \frac{z}{3\sqrt{O}}\right)^3, \qquad O_U = (O+1)\left(1 - \frac{1}{9(O+1)} + \frac{z}{3\sqrt{O+1}}\right)^3$$

where $O$ is the observed count, $O_L$ and $O_U$ are the lower and upper limits for the underlying Poisson mean of that count, and $z$ is the standard normal percentile for the chosen level, 1.96 for 95%. The limits for the ratio are $O_L / E$ and $O_U / E$, with $E$ the expected count. In their bladder cancer example, 15 observed deaths against 8.33 expected give a ratio of 1.80 with exact 95% limits of 1.01 and 2.97, and Byar's formula gives almost the same result. The common shortcut based on the standard error of the log ratio puts the lower limit at 1.09, which they call seriously in error.

Funnel plots display many providers at once. Spiegelhalter recommends them for institutional comparisons, with control limits forming a funnel around the target in close analogy to Shewhart control charts, and concludes that they avoid spurious ranking into league tables. Public Health England's guide plots ratios against the expected count with exact Poisson limits, and notes that about 5 in every 100 observations lie outside the 2 standard deviation funnel even under expected random variation. The method belongs to the wider family of statistical process control.

Real indicators often vary more than Poisson chance allows. Spiegelhalter, analysing English NHS trust indicators, warned that ignoring such over-dispersion could lead to many institutions being inappropriately classified as abnormal, and favoured additive random effects. AHRQ's smoothed rate weights the provider's risk-adjusted rate against the reference rate by its reliability, pulling a small hospital towards the average. A hierarchical model is the general form of that shrinkage.

Worked example with three risk bands and a small hospital (illustrative)

All figures are illustrative. Hospital A admits 2,000 emergency patients in a year. A national model assigns 1,200 of them a predicted 30-day mortality of 1%, 600 a risk of 4% and 200 a risk of 12%. The national reference mortality rate is taken as 3.5%.

Step 1, expected deaths. Summing predicted risks band by band gives 1,200 × 0.01 = 12, 600 × 0.04 = 24 and 200 × 0.12 = 24, a total of 60.

Step 2, the ratio. Observed deaths are 6 + 22 + 26 = 54, so the ratio is 54 / 60 = 0.90. The crude death rate is 54 / 2,000 = 0.027, or 2.7%, against an expected rate of 60 / 2,000 = 0.030, or 3.0%, so the hospital treats a lower-risk mix and has fewer deaths than that mix predicts.

Step 3, risk-adjusted rate. Multiplying by the reference rate gives 0.90 × 3.5% = 3.15%, which restates the ratio on the rate scale, 10% below the 3.5% reference rate.

Step 4, Byar limits. For 54 deaths, $\frac{1}{9 \times 54} = 0.00206$ and $\frac{1.96}{3 \times \sqrt{54}} = 0.08891$, so the bracketed lower term is 1 minus 0.00206 minus 0.08891, or 0.90903, its cube is 0.75118 and $O_L = 54 \times 0.75118 = 40.56$. For the upper limit, $\frac{1}{9 \times 55} = 0.00202$ and $\frac{1.96}{3 \times \sqrt{55}} = 0.08810$, so the bracketed term is 1 minus 0.00202 plus 0.08810, or 1.08608, its cube is 1.28109 and $O_U = 55 \times 1.28109 = 70.46$. Dividing by 60 gives 95% limits of 0.68 and 1.17. The interval includes 1, so the 10% shortfall is compatible with average care.

Step 5, the high-risk band. Low-risk patients did better than predicted (6 / 12 = 0.50), though on few deaths, while high-risk patients did slightly worse (26 / 24 = 1.08). A summary ratio below 1 can coexist with poorer care for the sickest patients.

Step 6, a small hospital. Hospital B treats 300 patients with an expected 5.0 deaths and records 9, a ratio of 9 / 5.0 = 1.80. Byar's formula gives $O_L = 4.11$ and $O_U = 17.09$, and exact Poisson limits agree to two decimals, so the 95% limits are 4.11 / 5.0 = 0.82 and 17.09 / 5.0 = 3.42. A ratio that looks twice Hospital A's sits inside the funnel, and the data cannot separate poor care from chance.

Comparing one provider's ratio with another's

Each ratio compares a provider with the reference standard applied to its own patients. Two ratios therefore use different weights, set by each provider's mix, a problem the standardised mortality ratio page treats in full.

In the example, ratios of 0.90 and 1.80 do not show that Hospital B's mortality is twice Hospital A's for comparable patients. A direct comparison needs both hospitals' band-specific rates applied to a common mix, or a model that estimates provider effects jointly.

What can bias the expected count

The expected count depends on recorded patient characteristics, so recording practice feeds straight into the ratio. Thinner coding of comorbidity lowers the expected count and inflates the ratio, a bias that Mohammed and colleagues documented in four English hospitals and that the case-mix adjustment page covers in depth.

The reverse risk arises when payment depends on the ratio. Fuller or inflated recording of comorbidity raises the expected count and lowers the ratio without any change in care, so a programme that pays on the ratio also rewards coding. Variables shaped by the provider's own care, such as complications arising in hospital, should stay out of the model, because adjusting for them hides the performance being measured. Model specification is treated more fully under risk adjustment. A ratio is a trigger for investigation, and the residual after adjustment still mixes quality of care with unmeasured case mix, recording and chance.

Sources

• Agency for Healthcare Research and Quality. Quality Indicator Empirical Methods. AHRQ Quality Indicators; September 2019. https://qualityindicators.ahrq.gov/Downloads/Resources/Publications/2019/Empirical_Methods_2019.pdf
• Breslow NE, Day NE. Statistical Methods in Cancer Research. Volume II, The Design and Analysis of Cohort Studies. IARC Scientific Publications No. 82. Lyon: International Agency for Research on Cancer; 1986. Chapter 2. https://publications.iarc.who.int/download/SP82_vol2-2.pdf
• Centers for Medicare and Medicaid Services. Hospital-Acquired Condition (HAC) Reduction Program, Fiscal Year 2027 Fact Sheet. August 2026. https://www.cms.gov/files/document/hospital-readmissions-reduction-program-fiscal-year-2027-fact-sheet.pdf
• Centers for Medicare and Medicaid Services. Hospital-Acquired Condition Reduction Program, Scoring Methodology. Infographic, August 2026. https://www.cms.gov/files/document/hospital-readmissions-reduction-program-payment-reduction-methodology.pdf
• Jarman B, Gault S, Alves B, Hider A, Dolan S, Cook A, Hurwitz B, Iezzoni LI. Explaining differences in English hospital death rates using routinely collected data. BMJ. 1999;318(7197):1515-1520. https://doi.org/10.1136/bmj.318.7197.1515
• Mohammed MA, Deeks JJ, Girling A, Rudge G, Carmalt M, Stevens AJ, Lilford RJ. Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals. BMJ. 2009;338:b780. https://doi.org/10.1136/bmj.b780
• Public Health England. Technical Guide: Funnel Plots. Public Health Data Science; version 13 December 2017. https://fingertips.phe.org.uk/documents/PHDS%20Guidance%20-%20Funnel%20Plots.pdf
• Spiegelhalter DJ. Funnel plots for comparing institutional performance. Statistics in Medicine. 2005;24(8):1185-1202. https://doi.org/10.1002/sim.1970
• Spiegelhalter DJ. Handling over-dispersion of performance indicators. Quality and Safety in Health Care. 2005;14(5):347-351. https://doi.org/10.1136/qshc.2005.013755
• Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMC Medicine. 2019;17:230. https://doi.org/10.1186/s12916-019-1466-7