Harm Rate

Harm Rate in Hospital Patient Safety Measurement

A harm rate counts the injuries that patients suffer because of the care they receive and divides that count by a measure of exposure, most often patient days or admissions. Hospitals use it to track whether care is becoming safer, and the Institute for Healthcare Improvement (IHI) names adverse events per 1,000 patient days as its recommended measure for tracking the harm rate with the Global Trigger Tool. For health economists the rate matters because each harm uses beds and staff time and costs the patient health, so a change in the rate drives the value of a safety programme. The figure depends heavily on how harm is defined and found and on the denominator chosen, so two hospitals reporting the same label can be measuring different things.

What counts as harm

The IHI Global Trigger Tool, set out in a white paper by Griffin and Resar (second edition, 2009), defines harm as unintended physical injury resulting from or contributed to by medical care that needs additional monitoring, treatment or hospital care, or that results in death. An error that reaches the patient without injuring them is not counted, while an injury is counted whether or not anyone made a mistake.

Further rules shape what the rate means. The tool counts harm from the active delivery of care and excludes, as far as possible, harm from omitted care, so a brain haemorrhage after an anticoagulant counts while a stroke after untreated hypertension does not. Reviewers do not judge preventability, because the white paper argues that a definition shifting with views on what was preventable would make the measure meaningless over time. Harm present on admission is still counted if medical care caused it, since the measure is what the patient experienced.

Severity is graded with five harm categories of the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) index, adapted by IHI. Category E is temporary harm needing intervention, F is temporary harm needing initial or prolonged hospital stay, G is permanent harm, H is an intervention needed to sustain life, and the last category is death. Categories A to D describe errors without harm and are left out, so a quality of care programme tracks those errors with separate measures.

Finding harm with record review

Voluntary incident reporting depends on staff noticing and choosing to report, so it records a small and shifting fraction of events. The Global Trigger Tool instead has trained reviewers search closed inpatient records for triggers, such as a readmission within 30 days or the administration of naloxone, and then check whether harm occurred.

The white paper recommends a random sample of 10 adult inpatient records every two weeks, 20 a month, limited to stays of at least 24 hours and patients aged 18 or over. Each review is limited to 20 minutes and a physician makes the final judgement. The paper warns that a single reviewer will catch about 75% of the adverse events, and states that the tool was never intended to identify every adverse event.

Classen and colleagues applied three detection methods to the same records from three hospitals. The Global Trigger Tool found at least ten times more confirmed serious events than voluntary reporting and the Patient Safety Indicators of the Agency for Healthcare Research and Quality, and adverse events occurred in one third of admissions. A systematic review by Hibbert and colleagues found that incident reporting detected between 2% and 8% of the adverse events detected with the tool, so an adverse event reporting count is not a harm rate.

Three ways to express the rate

The white paper asks for the review data in three forms. The first two count every harm, so a patient with two harms contributes two events, while the third counts patients.

$$R_{D} = \frac{H}{D} \times 1000 \qquad R_{A} = \frac{H}{A} \times 100 \qquad P_{A} = \frac{A_{H}}{A} \times 100$$

where $R_{D}$ is harms per 1,000 patient days, $R_{A}$ is harms per 100 admissions, $P_{A}$ is the percentage of admissions with at least one harm, $H$ is the number of harms found in the sampled records, $D$ is the total patient days in those records, $A$ is the number of admissions reviewed and $A_{H}$ is the number of those admissions with one or more harms.

The two rates are linked through length of stay.

$$R_{A} = \frac{R_{D} \times L}{10}$$

where $L = D/A$ is the mean length of stay in days among the reviewed admissions. A hospital that shortens stays without changing care will see harms per admission fall, because a shorter stay offers less time for harm, so the average length of stay belongs beside the rate.

The white paper notes that the percentage of admissions with harm understates the number of events, because some patients have more than one, and so it is less sensitive to improvement. A British record review by Vincent and colleagues shows the gap, with 10.8% of 1,014 patients having an adverse event and an overall rate of 11.7% when multiple events were included.

Worked example of a hospital before and after a safety programme

All numbers in this example are illustrative. A hospital reviews 20 records a month for 12 months before a programme aimed at medication harm and pressure injury, and for 12 months after it.

1. Baseline rates. The 240 baseline admissions account for 1,200 patient days, so $L = 1200/240 = 5.0$ days. Reviewers find 108 harms in 78 of the admissions. The rate is $R_{D} = 108/1200 \times 1000 = 90$ harms per 1,000 patient days, $R_{A} = 108/240 \times 100 = 45$ harms per 100 admissions, and $P_{A} = 78/240 \times 100 = 32.5\%$. The link formula agrees, since 90 × 5.0 divided by 10 is 45.

2. Uncertainty at baseline. Treating the count as a Poisson variable, the exact 95% interval for 108 events is 88.6 to 130.4 events, which divided by 1,200 patient days gives 73.8 to 108.7 harms per 1,000 patient days.

3. Rates after the programme. The 240 later admissions account for 1,150 patient days, so $L$ is about 4.79 days. Reviewers find 81 harms in 60 admissions. The rate is $R_{D} = 81/1150 \times 1000 = 70.4$ harms per 1,000 patient days, with an exact interval of 55.9 to 87.5, while harms per 100 admissions fall to 33.75 and the percentage with harm to 25.0%.

4. Rate ratio. The incidence rate ratio compares the two periods.

$$IRR = \frac{H_{1}/D_{1}}{H_{0}/D_{0}} = \frac{81/1150}{108/1200} = 0.783$$

where $H_{0}$ and $D_{0}$ are harms and patient days before the programme and $H_{1}$ and $D_{1}$ are the same after it. The standard error of the log ratio is $\sqrt{1/108 + 1/81} = 0.147$, and the 95% interval is $\exp(\ln 0.783 \pm 1.96 \times 0.147)$, or 0.587 to 1.044. The point estimate is a 22% fall, but the interval includes no change.

5. Harms avoided across the hospital. The hospital has 60,000 patient days a year. At the observed rates it avoids 90 minus 70.43, or 19.57 harms per 1,000 patient days, which is about 1,174 harms a year. If each harm adds an average of 1.5 bed days at GBP 350 a day, the bed days released are worth about 1,174 × 1.5 × 350, or GBP 616,000, against a programme cost of GBP 250,000.

6. Interpretation. At the lower limit of the ratio the hospital avoids about 2,232 harms, and at the upper limit it has about 237 more, so this sample cannot rule out a programme that does nothing. A before and after design also cannot separate the programme from secular change or from a high baseline that would have fallen anyway. The released bed days are capacity rather than cash, and they yield savings only if beds close or are used to treat other patients.

Monitoring the rate over time

The Global Trigger Tool was built to follow one organisation over time, and its white paper asks for harms per 1,000 patient days to be plotted on a run chart in two-week steps. Small samples swing widely, so the paper advises at least 12 data points before a baseline is set and reports that experienced organisations recommend 24.

Rules from statistical process control help separate common cause variation from a real shift, and a control chart for counts per unit of exposure suits a rate with a changing denominator. Formal comparisons usually use Poisson regression. Landrigan and colleagues used a random effects Poisson model, adjusted for hospital clustering, on 2,341 admissions in ten North Carolina hospitals between 2002 and 2007. Internal reviewers found 25.1 harms per 100 admissions, and harms per 1,000 patient days showed no significant change, with a reduction factor of 0.99 per year.

Why harm rates are poor league table measures

A review-based rate reflects the reviewers as well as the care. The IHI white paper states that the tool is meant to track an organisation's own progress and is not as useful for comparing organisations, because reviewer skill varies and only the stability of that bias within one hospital makes trends meaningful. Hibbert and colleagues found adverse event rates in general inpatient studies varying between 7% and 40%, linked the spread most likely to methodological differences and reviewer interpretation, and concluded that rates should not be compared between institutions or studies.

Case mix adds a second problem. Older patients and surgical or intensive care work carry more opportunity for harm, and a crude rate penalises hospitals that take on complex patients unless the comparison uses risk adjustment. The harm rate therefore suits internal benchmarking over time better than public ranking or payment.

The harm rate in economic evaluation

Vincent and colleagues found that each adverse event in two London hospitals led to an average of 8.5 additional days in hospital, and estimated that preventable adverse events could cost the NHS around GBP 1 billion a year in extra bed days alone. The OECD report on the economics of patient safety by Slawomirski, Auraaen and Klazinga concluded that the available evidence suggests 15% of hospital expenditure and activity in OECD countries can be attributed to treating safety failures, and it expressed the excess bed days from six adverse event types in an average English hospital as admissions forgone. That framing treats harm as an opportunity cost.

An evaluation of a safety intervention combines the change in the harm rate, preferably from a controlled comparison, with the cost and the health loss of each harm. Costs usually come from extra length of stay or micro-costing, avoiding counting the same bed days twice. The health loss is valued as a disutility over the duration of the harm, or as lost life years for the most severe categories, so that the result can be expressed in QALYs.

A harm rate per patient day is not a probability. A rate to probability conversion, which assumes a constant rate, gives the probability of harm within an admission or a model cycle, and the competing risks of discharge and death matter when stays are short. Panagioti and colleagues pooled 70 studies across medical care settings and estimated that 6% of patients are exposed to preventable harm. A model of a preventive intervention should apply its effect to the preventable share, because the Global Trigger Tool count also includes harm that no current intervention could avoid.

Harm rate beside neighbouring safety measures

Several measures share this arithmetic but count different events. The adverse event rate is also used for trial safety data, where events after treatment are counted whether or not the treatment caused them, the sense usually used in drug cost-effectiveness models. The sentinel event rate follows a short list of rare and severe events that each trigger investigation. The near miss rate counts errors caught before they caused harm, the events the harm rate excludes, and the falls rate applies the same exposure logic to one type of harm.

Common misreadings of a harm rate

1. Treating a reporting count as a harm rate. Incident reports capture a small fraction of harm, and a rise in reports can reflect a better safety culture rather than less safe care.

2. Reading small samples as trends. With 20 records a month, single points vary widely and a fall of a fifth can be consistent with chance.

3. Reading the percentage of admissions with harm as an event rate. It counts patients rather than harms, so it misses repeat harm and moves less when care improves.

Sources

• Classen DC, Resar R, Griffin F, Federico F, Frankel T, Kimmel N, Whittington JC, Frankel A, Seger A, James BC. 'Global trigger tool' shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs. 2011;30(4):581-589. https://doi.org/10.1377/hlthaff.2011.0190
• Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events. 2nd ed. IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2009. https://www.ihi.org/sites/default/files/IHIGlobalTriggerToolWhitePaper2009.pdf
• Hibbert PD, Molloy CJ, Hooper TD, Wiles LK, Runciman WB, Lachman P, Muething SE, Braithwaite J. The application of the Global Trigger Tool: a systematic review. International Journal for Quality in Health Care. 2016;28(6):640-649. https://doi.org/10.1093/intqhc/mzw115
• Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA, Sharek PJ. Temporal trends in rates of patient harm resulting from medical care. New England Journal of Medicine. 2010;363(22):2124-2134. https://doi.org/10.1056/NEJMsa1004404
• Panagioti M, Khan K, Keers RN, Abuzour A, Phipps D, Kontopantelis E, Bower P, Campbell S, Haneef R, Avery AJ, Ashcroft DM. Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. BMJ. 2019;366:l4185. https://doi.org/10.1136/bmj.l4185
• Slawomirski L, Auraaen A, Klazinga N. The economics of patient safety: strengthening a value-based approach to reducing patient harm at national level. OECD; March 2017. https://doi.org/10.1787/5a9858cd-en
• Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ. 2001;322(7285):517-519. https://doi.org/10.1136/bmj.322.7285.517