Risk Ratio

Risk Ratio in Meta-Analysis and Decision Models

A risk ratio divides the proportion of people who have an event during a stated period in one group by the proportion who have the same event in a comparison group. A value of 0.6 means the event is 40% less likely in the first group, and a value of 2 means it is twice as likely. The same quantity is called the relative risk. In health economics the risk ratio is one of the two relative effects that systematic reviews usually pool for binary outcomes, and a decision model turns the pooled ratio into event probabilities by applying it to a baseline risk. Most errors with the measure arise after estimation, when a ratio measured over one follow-up period in one population is carried into a model with a different cycle length and baseline risk.

Risks from a two-by-two table

The CDC's Principles of Epidemiology defines risk, or incidence proportion, as the proportion of an initially disease-free population that develops the condition, is injured or dies during a specified period, and lists cumulative incidence among its synonyms. A risk ratio is therefore always tied to a time window, such as admission within one year. For a trial or cohort laid out as a contingency table, the Cochrane Handbook (Box 6.4.a) gives the calculation below, written in Review Manager notation.

$$RR = \frac{a / n_1}{c / n_2}$$

where $RR$ is the risk ratio, $a$ is the number of people with the event among the $n_1$ people in the experimental or exposed group, and $c$ is the number with the event among the $n_2$ people in the comparator group. A value of 1 means identical risk in the two groups.

Ratio measures are typically analysed on the log scale. The Cochrane statistical algorithms for Review Manager give the standard error of the log risk ratio for one study as follows.

$$SE\{\ln(RR)\} = \sqrt{\frac{1}{a} + \frac{1}{c} - \frac{1}{n_1} - \frac{1}{n_2}}$$

where $SE\{\ln(RR)\}$ is the standard error of the natural log of the risk ratio and the other terms are as defined above. An approximate 95% confidence interval is $\exp[\ln(RR) \pm 1.96 \times SE\{\ln(RR)\}]$.

Chapter 6 of the Cochrane Handbook states that a risk ratio cannot be calculated when the comparator group has no events, and that the customary fix, applied automatically in Review Manager, is to add one half to each cell. A study with no events in either group carries no information about the relative effect and is omitted. The NICE Decision Support Unit's Technical Support Document 2 (TSD 2) notes that adding 0.5 generates biased estimates, and counts the handling of occasional zero cells without correction as a major strength of its Bayesian binomial models.

Which outcome counts as the event

A binary outcome gives two risk ratios, one for the event and one for its absence. Chapter 10 of the Cochrane Handbook warns that these give different summary results in a meta-analysis, sometimes dramatically so, and chapter 6 adds that switching the outcome can change the estimate and its statistical significance. The odds ratio avoids this, since its value for the non-event is the reciprocal. The Handbook's rule of thumb is to count the less common state, and where an intervention aims to reduce an adverse event, risk ratios of the adverse event have been found more consistent across studies than risk ratios of the non-event.

There is also a ceiling. Chapter 6 states that an observed risk ratio must lie between 0 and one divided by the comparator group risk, so with a comparator risk of 0.66 it cannot exceed 1.5, and that this can cause problems when results are extrapolated to a population with higher comparator risks than the studies. A harm ratio of 1.8 from trials with a comparator risk near 0.2 would predict an impossible probability of 1.08 in a subgroup with a baseline risk of 0.6.

Pooling risk ratios across trials

In meta-analysis of binary outcomes, chapter 10 of the Handbook generally recommends risk ratios or odds ratios. Relative measures are on average more consistent across studies than absolute ones, and there is little difference between risk ratios and odds ratios in consistency. Picking whichever measure looks most consistent in a given review is not recommended, since it may spuriously maximise precision.

The inverse-variance method averages the study log risk ratios with weights of one over each squared standard error. The Mantel-Haenszel method weights each study differently, as Review Manager sets out.

$$RR_{MH} = \frac{\sum_i w_i RR_i}{\sum_i w_i}, \qquad w_i = \frac{c_i \, n_{1i}}{N_i}$$

where $RR_{MH}$ is the pooled risk ratio, $RR_i$ is the risk ratio in study $i$, $c_i$ is its number of comparator group events, $n_{1i}$ is its experimental group size and $N_i$ is its total number of participants. Chapter 10 reports better statistical properties for Mantel-Haenszel when events are few and similar estimates from the two methods otherwise.

Model-based synthesis is more cautious. TSD 2 explains that a log risk model, unlike the usual logit model, has nothing to stop fitted probabilities leaving the range zero to one, so special coding is needed and the models can be less stable. Its main models for network meta-analysis of binary data work on the logit scale, and a risk ratio can be derived afterwards from an assumed absolute effect.

Follow-up time changes the ratio

Because each risk belongs to a period, the same treatment can produce different risk ratios at different follow-up times. TSD 2 handles trials of unequal duration by assuming a constant event rate in each arm, so that the probability of an event by follow-up time $f$ is one minus $\exp(-\lambda f)$, where $\lambda$ is the rate, and by pooling log hazard ratios through a complementary log-log link.

As an illustration, suppose a treatment has a constant hazard ratio of 0.6 and the comparator one-year risk is 0.10, a rate of 0.1054 per year. The one-year risks are 0.100 and 0.061, a risk ratio of 0.61. At five years they are 0.410 and 0.271, a risk ratio of 0.66, and at ten years 0.651 and 0.469, a risk ratio of 0.72. The underlying effect has not changed, yet the risk ratio moves towards 1 as events accumulate. Pooling ratios from trials of very different length therefore mixes different quantities.

Carrying a risk ratio into a cost-effectiveness model

NICE DSU Technical Support Document 5 (TSD 5) describes most cost-effectiveness analyses as combining a baseline model of absolute natural history under standard care with a model of relative treatment effects, usually from randomised trials. A risk ratio joins the two by multiplying the baseline risk.

$$p_T = RR \times p_0, \qquad ARR = p_0 \times (1 - RR)$$

where $p_T$ is the treated risk, $p_0$ is the assumed comparator risk over the same period as the risk ratio, $RR$ is the risk ratio and $ARR$ is the absolute risk reduction. Chapter 15 of the Handbook uses this calculation to re-express a pooled risk ratio as events avoided per 1,000 and as a number needed to treat, and notes that relative effects tend to be substantially more stable across risk groups than absolute effects. The relative risk reduction is one minus the risk ratio.

The baseline risk and the ratio must cover the same period, and a model with shorter cycles needs a rate-to-probability conversion of the treated risk, not the trial ratio applied to each cycle. A ratio above 1 must never push a transition probability above 1. Chapter 10 notes that when the assumed comparator risk differs from the trial risks, the predicted absolute benefit depends on whether risk ratios or odds ratios were pooled, so that choice belongs in the sensitivity analysis. In probabilistic sensitivity analysis the log risk ratio can be drawn from a normal distribution, which keeps every draw positive.

Worked example with two trials and a quarterly model

All numbers in this example are illustrative. Two trials of a heart failure support programme record emergency admission within one year. Trial A finds 30 admissions among 400 patients on the programme and 48 among 400 on usual care. Trial B finds 9 among 150 and 18 among 150.

1. Trial risk ratios. In trial A the risks are 30/400 = 0.075 and 48/400 = 0.12, a risk ratio of 0.075/0.12 = 0.625. In trial B they are 9/150 = 0.06 and 18/150 = 0.12, a risk ratio of 0.50.

2. Uncertainty for trial A. The variance of the log risk ratio is 1/30 + 1/48 minus 2/400, which is 0.05417 minus 0.005, or 0.04917, so the standard error is 0.2217. With a log risk ratio of minus 0.470 and 1.96 × 0.2217 = 0.435, the log limits are minus 0.905 and minus 0.035, which exponentiate to 0.405 and 0.965.

3. Mantel-Haenszel pooling. The weights are 48 × 400/800 = 24 for trial A and 18 × 150/300 = 9 for trial B. The weighted sum is 24 × 0.625 + 9 × 0.50 = 19.5, and dividing by the total weight of 33 gives a pooled risk ratio of 0.591, with a Review Manager standard error of 0.193 and a 95% interval of 0.405 to 0.862. The inverse-variance method gives 0.592.

4. Event coding. Counting patients who stay out of hospital, the risk ratios are 0.925/0.88 = 1.051 in trial A and 0.94/0.88 = 1.068 in trial B. Trials showing 38% and 50% relative reductions in admissions show only 5% and 7% relative gains in admission-free patients.

5. A higher-risk model population. The modelled population has a one-year admission risk of 0.20 on usual care. The treated risk is 0.20 × 0.591 = 0.118, an absolute reduction of 0.20 × 0.409 = 0.0818, or about 82 admissions avoided per 1,000 patients. The number needed to treat is 1/0.0818 = 12.2, rounded up to 13. At an illustrative GBP 3,500 per admission, the saving is 0.0818 × 3,500 = GBP 286 per patient in the first year.

6. Quarterly cycles. The usual-care rate is minus ln(0.80) = 0.2231 per year, so the quarterly probability is one minus exp(minus 0.2231/4) = 0.0543. Multiplying that by 0.591 gives 0.0321 per quarter, and four such quarters give a one-year treated risk of 0.122, about 4 more admissions per 1,000 than the 0.118 the trials imply. Converting the treated one-year risk instead gives a rate of minus ln(0.8818) = 0.1258 and a quarterly probability of 0.0310, so the risk ratio per quarter is 0.0310/0.0543, about 0.57.

7. Interpretation. Applying a ratio estimated at a comparator risk of 0.12 to a baseline of 0.20 assumes the relative effect holds at higher risk, which a sensitivity analysis using the pooled odds ratio can check. The quarterly model reproduces the trials only when the treated risk is converted through rates.

Risk ratio beside related measures

Chapter 6 of the Handbook treats risk ratio and relative risk as interchangeable names. The odds ratio drifts away from the risk ratio as events become common, so reading one as the other overstates the effect. The CDC lesson notes that risk ratios cannot be calculated from a typical case-control design, whereas a cohort design or trial measures risks directly. The incidence rate ratio divides events per unit of person-time. Chapter 10 notes that in a randomised trial it is often very similar to the risk ratio, unless the intervention changes how many people have repeated events. The risk difference is the absolute counterpart that a cost model needs in the end.

Common misreadings

Errors with risk ratios usually come from losing track of the period or the population behind the estimate. Each one changes modelled costs.

1. Ignoring the time window. Ratios from trials of different length are not directly comparable.
2. Switching the event without saying so. The ratios for an event and for its absence can differ sharply.
3. Applying a harm ratio above its ceiling. A ratio above 1 times a high baseline risk can exceed a probability of 1.
4. Reusing a trial ratio per model cycle. The treated risk must be converted through rates for the cycle length.
5. Quoting the ratio without the baseline risk. Chapter 6 notes that a risk ratio of 0.75 could describe 80% falling to 60% or 4% falling to 3%, and only the absolute change reaches costs and QALYs.

Sources

• Centers for Disease Control and Prevention. Principles of Epidemiology in Public Health Practice, Third Edition. Lesson 3: Measures of Risk, sections 2 and 5. Last reviewed May 2012. https://archive.cdc.gov/www_cdc_gov/csels/dsepd/ss1978/lesson3/section2.html ; https://archive.cdc.gov/www_cdc_gov/csels/dsepd/ss1978/lesson3/section5.html
• Deeks JJ, Higgins JPT. Statistical algorithms in Review Manager. Cochrane Statistical Methods Group; May 2022. https://documentation.cochrane.org/revman-kb/files/210600101/210600103/1/1654774257333/Statistical_methods_in_revman.pdf
• Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA (editors). Chapter 10: Analysing data and undertaking meta-analyses (last updated November 2024). In: Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-10
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. Decision Support Unit; August 2011, last updated September 2016. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD2-General-meta-analysis-corrected-2Sep2016v2.pdf
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 5: Evidence synthesis in the baseline natural history model. Decision Support Unit; August 2011, last updated April 2012. https://doi.org/10.15131/shef.data.33737536
• Higgins JPT, Li T, Deeks JJ (editors). Chapter 6: Choosing effect measures and computing estimates of effect (last updated August 2023). In: Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06
• Schünemann HJ, Vist GE, Higgins JPT, Santesso N, Deeks JJ, Glasziou P, Akl EA, Guyatt GH. Chapter 15: Interpreting results and drawing conclusions (last updated August 2023). In: Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15