Odds Ratio

Odds Ratio in Trials, Case-Control Studies and Economic Models

An odds ratio compares two groups by dividing the odds of an event in one group by the odds of the same event in the other, where the odds are the chance that the event happens set against the chance that it does not. In health economics the measure is a common relative treatment effect for a binary outcome, because logistic regression, case-control studies and many meta-analyses report results on this scale. A decision model runs on probabilities, so the odds ratio has to be combined with a baseline risk before it produces events and costs. A frequent error with odds ratios is reading them as if they were a relative risk, which they resemble only when the event is uncommon.

Odds, risks and the two-by-two table

The Cochrane Handbook, in chapter 6, defines the odds as the ratio of the probability that an event will occur to the probability that it will not, which in health care is the number of people with the event divided by the number without it. Risk divides the same number of events by everyone in the group. The two are close when the event is rare and far apart when it is common, so a risk of 0.5 corresponds to odds of 1 and a risk of 0.95 to odds of 19.

$$\text{odds} = \frac{p}{1 - p}, \qquad p = \frac{\text{odds}}{1 + \text{odds}}$$

where $p$ is the risk of the event, a probability between 0 and 1.

For a trial or cohort laid out as a contingency table, the odds ratio is the odds in the experimental group divided by the odds in the comparator group, which reduces to the cross-product of the four cells.

$$OR = \frac{a / b}{c / d} = \frac{a d}{b c}$$

where $a$ and $b$ are the numbers with and without the event in the experimental group, and $c$ and $d$ are the numbers with and without the event in the comparator group.

Reading the value and working on the log scale

An odds ratio of 1 means no difference between the groups, and values below 1 mean lower odds in the first group. Chapter 6 of the Cochrane Handbook points out that ratio scales are not symmetric, since an odds ratio of 0.5 and one of 2 are opposite effects whose plain average is 1.25, not 1. On the natural log scale they become minus 0.69 and 0.69, which average to 0, the log of no effect. Estimation, confidence intervals and meta-analysis therefore work with the log odds ratio, and the results are exponentiated for reporting.

$$SE\{\ln(OR)\} = \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d}}$$

where $SE\{\ln(OR)\}$ is the standard error of the natural log of the odds ratio and $a$, $b$, $c$ and $d$ are the cell counts defined above, as given in the Cochrane statistical algorithms for Review Manager. An approximate 95% confidence interval is $\exp[\ln(OR) \pm 1.96 \times SE\{\ln(OR)\}]$.

Two properties set the odds ratio apart from the risk ratio. If events and non-events are switched, the new odds ratio is the reciprocal of the original, while the risk ratios of an event and of no event can give quite different results. An observed risk ratio also cannot exceed one divided by the comparator group risk, so with a comparator risk of 0.66 it cannot pass 1.5, whereas chapter 10 calls the odds ratio the only unbounded statistic. Chapter 10 adds that there is no consensus on how much either property matters.

Why an odds ratio is not a risk ratio

Because odds and risks diverge as events become common, so do the two ratios. Chapter 6 of the Cochrane Handbook states that both are valid ways of describing an intervention effect, and that reading an odds ratio as a risk ratio overstates the effect of the intervention, especially when events are common, with risks above about 20%. An odds ratio above 1 is larger than the matching risk ratio and one below 1 is smaller, so the misreading exaggerates benefits and harms alike. Zhang and Yu made the same point for adjusted odds ratios from logistic regression, noting that once an outcome is common, above about 10%, the odds ratio no longer approximates the risk ratio.

Chapter 15 of the Handbook gives the conversion, which needs an assumed comparator risk.

$$RR = \frac{OR}{1 - ACR \times (1 - OR)}$$

where $RR$ is the risk ratio, $OR$ is the odds ratio and $ACR$ is the assumed comparator group risk. The Handbook suggests the median comparator group risk of the included studies as a reasonable choice.

Where odds ratios come from

The odds ratio is the natural measure for a case-control design, which samples people because they have or do not have the outcome and so cannot estimate risks directly. Labrecque and colleagues explain that such a study calculates an exposure odds ratio, the odds of exposure in cases divided by that in non-cases, which equals the disease odds ratio when controls are sampled from non-cases. When controls are sampled from the whole cohort the same calculation estimates the risk ratio, and from person-time the incidence rate ratio, so the label should follow the sampling design.

In logistic regression the coefficient on a binary treatment is a log odds ratio, so exponentiating it gives the odds ratio. Chapter 6 of the Cochrane Handbook regards adjusted odds ratios from logistic regression in non-randomised studies as generally preferable to crude ones, because adjustment usually reduces the impact of confounding.

In meta-analysis of binary outcomes, chapter 10 of the Handbook recommends either risk ratios or odds ratios, reports that the two show little difference in consistency on average, and calls the odds ratio the hardest summary statistic to understand and apply. The Peto method combines only odds ratios and has been observed to be less biased and more powerful for rare events when effects are not very large and group sizes are balanced. Other pooling choices are covered under the Mantel-Haenszel method.

Conditional and marginal odds ratios

Odds ratios are noncollapsible, which makes adjusted and unadjusted estimates different quantities. Daniel, Zhang and Farewell show that even in an ideal randomised trial the odds ratio for treatment changes when a baseline covariate that predicts the outcome is added to the model, however large the sample and with no confounding at all. The conditional odds ratio, for patients sharing the same covariate values, is typically further from 1 than the marginal odds ratio for the whole population. They note that naively pooling odds ratios adjusted for different covariate sets in a meta-analysis is clearly problematic. An adjusted and an unadjusted odds ratio from the same trial therefore give different absolute effects when applied to the same population-average baseline risk, so a model should state which one it uses.

Applying an odds ratio to a baseline risk

The NICE Decision Support Unit's Technical Support Document 5 (TSD 5) describes most cost-effectiveness analyses as combining a baseline model of absolute natural history under standard care with a model for relative treatment effects, usually taken from randomised trials. For an odds ratio, the two are joined on the log odds scale.

$$\text{logit}(p_T) = \text{logit}(p_0) + \ln(OR), \qquad \text{logit}(x) = \ln\left(\frac{x}{1 - x}\right)$$

where $p_T$ is the event probability on the new treatment, $p_0$ is the baseline risk under standard care and $OR$ is the odds ratio of the new treatment against standard care. TSD 5 gives the example of a baseline probability of 0.25 and an odds ratio of 0.8, which yield a probability of 0.21 on treatment. The resulting probabilities feed a decision tree or a transition probability in a Markov model.

The choice of scale is a modelling assumption. Chapter 10 of the Cochrane Handbook notes that when the assumed comparator risk is close to the comparator risks seen in the studies, odds ratios and risk ratios give similar absolute effects, but when it differs, the predicted absolute benefit depends on which summary statistic was pooled. Chapter 15 adds that confidence limits converted this way do not include uncertainty about the assumed comparator risk. In probabilistic sensitivity analysis the log odds ratio can be sampled from a normal distribution, which keeps every draw of the odds ratio positive.

Worked example with a common readmission outcome

All numbers in this example are illustrative. A trial of a discharge support programme records emergency readmission within one year in 60 of 200 patients on the programme and 90 of 200 on usual care.

1. Odds and odds ratio. The odds are 60/140 = 0.4286 on the programme and 90/110 = 0.8182 on usual care. The odds ratio is 0.4286/0.8182 = 0.524, the same as the cross-product (60 × 110)/(140 × 90) = 6,600/12,600 = 0.524.

2. Confidence interval. The standard error of the log odds ratio is the square root of 1/60 + 1/140 + 1/90 + 1/110 = 0.04401, which is 0.2098. With $\ln(0.524) = -0.647$ and 1.96 × 0.2098 = 0.411, the 95% interval runs over minus 1.058 and minus 0.235 on the log scale, which exponentiate to 0.347 and 0.790.

3. Risk ratio. The risks are 0.30 and 0.45, so the risk ratio is 0.30/0.45 = 0.667, a relative risk reduction of 33%. The conversion formula with an assumed comparator risk of 0.45 gives 0.524 divided by (1 minus 0.45 × 0.476), which is 0.524/0.786 = 0.667. Reading the odds ratio as a risk ratio would claim a 48% reduction, and applied to the trial's own baseline it would predict 0.45 × 0.524 = 0.236 instead of the observed 0.30, overstating the benefit by about 64 readmissions per 1,000.

4. Event coding. Counting patients who avoid readmission gives an odds ratio of (140 × 90)/(60 × 110) = 1.909, exactly 1/0.524. The risk ratio of avoiding readmission is (140/200)/(110/200) = 1.273, not 1/0.667 = 1.5.

5. A lower-risk population. The model population has a one-year readmission risk of 0.10 under usual care. The baseline odds are 0.10/0.90 = 0.1111, the treated odds are 0.1111 × 0.524 = 0.0582 and the treated risk is 0.0582/1.0582 = 0.055. That is 45 readmissions avoided per 1,000 patients, a number needed to treat of 1/0.045 = 22.2, rounded up to 23. Assuming a constant risk ratio instead gives 0.10 × 0.667 = 0.0667, or 33 avoided per 1,000 and a number needed to treat of 30. At an illustrative GBP 3,000 per readmission, the two routes save 0.045 × 3,000 = GBP 135 and 0.0333 × 3,000 = GBP 100 per patient.

6. Interpretation. In the trial population both routes agree, because the odds ratio and risk ratio were estimated from the same table. In the lower-risk population they differ by about 12 readmissions per 1,000 and GBP 35 per patient, only because of which ratio is assumed constant across baseline risks. The absolute risk reduction carried into the model, and the number needed to treat quoted alongside it, should be reported with that assumption, and the alternative tested in sensitivity analysis.

Odds ratio compared with nearby measures

The risk ratio compares probabilities over a stated period and is easier to explain, but is bounded when events are common. The risk difference is the absolute measure a cost model needs, but chapter 10 of the Cochrane Handbook advises against pooling it without a clear reason to expect consistency across studies. The hazard ratio compares instantaneous event rates and belongs to survival models, where an odds ratio taken at one time point loses the timing of events.

Common misreadings

Errors with odds ratios tend to arise when an estimate is read or applied outside the study that produced it.

1. Calling it a risk ratio. For common outcomes this exaggerates the effect, and chapter 6 of the Cochrane Handbook reports the error as quite common in published reports and reviews.
2. Ignoring the baseline risk. The same odds ratio gives very different absolute effects at different baseline risks, and only absolute effects reach costs and QALYs.
3. Mixing adjusted and crude estimates. Conditional and marginal odds ratios differ even without confounding, so pooling or applying them interchangeably distorts the result.
4. Assuming a case-control odds ratio estimates a disease odds ratio. With case-cohort or person-time sampling it estimates a risk ratio or a rate ratio.

Sources

• Daniel R, Zhang J, Farewell D. Making apples from oranges: comparing noncollapsible effect estimators and their standard errors after adjustment for different covariate sets. Biometrical Journal. 2021;63(3):528-557. https://doi.org/10.1002/bimj.201900297
• Deeks JJ, Higgins JPT. Statistical algorithms in Review Manager. Cochrane Statistical Methods Group; May 2022. https://documentation.cochrane.org/revman-kb/files/210600101/210600103/1/1654774257333/Statistical_methods_in_revman.pdf
• Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA (editors). Chapter 10: Analysing data and undertaking meta-analyses (last updated November 2024). In: Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-10
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 5: Evidence synthesis in the baseline natural history model. Decision Support Unit; August 2011, last updated April 2012. https://doi.org/10.15131/shef.data.33737536
• Higgins JPT, Li T, Deeks JJ (editors). Chapter 6: Choosing effect measures and computing estimates of effect (last updated August 2023). In: Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06
• Labrecque JA, Hunink MMG, Ikram MA, Ikram MK. Do case-control studies always estimate odds ratios? American Journal of Epidemiology. 2021;190(2):318-321. https://doi.org/10.1093/aje/kwaa167
• Schünemann HJ, Vist GE, Higgins JPT, Santesso N, Deeks JJ, Glasziou P, Akl EA, Guyatt GH. Chapter 15: Interpreting results and drawing conclusions (last updated August 2023). In: Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15
• Zhang J, Yu KF. What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA. 1998;280(19):1690-1691. https://doi.org/10.1001/jama.280.19.1690