Heterogeneous Treatment Effect

Heterogeneous Treatment Effect in Trials and Cost-Effectiveness Decisions

In health economics, a heterogeneous treatment effect means that patients given the same treatment differ in how much health they gain from it, and some may lose health while others benefit. Heterogeneity matters because reimbursement bodies decide on averages, and an average that blends gains with losses can lead a health system to fund a treatment for patients who do not benefit or withhold it from those who would. The population mean is the average treatment effect, while heterogeneity is the spread of individual effects around it. Whether an effect looks heterogeneous depends on the scale used, so a treatment with the same relative effect in every patient can give different absolute gains in QALYs.

Individual effects and what a trial can reveal

Kravitz, Duan and Braslow define the individual treatment effect as the difference between a person's outcome on one treatment and the same person's outcome on the alternative. Heterogeneity of treatment effects is then the magnitude of the variation in these effects across a population, typically expressed as their standard deviation, and in statistical terms it is an interaction between treatment and the individual patient.

$$\Delta_i = Y_i(1) - Y_i(0)$$

where $\Delta_i$ is the individual treatment effect for patient $i$, $Y_i(1)$ is that patient's outcome with the treatment and $Y_i(0)$ is the outcome with the comparator, both in the same units, for example QALYs.

A parallel-group randomised controlled trial observes each patient under only one treatment, so no individual effect is ever seen. The arm means identify the average, but the spread of individual effects depends on how a patient's two outcomes correlate, as follows.

$$\text{SD}(\Delta) = \sqrt{\sigma_1^2 + \sigma_0^2 - 2\rho\,\sigma_1\sigma_0}$$

where $\text{SD}(\Delta)$ is the standard deviation of the individual effects, $\sigma_1$ and $\sigma_0$ are the standard deviations of the outcome with the treatment and with the comparator, and $\rho$ is the correlation between a patient's outcomes under the two. Kravitz and colleagues point out that this correlation is usually unknown. With both standard deviations at an illustrative 0.20 QALYs, a correlation of 0.9 gives a spread of 0.089 QALYs and a correlation of 0.5 gives 0.20 QALYs, with the same average effect.

What can be estimated is the part of the variation explained by measured characteristics, through the conditional average effect.

$$\tau(x) = E[\,Y(1) - Y(0) \mid X = x\,]$$

where $\tau(x)$ is the conditional average treatment effect and $E[\cdot \mid X = x]$ is the mean over patients with baseline characteristics $X = x$, such as a risk score or biomarker status. The ISPOR-SMDM task force report on parameter estimation uses heterogeneity in this explained sense, keeps it apart from stochastic and parameter uncertainty, and places its relevance in identifying subgroups for separate cost-effectiveness analyses.

Where the variation in benefit comes from

Kravitz and colleagues name four dimensions behind heterogeneity, namely risk of the outcome without treatment, responsiveness to the treatment, vulnerability to its adverse effects, and utilities for different outcomes. In their warfarin example, patients with atrial fibrillation who are younger than 65 and have no history of hypertension, diabetes, previous stroke or transient ischaemic attack have such a low stroke risk, approaching 1% a year, that the risk of bleeding exceeds any possible benefit.

In an economic model these become inputs. A review by Grutters and colleagues linked sources of patient heterogeneity to baseline risk, treatment effect, health state utility and resource use. Sculpher argues that the sources relevant to cost-effectiveness extend beyond the relative treatment effect, the focus of trial analysis, to baseline event rates.

Not every source can define a subgroup in an appraisal. The NICE manual (PMG36, section 4.9.12) states that subgroups based solely on individual utilities for health states and patient preference are not considered relevant, nor are those based solely on cost differences linked to social characteristics or to location within the UK.

Scale dependence and absolute benefit

Kent, Steyerberg and van Klaveren describe heterogeneity of treatment effect as fundamentally scale dependent, so the scale should be stated. Subgroup differences are usually tested on a relative scale, such as the odds ratio or relative risk, whereas the absolute risk difference, or its inverse the number needed to treat, matters most for decisions. The absolute effect combines the relative effect with baseline risk, as follows.

$$\text{ARR}(x) = p_0(x) \times \left(1 - \text{RR}(x)\right)$$

where $\text{ARR}(x)$ is the absolute risk reduction for patients with characteristics $x$, $p_0(x)$ is their risk of the event under the comparator and $\text{RR}(x)$ is the relative risk with the treatment.

Kent and colleagues show that when outcome risk varies across a trial population, at most one measure of treatment effect can stay constant, so prognostic factors become effect modifiers on the absolute scale. PMG36 (section 4.9.7) asks for the scale of any effect modification to be specified. An interaction effect can vanish or reverse when the scale changes.

Finding heterogeneity in trial data

The usual tool is subgroup analysis, one variable at a time, with a test of interaction between treatment and a baseline characteristic. Kent and colleagues note that trials typically have very low power for interaction effects, and that single-variable subgroups under-represent the heterogeneity between patients who differ on many variables at once. PMG36 (section 4.9.8) points to the opposite risk, differences emerging by chance when many subgroups are reported, and states that prespecification with a clear rationale and a predicted direction increases credibility. Section 4.9.4 warns that post hoc data dredging will be viewed sceptically.

The PATH statement, by Kent and colleagues, separates risk modelling, in which a multivariable model of outcome risk divides trial patients into risk strata, from effect modelling, which adds treatment-by-covariate interaction terms. It encourages reporting results by risk strata when the overall trial result is positive, recommends reporting important treatment-related harms in each stratum, and warns that predictive analyses of null trials are unlikely to give reliable evidence. Because interaction terms are prone to overfitting, it recommends penalised estimation whenever they are included. PMG36 (section 4.9.10) prefers individual patient data for subgroup-specific parameters, as in individual patient data meta-analysis.

Between-study heterogeneity, summarised by I-squared, is a different idea, describing how results differ across the studies in a meta-analysis.

Worked example with a fixed harm across three risk strata

All numbers in this example are illustrative. A preventive drug is assessed over five years in a population split into three equal strata, with risks of a major event under usual care of 0.02, 0.08 and 0.20. The relative risk of the event with the drug is 0.75 in every stratum, so the effect is homogeneous on the relative scale. The drug also causes a serious adverse event in 1% of treated patients whatever their risk. Each major event costs GBP 8,000 and loses 2.0 QALYs, each adverse event costs GBP 3,000 and loses 1.5 QALYs, and the drug costs GBP 1,000 per patient. Discounting is ignored.

1. Absolute benefit. The absolute risk reduction is 0.25 times the baseline risk, giving 0.005, 0.02 and 0.05, or one major event avoided per 200, 50 and 20 patients treated.

2. QALYs gained. The QALY change per patient is found as follows.

$$\Delta Q = q_E \times \text{ARR} - q_A \times h$$

where $\Delta Q$ is the QALY change per patient treated, $q_E$ is the QALY loss per major event (2.0), $q_A$ is the QALY loss per adverse event (1.5) and $h$ is the adverse event risk (0.01). The harm term is 1.5 × 0.01, or 0.015 QALYs, in every stratum. Subtracting it from gains of 0.010, 0.040 and 0.100 leaves minus 0.005, plus 0.025 and plus 0.085 QALYs.

3. Incremental cost. Drug and adverse event costs are 1,000 + 3,000 × 0.01 = GBP 1,030 per patient, less avoided event costs of 8,000 × ARR (GBP 40, GBP 160 and GBP 400), leaving GBP 990, GBP 870 and GBP 630.

4. Results by stratum. The ICER is incremental cost divided by QALYs gained, and net monetary benefit (NMB) is the value per QALY times QALYs gained, minus incremental cost. The last row averages the three equal strata.

The high-stratum ICER is 630 / 0.085 = GBP 7,412 and the whole-population ICER 830 / 0.035 = GBP 23,714.

5. The decision. PMG36 (section 4.2.16) asks for net health benefit at GBP 25,000 and GBP 35,000 per QALY and allows net monetary benefit to be shown alongside. At GBP 25,000, treating everyone gives 25,000 × 0.035 minus 830 = GBP 45 per patient. The low stratum loses health and money, because one adverse event per 100 patients outweighs one major event avoided per 200. Treating only the high stratum gives 1,495 / 3 = GBP 498 per member of the population, GBP 453 more than treating everyone. At GBP 35,000 the middle stratum only just breaks even, adding 5 / 3 = GBP 1.67 per member of the population, so its inclusion turns on the threshold and on the uncertainty in its estimates.

6. Interpretation. The relative effect is identical in all three strata, yet the net effect changes sign, because a fixed absolute harm is set against a benefit that grows with risk. The PATH statement shows the same pattern for a hypothetical odds ratio of 0.75 with a 1% absolute harm.

Heterogeneity in reimbursement decisions

PMG36 (section 4.9.1) states that for many technologies the level of benefit will differ for patients with differing characteristics, and asks for clinical- and cost-effectiveness estimates for each relevant subgroup. Section 4.9.7 asks for the precision of all subgroup estimates to be reflected in the analysis of parameter uncertainty, and section 4.7.23 states that threshold analysis should not be used to justify restricting the population to a subgroup based on cost-effectiveness.

Espinoza and colleagues call the gain from acting on known heterogeneity with existing evidence its static value, like the GBP 453 per patient above, and the gain from further subgroup research its dynamic value, a value of information question. They also note the costs of acquiring patient characteristics and of monitoring adherence to subgroup guidance, which for a biomarker include the test, as discussed under precision medicine.

Common misreadings of heterogeneity

Several errors recur in appraisals. Most come from reading a summary statistic as if it described every patient.

1. A non-significant interaction is taken as uniform benefit. In the DANAMI-2 trial, Kent and colleagues report, conventional subgroup analyses showed consistent effects, while a later risk-stratified analysis found that low-risk patients were less likely to benefit than high-risk patients.
2. A constant relative effect is taken as equal benefit. In the example, absolute gains differ in size and direction.
3. A response-based continuation rule is analysed as a subgroup. PMG36 (section 4.9.13) asks for such a rule to be modelled as a separate treatment strategy, so that costs such as additional monitoring are included.

Sources

• Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical Decision Making. 2012;32(5):722-732. https://doi.org/10.1177/0272989X12458348
• Espinoza MA, Manca A, Claxton K, Sculpher MJ. The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application. Medical Decision Making. 2014;34(8):951-964. https://doi.org/10.1177/0272989X14538705
• Grutters JPC, Sculpher M, Briggs AH, et al. Acknowledging patient heterogeneity in economic evaluation: a systematic literature review. PharmacoEconomics. 2013;31(2):111-123. https://doi.org/10.1007/s40273-012-0015-4
• Kent DM, Steyerberg E, van Klaveren D. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. BMJ. 2018;363:k4245. https://doi.org/10.1136/bmj.k4245
• Kent DM, van Klaveren D, Paulus JK, et al. The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: explanation and elaboration. Annals of Internal Medicine. 2020;172(1):W1-W25. https://doi.org/10.7326/M18-3668
• Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Quarterly. 2004;82(4):661-687. https://doi.org/10.1111/j.0887-378X.2004.00327.x
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 4.2.16, 4.7.23 and 4.9. https://www.nice.org.uk/process/pmg36/chapter/economic-evaluation
• Sculpher M. Subgroups and heterogeneity in cost-effectiveness analysis. PharmacoEconomics. 2008;26(9):799-806. https://doi.org/10.2165/00019053-200826090-00009