Causal Inference

Causal Inference and Treatment Effects for Health Economic Decisions

Causal inference asks how costs and health outcomes would differ if the same population received one treatment or policy rather than another, and whether the available data can answer that question. Every incremental cost and incremental QALY in an economic evaluation is meant to be such a contrast, so these methods decide whether a cost-effectiveness result describes the effect of a technology or only the kind of patients who received it. The NICE real-world evidence framework (ECD9) applies these ideas to non-randomised evidence through the target trial approach.

Counterfactual outcomes and the missing half of the data

Hernán and Robins define an individual causal effect by comparing the outcome when an action is taken with the outcome when it is withheld. For a binary treatment $A$ (1 treated, 0 untreated), each person has two potential, or counterfactual, outcomes, $Y^{a=1}$ and $Y^{a=0}$, and only the one under the treatment actually received is observed. The NICE Decision Support Unit's Technical Support Document 17 (TSD 17) calls this a fundamental problem in evaluating a treatment, because nobody is observed in both states at the same point in time. The usual target is therefore an average over a defined population, written as follows.

$$\text{ACE} = E[Y^{a=1}] - E[Y^{a=0}]$$

where ACE is the average causal effect, $E[Y^{a=1}]$ is the mean outcome if everyone had been treated and $E[Y^{a=0}]$ the mean outcome if nobody had been. For a binary outcome the contrast is a causal risk difference; dividing instead gives a causal risk ratio.

Association compares the risk in people who happened to be treated with the risk in people who happened not to be, two different subsets of the population. Causation compares the same whole population under two different actions, so a correlation between treatment and outcome in routine data is not, by itself, a treatment effect.

Which average effect the decision needs

TSD 17 lists the average treatment effect (ATE), the average treatment effect on the treated (ATT), the local average treatment effect (LATE) and the marginal treatment effect. It states that the ATE is typically the effect of interest in NICE technology appraisals and describes it as the expected effect if the population under consideration were randomly allocated to treatment. When effects vary and selection into treatment depends on that variation, these averages differ; the average treatment effect and heterogeneous treatment effects have their own pages.

ECD9 notes that trials usually estimate the effect of assignment (intention to treat) or of adhering to the protocol (per protocol). Observational data rarely record assignment, so the effect of starting treatment, the as-started effect, serves as a proxy and is usually of primary interest to NICE, with an on-treatment analysis also needed when switching or discontinuation is substantial or unlike NHS practice.

Three conditions that let data stand in for the counterfactual

Hernán and Robins analyse an observational study as if it were a conditionally randomised experiment, an analogy that holds only under three identifiability conditions. These hold by design in an ideal randomised experiment but are often heroic in observational studies.

In notation, conditional exchangeability and positivity are written as follows.

$$Y^{a} \perp\!\!\!\perp A \mid L \qquad \text{and} \qquad \Pr[A=a \mid L=l] > 0$$

where $Y^{a}$ is the counterfactual outcome under treatment level $a$, $A$ is the treatment received, $L$ is the set of measured covariates and $\perp\!\!\!\perp$ denotes independence; positivity applies to every $a$ in the contrast and every value $l$ present in the population of interest. The counterfactual definition also assumes no interference between people; Hernán and Robins note that interference is common in studies of contagious agents. TSD 17 calls this assumption ignorability or unconfoundedness, states that it cannot be tested and must be justified from the literature and expert opinion, and adds that overlap is also needed to identify the ATE. Positivity, unlike exchangeability, can sometimes be checked in the data.

Choosing a design under randomisation, observed and unobserved confounding

Randomisation makes the identifiability conditions hold by design, and ECD9 names randomised controlled trials as the preferred design for comparative effects, with non-randomised evidence adding value when trials are absent, not directly relevant or of poor quality. For non-randomised individual patient data, TSD 17 groups the common methods by the assumption each needs.

ECD9 adds that quasi-experimental methods usually depend strongly on assumptions that are difficult to test, and TSD 17 notes that difference-in-differences generally identifies the ATT, so the design also fixes the estimand. ECD9 asks developers to state their causal assumptions, ideally as directed acyclic graphs, and to avoid adjusting for variables on the causal pathway, colliders or instruments, or choosing covariates by statistical significance.

ECD9 also asks for non-randomised studies to mimic the randomised trial that would ideally have been run, the target trial approach, defined across seven dimensions, namely eligibility criteria, treatment strategies, assignment procedure, follow-up period, outcomes, causal effect of interest and analysis plan. Gomes and colleagues describe how emulation can complement trials in health technology assessment, for example on longer-term effects or comparators that trials omit.

Worked example of confounding by severity reversing a cost result

All numbers are illustrative. A health board evaluates a care-management programme for 2,000 adults with heart failure. The outcome is an emergency admission within a year; the programme costs GBP 150 per patient and each admission GBP 3,000. Clinicians enrol sicker patients more often, taking 640 of 800 high-severity patients (80%) and 300 of 1,200 low-severity patients (25%).

1. Crude comparison. Programme admissions are 640 × 0.30 + 300 × 0.10 = 222 among 940 patients, a risk of 0.236. Usual-care admissions are 160 × 0.40 + 900 × 0.15 = 199 among 1,060, a risk of 0.188. The crude difference is 0.048, about 4.8 extra admissions per 100, and the crude incremental cost is 150 + 0.0484 × 3,000 = GBP 295 per patient.

2. Standardise to the whole population. Under exchangeability given severity, the risk if everyone were enrolled weights each stratum's risk by its population share (0.4 high, 0.6 low), as follows.

$$\Pr[Y^{a=1}=1] = \sum_{l} \Pr[Y=1 \mid A=1, L=l] \Pr[L=l] = 0.30 \times 0.4 + 0.10 \times 0.6 = 0.18$$

where $Y$ is an admission within a year, $A=1$ denotes enrolment, $L$ is the severity stratum and $\Pr[L=l]$ is the population share of stratum $l$. For usual care, 0.40 × 0.4 + 0.15 × 0.6 = 0.25. The causal risk difference is 0.18 minus 0.25, or 0.07 fewer admissions per patient, and the causal risk ratio is 0.18 / 0.25 = 0.72.

3. Check by inverse probability weighting. Weighting each patient by the inverse of the probability of the option received (1/0.8 = 1.25 and 1/0.25 = 4 for enrolled patients; 1/0.2 = 5 and 1/0.75 = 1.333 for usual care) rebuilds 800 high-severity and 1,200 low-severity patients in each arm. The weighted risks are 360 / 2,000 = 0.18 and 500 / 2,000 = 0.25, as Hernán and Robins's result that standardisation and inverse probability weighting are mathematically equivalent predicts.

4. Cost consequence. Avoided admissions save 0.07 × 3,000 = GBP 210 per patient, so the programme saves 210 minus 150, or GBP 60 per patient, against an apparent extra cost of GBP 295 in the crude comparison.

5. Interpretation. Severity raised both the chance of enrolment and the risk of admission, so the crude comparison charged the programme with its patients' prognosis. The GBP 60 saving is causal only if severity is the only common cause (exchangeability), each severity group had some chance of both options (positivity; 0.8 and 0.25 lie strictly between 0 and 1) and the programme was the same service for everyone (consistency). An unrecorded driver of enrolment such as frailty would bias it by an unknown amount.

Causal estimates inside economic evaluation and HTA

Model inputs that describe the consequences of a technology, including differences in resource use, costs and utilities as well as relative effects, are causal quantities. Kreif, Grieve and Sadique reviewed 81 cost-effectiveness analyses that used observational data. Studies tended to use regression (51%), matching on individual covariates (25%) or matching on the propensity score (22%), and the authors concluded that published analyses did not assess the main assumptions behind these methods. Their checklist asks whether the "no unobserved confounding" assumption and covariate overlap were assessed, and treats the choice of statistical approach as structural uncertainty.

ECD9 states that sensitivity analyses can be propagated through to cost-effectiveness analyses, and that quantitative bias analysis can show how much bias would be needed to change results or affect a decision threshold. It names the e-value as one of the simpler approaches; in its alectinib example an unobserved confounder would have needed an estimated relative risk of 2.2 to remove the effect on mortality, higher than for any observed confounder. A causal estimate entering a model should carry its estimand, population and assumptions, so that sensitivity analysis can test what the data cannot.

Where causal inference ends and neighbouring ideas begin

Observational studies and real-world evidence describe sources of data; causal inference decides what those data can say about an intervention. Prediction models forecast an outcome from given characteristics and can perform well with no causal interpretation, so a strong predictor of admission is not necessarily a lever for reducing admissions.

Transportability, applying an effect from one population to another, is in Hernán and Robins's words an unverifiable assumption relying heavily on subject-matter knowledge. Quasi-experimental designs and synthetic control methods have their own pages.

Common misunderstandings behind failed causal claims in HTA

Weak causal claims usually fail on design and assumptions rather than on the estimator. The sources above warn against these errors.

• Treating adjustment as identification. ECD9 notes that stratification, matching, regression and propensity score methods all assume no unmeasured confounding; balance on measured covariates says nothing about unmeasured ones.
• Adjusting for the wrong variables. TSD 17 states that including instrumental variables in the adjustment set increases bias unless they are exogenous.
• Ill-defined interventions. Hernán and Robins argue that "the causal effect of aspirin" is meaningless until the contrast is specified.
• Ignoring poor overlap. TSD 17 notes that the ATE cannot be estimated for a dataset that includes people whose characteristics place them always in one group.
• Invalid instruments. TSD 17 warns that even moderate endogeneity of an instrument can give estimates more inconsistent than methods that wrongly assume selection on observables.

Sources

• Hernán MA, Robins JM. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC; 2020. Online edition of 19 August 2026, chapters 1 to 4. https://miguelhernan.org/whatifbook
• Faria R, Hernández Alava M, Manca A, Wailoo AJ. NICE DSU Technical Support Document 17: The use of observational data to inform estimates of treatment effectiveness in technology appraisal: methods for comparative individual patient data. Sheffield: Decision Support Unit, ScHARR; 2015. Sections 2.1 to 2.5. https://orda.shef.ac.uk/articles/report/TSD_17_The_use_of_observational_data_to_inform_estimates_of_treatment_effectiveness_in_technology_appraisal_Methods_for_comparative_individual_patient_data/33847855
• National Institute for Health and Care Excellence. NICE real-world evidence framework (ECD9). London: NICE; 2022, updated September 2025. Methods for real-world studies of comparative effects. https://www.nice.org.uk/corporate/ecd9/chapter/methods-for-real-world-studies-of-comparative-effects
• Kreif N, Grieve R, Sadique MZ. Statistical methods for cost-effectiveness analyses that use observational data: a critical appraisal tool and review of current practice. Health Economics. 2013;22(4):486-500. https://doi.org/10.1002/hec.2806
• Gomes M, Latimer N, Soares M, Dias S, Baio G, Freemantle N, Dawoud D, Wailoo A, Grieve R. Target trial emulation for transparent and robust estimation of treatment effects for health technology assessment using real-world data: opportunities and challenges. PharmacoEconomics. 2022;40(6):577-586. https://doi.org/10.1007/s40273-022-01141-x