Randomised Controlled Trial

Randomised Controlled Trial Evidence in Health Technology Assessment

In a randomised controlled trial, chance and not the clinician or the patient decides who receives the new intervention and who receives the comparator. Because the groups then differ only by chance and by the treatment assigned, a difference in their outcomes can be read as the treatment effect that a cost-effectiveness model needs. The NICE manual for technology appraisal (PMG36) accordingly states a strong preference for high-quality randomised trials as the source of relative treatment effects. That protection is easily lost after allocation, and an analysis restricted to adherers can turn a modest benefit into an apparent cost saving. Trial planning and sample size belong to clinical trial design.

Exchangeability as the case for randomisation

The case for randomisation is a causal argument. Each participant can be thought of as having two potential outcomes, one under the intervention and one under the comparator, of which only one is ever observed. The average treatment effect compares the mean of these potential outcomes across the whole population.

$$\text{ATE} = E\left[Y^{a=1}\right] - E\left[Y^{a=0}\right] = E[Y \mid A=1] - E[Y \mid A=0]$$

where $Y^{a=1}$ and $Y^{a=0}$ are a participant's potential outcomes under the intervention and the comparator, $A$ is the treatment actually assigned (1 for the intervention, 0 for the comparator), $Y$ is the outcome observed, and $E[\cdot]$ is the population mean. The second equality holds when the groups are exchangeable, meaning that the treatment assigned carries no information about the potential outcomes.

Hernán and Robins, in Causal Inference, write that randomisation is so highly valued because it is expected to produce exchangeability, and that in ideal randomised experiments association is causation. The ICH E9 trial statistics guideline describes randomisation as a deliberate element of chance in the assignment of treatments, and says it tends to produce groups in which the distributions of prognostic factors, known and unknown, are similar. Balance on unknown factors separates a trial from an adjusted observational study, which can balance only measured confounding factors.

Exchangeability holds in expectation, not in every trial. A small trial can show chance baseline imbalance, and restricted schemes such as stratified blocks or minimisation reduce that risk for chosen factors.

Safeguards that keep the comparison fair

Random assignment creates comparable groups only at allocation, and later events decide whether they stay comparable. The revised Cochrane risk-of-bias tool for randomised trials (RoB 2), edited by Higgins and colleagues in 2019, structures the threats into five domains, each of which it treats as mandatory for every result assessed.

NICE PMG36 (section 3.3.6) judges trial evidence on similar grounds and adds external validity, whether results apply to routine clinical practice. The CONSORT 2025 statement, a 30-item checklist with a participant flow diagram, sets out what a trial report should include so these points can be judged.

Estimands, intention to treat and per-protocol sets

The ICH E9(R1) addendum on estimands asks trialists to state in advance how events after randomisation, such as stopping treatment, are handled, and it links its treatment policy strategy to the intention-to-treat analysis principle of analysing participants as randomised. A hypothetical strategy instead needs modelling assumptions that randomisation does not supply.

The per-protocol set keeps only participants who followed the protocol. ICH E9 notes that it may maximise the chance of showing efficacy, and warns that the resulting bias, which may be severe, arises because adherence to the protocol may be related to treatment and outcome. For economic evaluation the treatment policy estimand is usually the natural one, because a health system pays for offering a treatment, including to people who stop it.

How randomised evidence enters an economic evaluation

Trial results usually reach a reimbursement decision by one of two routes. In a within-trial evaluation, resource use and health-related quality of life are collected from each participant alongside the clinical outcome, and incremental costs and QALYs are estimated directly from the randomised groups.

In the second route, the trial supplies a relative effect, such as a relative risk or hazard ratio, which a decision model applies to a baseline risk drawn from the population the decision concerns. The relative effect is usually pooled across trials by meta-analysis, or by a network meta-analysis when the trials do not share a comparator.

Petrou and Gray list truncated time horizons and limited comparators among the limitations of trial-based evaluation, which is why trial results are often placed inside a model, with extrapolation beyond the trial to a lifetime time horizon.

Worked example of intention to treat against per-protocol (illustrative)

All numbers in this example are hypothetical. A trial randomises 1,600 adults at high risk of a hospital admission, 800 to a new preventive drug and 800 to usual care, with one year of follow-up and no missing outcomes. In the drug arm, 640 participants take the drug throughout and 160 stop early, mostly the frailer participants.

1. Event counts. Of the 640 adherers, 64 are admitted, and of the 160 who stop, 40 are admitted, so the drug arm has 104 admissions in 800. The usual-care arm has 136 in 800.

2. Intention-to-treat risks. The risk in the drug arm is $\hat{p}_1 = 104 / 800 = 0.130$ and in the usual-care arm $\hat{p}_0 = 136 / 800 = 0.170$.

3. Effect and its precision. The risk difference and its standard error follow from the two risks.

$$\widehat{RD} = \hat{p}_1 - \hat{p}_0, \qquad SE = \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_0(1-\hat{p}_0)}{n_0}}$$

where $\widehat{RD}$ is the estimated risk difference, $\hat{p}_1$ and $\hat{p}_0$ are the observed risks in the drug and usual-care arms, $n_1$ and $n_0$ are the numbers randomised to each arm, and $SE$ is the large-sample standard error of the difference. Here $\widehat{RD} = -0.040$. The variance terms are 0.130 × 0.870 / 800 = 0.000141 and 0.170 × 0.830 / 800 = 0.000176, their sum is 0.000318, and $SE = 0.0178$. The 95% confidence interval is −0.040 ± 1.96 × 0.0178, from −0.075 to −0.005. The relative risk is 0.130 / 0.170 = 0.765, and the number needed to treat is 1 / 0.040 = 25.

4. Per-protocol risks. Dropping the 160 who stopped gives a drug-arm risk of 64 / 640 = 0.100, compared with the whole usual-care arm at 0.170. The risk difference becomes 0.100 − 0.170 = −0.070, the relative risk 0.100 / 0.170 = 0.588 and the number needed to treat 1 / 0.070 = 14.3, rounded up to 15.

5. Where the bias comes from. The usual-care arm still contains the frail people who would have stopped the drug. Suppose, as an untestable assumption, that there are 160 of them and that their risk is the same 40 in 160 (0.25) in either arm because the drug did nothing for them. The other 640 usual-care participants then had 136 − 40 = 96 admissions, a risk of 96 / 640 = 0.150. The effect among people able to adhere is 0.100 − 0.150 = −0.050, so the per-protocol analysis overstates even that effect by comparing adherers with an arm that still includes the frail.

6. Consequence for cost-effectiveness. Suppose the drug and its monitoring cost GBP 400 per participant offered it and each admission costs GBP 6,000. Under intention to treat, the incremental cost is 400 − 0.040 × 6,000 = 160, so GBP 160 per participant, and the cost per admission avoided is 160 / 0.040 = 4,000, so GBP 4,000. Under the per-protocol estimate it is 400 − 0.070 × 6,000 = −20, an apparent saving of GBP 20 per participant, so the drug would appear dominant.

7. Interpretation. The intention-to-treat estimate answers the payer's question, the effect of offering the drug to people some of whom will stop it. The per-protocol figure mixes the drug effect with the better prognosis of adherers and here reverses the cost conclusion.

Where randomised controlled trials sit among other designs

A randomised controlled trial is one kind of clinical trial, and the label covers several layouts. A cluster randomisation design allocates practices or wards when the intervention acts on a service, and a crossover trial gives each participant a randomised sequence of treatments. An explanatory trial tests a treatment under ideal conditions, while pragmatic designs move closer to routine care.

A non-randomised controlled trial still assigns treatment prospectively and has a control group, but the assignment is not by chance, so the exchangeability argument above no longer applies. PMG36 (section 3.3.9) says non-randomised studies tend to be at high risk of bias because the factors that influence treatment assignment may also predict outcomes. Real-world evidence may be preferred for natural history and treatment patterns (PMG36, section 3.3.2), so it complements a trial.

Limitations and common misreadings

Most errors in using trial evidence treat randomisation as a guarantee about everything a decision needs. It protects only the comparison made in the population enrolled.

1. Reading "randomised" as "unbiased". Poor concealment, unblinded outcome assessment, differential dropout or selective reporting can each bias a randomised result, which is why RoB 2 assesses five domains and not one.
2. Reading internal validity as relevance. A placebo-controlled trial in a selected population may leave the NICE comparator unanswered.
3. Reading the primary result as the economic result. Petrou and Gray report that power calculations are almost invariably based on clinical outcomes, and follow-up rarely covers the model's time horizon.
4. Treating a per-protocol analysis as the true drug effect. Adherence is not randomised, and the bias can reach the cost conclusion.

Sources

• Hernán MA, Robins JM. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC; 2020. Online version of 19 August 2026, chapter 2. https://miguelhernan.org/whatifbook
• Higgins JPT, Savović J, Page MJ, Sterne JAC, eds, on behalf of the RoB2 Development Group. Revised Cochrane risk-of-bias tool for randomized trials (RoB 2). 22 August 2019. https://www.riskofbias.info/welcome/rob-2-0-tool/current-version-of-rob-2
• Hopewell S, Chan AW, Collins GS, et al. CONSORT 2025 statement: updated guideline for reporting randomised trials. BMJ. 2025;389:e081123. https://doi.org/10.1136/bmj-2024-081123
• International Conference on Harmonisation. ICH E9: Statistical principles for clinical trials. Step 4 guideline, 5 February 1998. https://database.ich.org/sites/default/files/E9_Guideline.pdf
• International Council for Harmonisation. ICH E9(R1) addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials. Step 4, 20 November 2019. https://database.ich.org/sites/default/files/E9-R1_Step4_Guideline_2019_1203.pdf
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 3.3.2, 3.3.6 and 3.3.9. https://www.nice.org.uk/process/pmg36/chapter/evidence-2
• Petrou S, Gray A. Economic evaluation alongside randomised controlled trials: design, conduct, analysis, and reporting. BMJ. 2011;342:d1548. https://doi.org/10.1136/bmj.d1548