Historical Control

Historical Control in Trials and Health Technology Assessment

A trial with a historical control judges a new treatment against how earlier patients fared, assuming those patients would have done as well or as badly as today's participants without the new treatment. Nothing in the design makes the earlier patients resemble the new ones, and the comparison tends to make new treatments look better than they are. The design matters in health economics because more medicines now reach health technology assessment after regulatory approval based on uncontrolled single-arm trials, where an earlier patient group supplies the only estimate of what would have happened without the technology. That estimate becomes the comparator arm of the cost-effectiveness model, so any bias in it passes straight into the QALY gain and the ICER.

How a historical control differs from a concurrent control

The ICH E10 guideline on the choice of control group (2000) sorts control groups into five types. The first four are concurrent, meaning the control and test groups come from the same population and are treated over the same period, usually with random assignment. External controls, including historical controls, form the fifth type, set apart because of serious concerns about whether such trials can make the groups comparable and keep important biases small.

E10 describes the usual external control as a well-documented population of patients observed at an earlier time, although a group treated at the same time in another setting also counts. A nondefined control, based on general medical knowledge of outcome rather than a specific group of patients, it calls particularly treacherous.

In causal inference terms, a trial needs an estimate of the outcomes the treated patients would have had without the new treatment. A randomised controlled trial supplies a concurrent group whose outcomes estimate that counterfactual. A historical control substitutes a different group from a different time, so the estimated effect carries an extra term.

$$\hat{\Delta} = \bar{Y}_T - \bar{Y}_H = \Delta + \left(\bar{Y}_C - \bar{Y}_H\right)$$

where $\bar{Y}_T$ is the mean outcome in the treated group, $\bar{Y}_H$ is the mean outcome in the historical control, $\bar{Y}_C$ is the mean outcome a concurrent randomised control would have shown, and $\Delta = \bar{Y}_T - \bar{Y}_C$ is the true treatment effect. The bracketed term is the bias, and nothing in the historical comparison itself reveals its size.

Why historical controls tend to exaggerate benefit

E10 lists ways the groups can differ apart from the treatment, including demographic characteristics, diagnostic criteria, stage or severity of disease, concomitant treatments and the way outcomes are assessed. Some differences involve prognostic factors that were never measured, and neither blinding nor randomisation is available to protect against them. The NICE real-world evidence framework adds differences in care, data collection and time period, each of which can act as confounding of the treatment comparison.

The direction of the bias usually favours the new treatment. E10 notes that untreated historical control groups tend to have worse outcomes than apparently similar control groups in randomised studies, possibly reflecting a selection bias, because trial eligibility criteria usually identify a less sick population than is typical of external controls. An external control group is also often identified retrospectively, which opens a further route for bias in its selection.

The classic empirical evidence comes from Sacks, Chalmers and Smith (1982), who found six therapies studied with both designs, in 50 randomised and 56 historically controlled trials. Of the historically controlled trials, 44 (79%) found the therapy better than the control regimen, against 10 (20%) of the randomised trials. The gap arose largely from the control groups, with historical control patients generally doing worse, and adjusting for prognostic factors, where possible, did not appreciably change the results. The authors concluded that biases in patient selection may irretrievably weight historically controlled trials in favour of new therapies.

Conditions for a credible historical comparison

Wied, Posch and Hilgers (2024) recommend following six criteria from Pocock (1976) for the safe use of historical controls. As they report them, the standard treatment must be precisely defined and the same as for randomised controls, and the historical group must come from a clinical study with the same eligibility requirements. Treatment evaluation methods must match, patient characteristics must be comparable, the study must have been run in the same organisation with the same investigators, and nothing should lead one to expect a difference. Taken together, the conditions describe an earlier trial run by the same organisation, which most registry and routine data sets are not.

E10 adds design steps that make an externally controlled trial more persuasive. The control group should have detailed information, including individual patient data where pertinent, and should be chosen before comparative analyses are run, with any matching or adjustment specified in advance. E10 says the design should generally be considered only when prior belief in the test therapy is so strong that alternative designs appear unacceptable and the disease has a well-documented, highly predictable course. Even then it judges the possibility of substantial bias to be inherent in the design and impossible to quantitate.

Single-arm trials and external controls in regulatory and HTA submissions

Regulators treat a single-arm trial as a departure from the standard that needs justification. The EMA reflection paper on single-arm trials, adopted by the CHMP on 9 September 2024, states that randomised controlled trials are the standard for confirmatory evidence of efficacy, and that without a concurrent control arm or randomisation, interpretation must rely on knowledge external to the trial. It excludes guidance on external controls themselves, but warns that a success threshold based on external clinical information carries an inherent risk of erroneous conclusions from comparing results across data sources.

The NICE manual for technology appraisal and highly specialised technologies (PMG36) lists single-arm trials using external control among non-randomised studies, which tend to be at high risk of bias, and states that inferences about relative effects without randomisation will often be more uncertain than those from RCTs. The NICE real-world evidence framework (ECD9) describes external controls formed from real-world data or previous clinical trials, noting that earlier trials may offer greater similarity in inclusion criteria and outcome determination but often only aggregate data.

Unanchored comparisons and the cost-effectiveness model

In an HTA submission the historical control usually enters through an indirect comparison. With only single-arm data for the new treatment there is no common comparator arm, so the comparison is unanchored. NICE Decision Support Unit Technical Support Document 18 allows unanchored population adjustment only without a connected network of randomised studies, or where single-arm studies are involved.

Methods such as matching-adjusted indirect comparison and simulated treatment comparison can reweight or model away differences in observed covariates. TSD 18 explains that an unanchored version effectively assumes all effect modifiers and prognostic factors are accounted for, an assumption it calls very strong and largely considered impossible to meet, whose failure leads to an unknown amount of bias. Differences in treatment administration, co-treatments or treatment switching cannot be adjusted for in this way, since they are perfectly confounded with treatment.

For the economic evaluation, the bias term lands in the relative effect that drives incremental survival and QALYs. Probabilistic analysis describes the precision of inputs, so a systematic shift in the control arm leaves the ICER biased however narrow its interval appears. The bias has to be explored separately, through scenario analysis, ranges for residual bias of the kind TSD 18 suggests in its Appendix C, or tipping-point analysis, and the committee has to weigh it alongside the decision uncertainty that probabilistic analysis reports.

Worked example of a biased historical comparison

The following figures are illustrative only. A single-arm trial of a new oncology drug feeds a model in which patients on the drug accrue a mean of 1.60 QALYs. The only comparator data come from a registry of patients treated five years earlier, whose modelled mean is 1.10 QALYs, and the incremental cost is GBP 13,500.

The incremental gain is 1.60 minus 1.10, or 0.50 QALYs, so the base-case ICER is GBP 13,500 / 0.50 = GBP 27,000 per QALY gained.

Suppose a concurrent randomised control would have achieved 1.25 QALYs, because the registry patients were sicker and received older supportive care. The true gain is then 1.60 minus 1.25, or 0.35 QALYs, and the bias term is 1.25 minus 1.10, or 0.15 QALYs, so the estimated 0.50 equals 0.35 plus 0.15. Holding the incremental cost fixed for simplicity, the ICER becomes GBP 13,500 / 0.35 = GBP 38,571 per QALY gained, about 43% higher.

Against an illustrative threshold of GBP 30,000 per QALY, the ICER crosses the threshold once the incremental gain falls below 13,500 divided by 30,000, or 0.45 QALYs. Any control-arm bias above 0.05 QALYs, a tenth of the estimated gain, changes the decision, and probabilistic analysis around the 0.50 estimate would not show it.

Borrowing historical data alongside a randomised control

Historical data can also supplement a concurrent control instead of replacing it. Pocock's 1976 paper addressed this combination, and ECD9 notes that external data can augment concurrent controls within a randomised trial, using methods that place different weight on the external data.

Viele and colleagues (2014) review such borrowing. When the historical data are sufficiently similar to the current controls, it can give more accurate estimates, more power and reduced type I error, but when they are not, it brings bias and either reduced power or increased type I error. A power prior with a fixed weight borrows statically, giving the historical data the same weight whatever the current data show, while dynamic methods such as test-then-pool and hierarchical models, which place a prior distribution on the variation between studies, borrow most when the current controls agree with the historical data. The Fill-it-up design studied by Wied and colleagues first tests whether the two control groups are equivalent, pools them only if so, and otherwise extends randomisation.

Historical control, external control arm and single-arm trial

A historical control is the kind of external control arm drawn from an earlier period, and an external control can also be contemporaneous but from another setting. A single-arm trial is the design that usually needs one, since every participant receives the new treatment. An anchored indirect comparison differs because both trials share a randomised common comparator, and real-world evidence studies that emulate a target trial can reduce, but not remove, the comparability problems.

Common misreadings of historical comparisons

Statistical adjustment does not make a historical control equivalent to randomisation. Sacks and colleagues found that adjustment for prognostic factors made little difference, and E10 points to unmeasured prognostic factors that no adjustment can reach.

Data from an earlier trial are better than registry data on several counts, but they remain non-randomised, and the patients may still differ in time period and care.

A large observed effect persuades only under strict conditions. E10 accepts general knowledge as a plainly reasonable comparator when an effect is dramatic, rapid and unlikely to occur spontaneously, as with measurable tumour shrinkage, but notes that many promising therapies have had less dramatic effects than expected, or none, in controlled trials.

Sources

• European Medicines Agency, Committee for Medicinal Products for Human Use. Reflection paper on establishing efficacy based on single-arm trials submitted as pivotal evidence in a marketing authorisation application. Adopted 9 September 2024. https://www.ema.europa.eu/en/scientific-guidelines/establishing-efficacy-based-single-arm-trials-submitted-pivotal-evidence-marketing-authorisation
• International Conference on Harmonisation. ICH E10: Choice of control group and related issues in clinical trials. Step 4 version, 20 July 2000. https://database.ich.org/sites/default/files/E10_Guideline.pdf
• National Institute for Health and Care Excellence. NICE real-world evidence framework (ECD9). London: NICE; 2022, updated September 2025. https://www.nice.org.uk/corporate/ecd9
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36
• Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. NICE DSU Technical Support Document 18: Methods for population-adjusted indirect comparisons in submissions to NICE. Sheffield: NICE Decision Support Unit; 2016. https://www.sheffield.ac.uk/nice-dsu/tsds/population-adjusted
• Sacks H, Chalmers TC, Smith H Jr. Randomized versus historical controls for clinical trials. American Journal of Medicine. 1982;72(2):233-240. https://doi.org/10.1016/0002-9343(82)90815-4
• Viele K, Berry S, Neuenschwander B, et al. Use of historical control data for assessing treatment effects in clinical trials. Pharmaceutical Statistics. 2014;13(1):41-54. https://doi.org/10.1002/pst.1589
• Wied S, Posch M, Hilgers RD. Evaluation of the Fill-it-up-design to use historical control data in randomized clinical trials with two arm parallel group design. BMC Medical Research Methodology. 2024;24:197. https://doi.org/10.1186/s12874-024-02306-2