Large Simple Trial

Large Simple Trial Design and Mortality Evidence for HTA

Large simple trials exist because treatments for the common causes of death rarely halve mortality, and a moderate benefit can only be told apart from chance when thousands of patients are compared. The design buys that size by removing everything that does not protect the randomised comparison, so that busy clinicians can enrol patients as part of normal care. For health technology assessment the result is a precise relative effect on mortality in patients who resemble routine practice, often at low cost per participant. The price is thin data on costs and quality of life, which analysts must find elsewhere.

Why moderate effects on survival need very large numbers

Peto and Baigent argued in 1998 that a trial about life or death must avoid being misled by the play of chance as well as by bias. A very large effect on survival, such as penicillin's, can be recognised without randomisation, so any treatment still in doubt is likely to have a small or moderate effect. A realistic hope for acute stroke or myocardial infarction is a fall in death or recurrence in hospital from 10% to 9% or 8%, as with aspirin, which prevents 10,000 or 20,000 deaths per million treated.

Yusuf, Collins and Peto had set out the case for such trials in Statistics in Medicine in 1984. Yusuf later explained that early infectious disease trials expected large effects and could be small, whereas treatments for cardiovascular disease or cancer were likely to have at best moderate effects, which led to trials of tens of thousands of people. Califf adds that a small benefit across a very common disease becomes a major public health effect.

A trial's statistical power to detect a modest absolute difference depends largely on the number of events, so a moderate effect on a 10% to 20% mortality risk needs thousands of patients per arm. Meta-analysis can pool smaller trials, but Peto and Baigent noted that even all the trials in the world on a question are sometimes too few.

Simplicity as the means to size

Peto and Baigent saw extreme simplicity in entry criteria, treatments and data as generally the only practical way to randomise very large numbers. They observed that many trials collect ten or a hundred times too much information per patient, and that defensive documentation which makes trials smaller can reduce the reliability of the answer.

Entry rests on the uncertainty principle. A patient is entered if, and only if, the responsible clinician is substantially uncertain which trial treatment is most appropriate for that patient. It is a judgement about one patient, unlike equipoise among informed experts.

Yusuf describes the International Studies of Infarct Survival as the model, with one page of data per participant, no site monitoring, no adjudication of events and no fees to investigators. Differences between centres in entry decisions and ancillary care did not matter, because randomisation within each centre spread them evenly across the arms.

ISIS-2 and RECOVERY as reference cases

ISIS-2, published in 1988, randomised 17,187 patients with suspected acute myocardial infarction in 417 hospitals, within 24 hours of symptom onset, to streptokinase, aspirin, both or neither against placebo. Five-week vascular mortality was 9.2% with streptokinase against 12.0% with placebo infusion, and 9.4% with aspirin against 11.8% with placebo tablets. Patients allocated both active agents had 8.0% vascular mortality against 13.2% with neither, about five fewer vascular deaths per 100 treated.

The RECOVERY trial in hospitalised COVID-19 patients is the modern example, and its dexamethasone report says the protocol combined the methods of the 1980s infarction trials with digital health care. Pessoa-Amorim and colleagues describe how any patient hospitalised with proven or suspected COVID-19 and judged suitable by the clinical team was eligible. Entry took a short form averaging 10 minutes, a brief follow-up form was completed at discharge, death or 28 days, and only 9 days passed from protocol finalisation to the first patient.

Linkage to more than 25 routine NHS and national registry datasets, including hospital admissions, intensive care and death registrations, gave over 99% follow-up for the primary outcome. In the final dexamethasone analysis, 482 of 2,104 patients (22.9%) allocated dexamethasone and 1,110 of 4,321 (25.7%) allocated usual care died within 28 days, an age-adjusted rate ratio of 0.83 (95% CI 0.75 to 0.93). Dexamethasone is available worldwide at low cost.

Registry-based and routine-data versions

Registry-based randomised trials place recruitment, randomisation or outcome collection inside an existing clinical registry. The TASTE trial enrolled 7,244 patients with ST-elevation myocardial infarction through the Swedish coronary angiography and angioplasty registry and assessed end points through national registries, with no patient lost to follow-up. All-cause death at 30 days was 2.8% with thrombus aspiration and 3.0% without, a hazard ratio of 0.94 (95% CI 0.72 to 1.22), so routine thrombus aspiration did not reduce 30-day mortality.

A 2024 systematic review by Shiely and colleagues found 24 such trials, recruiting from 209 to 106,000 participants, and lists recruitment efficiency, cost effectiveness and complete outcome data among the advantages. Challenges included data management, quality assurance and the timing of consent, and only three of the trials assessed cost-effectiveness. The NICE real-world evidence framework (ECD9, 2022) notes that randomised trials using real-world data for design or outcome measurement can combine the internal validity of randomisation with the generalisability of routine practice.

What the evidence offers an economic evaluation

The NICE manual (PMG36) treats randomised trials as giving a minimally biased estimate of relative treatment effect, but warns that results may not generalise to the population, setting or treatment pathway of interest. A large simple trial eases that concern through broad entry, and gives a precise relative effect on hard outcomes such as all-cause mortality, which drive a model's life-years and QALYs. PMG36 suggests applying trial relative effects to the baseline risk of the population of interest, and a very large trial has the numbers to check whether the relative effect holds across baseline risks.

The gaps lie on the cost and utility side. The 2015 ISPOR task force recommends that trials with an economic evaluation alongside include clinical outcome measures and obtain health resource use and health state utilities directly from participants, which sits uneasily with a minimal case report form. The PMG36 reference case wants health-related quality of life reported directly by patients or carers, and treats mapping from other measures to EQ-5D as a departure from it. Analysts therefore combine the trial's mortality effect with utilities and unit costs from other studies, or draw resource use from linked hospital records.

Trial cost is itself an economic question. Eisenstein and colleagues modelled a 20,000-patient trial at 1,000 sites costing USD 421 million (2007 prices); in their model a streamlined industry design cut total cost by 59%, and an even simpler design by 90%. Yusuf argued that bureaucratic and funding barriers have potentially hindered trials of generic therapies, such as new uses of old drugs, for which industry funding is usually hard to obtain. Value of information analysis weighs the cost of such a trial against the expected gain from reducing decision uncertainty.

Worked example of power and precision for a moderate mortality effect

All numbers are illustrative and follow the planning statement reported for RECOVERY's dexamethasone comparison. The trial asks whether a cheap treatment cuts 28-day mortality from 20% with usual care to 16%, with 4,000 patients on usual care and 2,000 on treatment.

$$SE = \sqrt{\frac{p_0(1-p_0)}{n_0} + \frac{p_1(1-p_1)}{n_1}}$$

where $SE$ is the standard error of the difference in risk, $p_0 = 0.20$ and $p_1 = 0.16$ are the mortality risks with usual care and treatment, and $n_0 = 4000$ and $n_1 = 2000$ are the arm sizes. This gives $SE = \sqrt{0.00004 + 0.0000672} = 0.0104$.

$$Power \approx \Phi\left(\frac{p_0 - p_1}{SE} - z_{1-\alpha/2}\right)$$

where $\Phi$ is the standard normal distribution function and $z_{1-\alpha/2} = 2.576$ is the critical value for a two-sided significance level of 0.01. The difference of 0.04 divided by 0.0104 is 3.86, so power is about $\Phi(1.29) = 0.90$, matching the reported design of over 90% power.

The same true effect gives very different inputs to a cost-effectiveness analysis. At GBP 50 per patient, treating 1,000 patients costs GBP 50,000 and avoids 40 deaths at the point estimate, or GBP 1,250 per death avoided, and the large trial's limits give GBP 2,500 (20 deaths) and GBP 833 (60 deaths). The small trial's interval includes harm, leaving real decision uncertainty about whether the treatment does any good.

Boundaries with pragmatic, explanatory and registry trials

Size and simplicity are separate from where a trial sits between the explanatory and pragmatic poles. ISIS-2 used placebo controls, which Thorpe and colleagues treat as a sign of an explanatory trial, yet it was large and simple. RECOVERY compared treatments with usual care in an open-label design, and its investigators call it pragmatic. Registry-based trials are one way to make a trial both large and pragmatic.

RECOVERY also used an adaptive trial design, adding and dropping arms under a master protocol. Califf contrasts the large simple trial with large complex trials, which carry far higher costs, and with small biomarker-driven trials seeking a large effect in a targeted group.

Limitations and common misreadings

Califf cited an analysis of ClinicalTrials.gov in which only 3% to 4% of interventional trials enrolled more than 5,000 participants and over 90% fewer than 100. He also noted that the focus on hard outcomes ignores patient-reported outcomes, which become critically important when hard outcomes do not differ.

One misreading equates simple with careless. Yusuf stresses that randomisation and unbiased outcome assessment protect the main result, and RECOVERY limited open-label bias through objective outcomes and record linkage.

A second misreading treats a huge trial as licence for subgroup claims. Peto and Baigent warned that selective emphasis on particular subgroup analyses can mislead even when large randomised evidence exists.

A third misreading assumes that a cheap trial gives cheap economic evidence. The trial supplies a precise effect on death or other hard outcomes, but costs, utilities and long-term extrapolation still come from elsewhere.

Sources

• Califf RM. Large simple trials: really, it can't be that simple! European Heart Journal. 2014;35(9):549-551. https://doi.org/10.1093/eurheartj/eht556
• Eisenstein EL, Collins R, Cracknell BS, et al. Sensible approaches for reducing clinical trial costs. Clinical Trials. 2008;5(1):75-84. https://doi.org/10.1177/1740774507087551
• Fröbert O, Lagerqvist B, Olivecrona GK, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. New England Journal of Medicine. 2013;369(17):1587-1597. https://doi.org/10.1056/NEJMoa1308789
• ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 1988;2(8607):349-360. https://pubmed.ncbi.nlm.nih.gov/2899772/
• National Institute for Health and Care Excellence. NICE real-world evidence framework (ECD9). Published 23 June 2022. https://www.nice.org.uk/corporate/ecd9
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36
• Pessoa-Amorim G, Campbell M, Fletcher L, et al. Making trials part of good clinical care: lessons from the RECOVERY trial. Future Healthcare Journal. 2021;8(2):e243-e250. https://doi.org/10.7861/fhj.2021-0083
• Peto R, Baigent C. Trials: the next 50 years. Large scale randomised evidence of moderate benefits. BMJ. 1998;317(7167):1170-1171. https://doi.org/10.1136/bmj.317.7167.1170
• Ramsey SD, Willke RJ, Glick H, et al. Cost-effectiveness analysis alongside clinical trials II. An ISPOR Good Research Practices Task Force report. Value in Health. 2015;18(2):161-172. https://doi.org/10.1016/j.jval.2015.02.001
• RECOVERY Collaborative Group. Dexamethasone in hospitalized patients with Covid-19. New England Journal of Medicine. 2021;384(8):693-704. https://doi.org/10.1056/NEJMoa2021436
• Shiely F, O Shea N, Murphy E, Eustace J. Registry-based randomised controlled trials: conduct, advantages and challenges, a systematic review. Trials. 2024;25:375. https://doi.org/10.1186/s13063-024-08209-3
• Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ. 2009;180(10):E47-E57. https://doi.org/10.1503/cmaj.090523
• Yusuf S. Randomized clinical trials: slow death by a thousand unnecessary policies? CMAJ. 2004;171(8):889-892. https://doi.org/10.1503/cmaj.1040884
• Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Statistics in Medicine. 1984;3(4):409-422. https://doi.org/10.1002/sim.4780030421