Non-Randomised Controlled Trial

Non-Randomised Controlled Trial in Health Technology Assessment

In a non-randomised controlled trial the investigators still decide who receives the new intervention and who forms the comparison group, but the rule they use, such as alternation, date of birth or clinician choice, contains no element of chance. That one departure from a randomised controlled trial lets prognostic factors influence allocation, so the estimated effect can be confounded in ways that statistical adjustment repairs only in part. For health economists the design matters because NICE allows non-randomised studies to be the primary source of relative effectiveness when no randomised evidence exists, and a biased relative effect moves costs, QALYs and the ICER together.

Where the design sits among studies of interventions

The Cochrane Effective Practice and Organisation of Care group (EPOC) sorts designs by two questions, whether the investigators assigned the interventions and whether they did so randomly. A yes and then a no gives a non-randomised trial. EPOC prefers that name to "controlled clinical trial" and to "quasi-randomised controlled trial", a term it says different authors use to mean different things.

The Cochrane Handbook counts controlled trials with inappropriate allocation strategies, sometimes called quasi-randomised studies, among non-randomised studies of interventions (NRSI). Quasi-random allocation is therefore a subtype of the non-randomised controlled trial, in which a fixed rule such as alternation, date of birth or day of admission replaces a chance process. Because the researcher actively assigns each intervention, the design differs from an observational study, in which allocation happens in usual care or by people's own choices, and the Handbook notes that misclassification of intervention status is seldom a problem in experimental studies.

How allocation without chance lets confounding in

Randomisation makes differences in baseline characteristics, measured or not, a matter of chance. Without it, confounding occurs whenever a factor influences both the choice of intervention and the outcome, such as disease severity, comorbidities, healthcare use, prescribing practice or socio-economic status. When clinicians allocate, the reason a patient is offered the new treatment is often tied to prognosis, which is confounding by indication.

A quasi-random rule looks safer, since a date of birth has no link to outcome. The Handbook explains that knowledge of a deterministic rule lets recruiters enrol participants selectively by prognosis, for example by rejecting a patient or delaying entry until the desired allocation comes up, so a predictable rule removes the protection of allocation concealment. Blinding of participants, professionals or outcome assessors is also usually not feasible in non-randomised studies.

Assessing risk of bias with ROBINS-I

ROBINS-I judges each result against a target trial, a hypothetical pragmatic randomised trial of the same interventions in the same participants with no flaws in its conduct. Bias is the systematic difference from that target trial, kept separate from chance and from applicability. The 2016 version described in the Handbook has seven domains.

Judgements are low, moderate, serious or critical risk. Low means comparable to a well-performed randomised trial, which the Handbook expects design or analysis to achieve only rarely for intended effects, because of confounding. A result at critical risk should not be included in any synthesis. A draft ROBINS-I V2, posted in November 2025 for follow-up (cohort) studies and still subject to change, adds algorithms that map answers to judgements and a triage step to critical risk, and drops the separate domain on deviations from intended interventions.

Adjustment methods and what they can repair

The NICE real-world evidence framework (ECD9) lists stratification, matching, multivariable regression and propensity score methods, all of which assume no unmeasured confounding. Residual confounding remains when a confounder is measured with error or modelled imperfectly, and the Handbook states that unmeasured confounding can usually not be excluded.

Deeks and colleagues built non-randomised comparisons by resampling patients from two large randomised trials. None of four case-mix adjustment strategies adequately removed the bias in historically and concurrently controlled comparisons, logistic regression on average increased it, and propensity score methods did better but were not satisfactory in most situations. A Cochrane methodology review found non-randomised studies giving larger effects than randomised trials in some comparisons and smaller in others, and concluded that the size and even the direction of the distortion cannot generally be predicted. ECD9 suggests quantitative bias analysis for key risks, including the E-value approach, which asks how strong an unmeasured confounder would need to be to remove the effect, and negative control outcomes.

How NICE and Cochrane treat the evidence

The NICE manual (PMG36) states a strong preference for high-quality randomised trials for relative treatment effects, but allows non-randomised studies to complement them when evidence is limited or to form the primary source when there is none. It accepts that randomised trials may be unethical or not feasible, and that learning effects with some medical devices may not be captured in one. Where a randomised trial is not available or appropriate, the choice of evidence must be justified and potential bias explored in a formal, transparent and pre-specified way.

PMG36 describes non-randomised studies, interventional or observational, as tending to be at high risk of bias from confounding, with relative effects that will often be more uncertain than those from randomised trials. It asks for each study to be assessed with a validated tool such as ROBINS-I and for extensive sensitivity analyses covering all key risks of bias, and points to Decision Support Unit technical support document 17 for adjusting for confounding with individual patient data from observational studies. ECD9 adds that such studies should be designed to emulate the randomised trial that would ideally have been run.

For systematic reviews, the Handbook recommends analysing randomised trials and NRSI separately rather than combining them in one meta-analysis. EPOC considers non-randomised trials for health system interventions and advises that non-randomised cluster trials include at least two intervention and two control sites, so that the intervention is not confounded by site.

Carrying a non-randomised effect into an economic model

A cost-effectiveness analysis usually applies a relative treatment effect to a baseline risk, so bias in the relative effect passes directly into incremental costs and QALYs, and PMG36 asks how bias was assessed and addressed for every model input source. The reported confidence interval reflects random error only, and the Handbook warns that intervals from larger NRSI are less likely to represent the true uncertainty. A probabilistic analysis built on that interval alone therefore understates parameter uncertainty. A scenario analysis with unadjusted and adjusted estimates, and a threshold analysis on the effect size, show how far the decision depends on the design.

Worked example with clinician-selected allocation to a rehabilitation service

All numbers are illustrative. In a non-randomised controlled trial of a rehabilitation service, clinicians allocate 500 patients to the service and 500 to usual care, tending to choose patients with less severe disease. The service arm has 350 low-severity and 150 high-severity patients, the usual care arm 200 and 300. Within each stratum the service truly lowers readmission risk by the same proportion, from 0.10 to 0.08 in low severity and from 0.30 to 0.24 in high severity, a relative risk of 0.80 in both.

The service arm has 28 plus 36, or 64, readmissions, a risk of 0.128. The usual care arm has 20 plus 90, or 110, a risk of 0.22. The crude relative risk is 0.128 divided by 0.22, or 0.58, whereas stratifying by severity recovers 0.80. A model applies each estimate to a baseline readmission risk of 0.20.

$$ARR = p_0 (1 - RR), \qquad \Delta C = c_T - ARR \times c_R, \qquad \Delta E = ARR \times q_R$$

where $ARR$ is the absolute risk reduction in readmissions, $p_0$ is the baseline risk (0.20), $RR$ is the relative risk, $\Delta C$ is the incremental cost per patient in GBP, $c_T$ is the cost of the service (GBP 600), $c_R$ is the cost of a readmission (GBP 5,000), $\Delta E$ is the incremental QALYs per patient and $q_R$ is the QALY loss from a readmission (0.30).

The crude column uses unrounded values. At an illustrative cost-effectiveness threshold of GBP 20,000 per QALY, the net health benefit, incremental QALYs minus incremental cost divided by 20,000, changes sign. Confounding by severity alone makes a service that displaces more health than it adds look cost-effective.

Adjusting for measured severity does not settle the question, because clinicians may have selected on factors nobody recorded. A threshold analysis gives the relative risk at which the ICER equals the threshold.

$$RR^{*} = 1 - \frac{c_T}{p_0 (c_R + \lambda q_R)} = 1 - \frac{600}{0.20 \times 11000} = 0.727$$

where $RR^{*}$ is the threshold relative risk and $\lambda$ is the threshold (GBP 20,000 per QALY), so $\lambda q_R$ is GBP 6,000. The service is cost-effective only if the true relative risk is below about 0.73.

Boundaries with neighbouring designs

A randomised trial that allocates whole groups by chance uses cluster randomisation and is still a randomised design. A quasi-experimental design such as an interrupted time series or difference-in-differences analysis exploits external variation in exposure rather than allocation by the investigators, and in a controlled before-after study EPOC notes that the investigators do not make the allocation decisions. PMG36 also lists single-arm trials with an external control among non-randomised studies. These lack a concurrent comparison group, and ECD9 notes that differences in patients, settings, care, data collection and time periods can limit comparability with the external data.

Common misreadings of non-randomised trials

A quasi-randomised trial is not nearly randomised, because its rule can be predicted and subverted. Deeks and colleagues found that non-randomised studies may still mislead seriously when groups look similar in key prognostic factors, and that adjusted results can sometimes appear more biased than unadjusted ones. The Handbook accepts that large effects are less likely to be wholly explained by bias, but notes that clinical and economic decisions still need unbiased estimates of their size, which is the input a cost-effectiveness model requires.

Sources

• Cochrane Effective Practice and Organisation of Care (EPOC). What study designs can be considered for inclusion in an EPOC review and what should they be called? EPOC Resources for review authors; 2017. https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/Resources-for-authors2017/what_study_designs_should_be_included_in_an_epoc_review.pdf
• Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomised intervention studies. Health Technology Assessment. 2003;7(27):iii-x, 1-173. https://doi.org/10.3310/hta7270
• Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial (last updated October 2019). In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08
• National Institute for Health and Care Excellence. NICE real-world evidence framework (ECD9). London: NICE; 2022. https://www.nice.org.uk/corporate/ecd9
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36
• Odgaard-Jensen J, Vist GE, Timmer A, et al. Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews. 2011;(4):MR000012. https://doi.org/10.1002/14651858.MR000012.pub3
• Reeves BC, Deeks JJ, Higgins JPT, Shea B, Tugwell P, Wells GA. Chapter 24: Including non-randomized studies on intervention effects (last updated October 2019). In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-24
• Sterne JAC, Hernán MA, McAleenan A, Reeves BC, Higgins JPT. Chapter 25: Assessing risk of bias in a non-randomized study (last updated October 2019). In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-25
• Sterne JAC, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919
• ROBINS-I V2 development group. Risk Of Bias In Non-randomized Studies of Interventions, Version 2 (ROBINS-I V2), draft of November 2025. https://www.riskofbias.info/welcome/robins-i-v2