Contamination

Contamination in Randomised Trials and Economic Evaluation

A randomised controlled trial can only measure what an intervention adds if the comparison group stays free of it. When people in the control arm pick up the intervention from shared clinicians or from other participants, the two arms become more alike and the estimate from an intention-to-treat analysis shrinks towards no difference. For health economists the problem carries straight into trial-based cost-effectiveness estimates, which can dilute health gains more than costs, and the usual design remedy, cluster randomisation, has its own price in sample size and bias. The risk is high in trials of service and educational interventions, where advice and new ways of working pass easily between people in the same practice or school.

How contamination arises in trials of health services

Cochrane's risk of bias guidance describes contamination as the application of one of the interventions in participants intended to receive the other. Torgerson gives the classic example of a dietary trial in which people in the control group learn about the experimental diet and adopt it themselves.

Magill and colleagues reviewed 234 trials of complex interventions in mental health and grouped the processes the trialists described. The two most common were staff delivering the active intervention to control participants, reported in 82 trials (35%), usually because one clinician treated both arms, and communication between people in different arms, reported in 79 trials (34%). Less common routes were participants switching to a clinician trained in the intervention, participants seeking the treatment outside the trial, and background noise, where the treatment already existed to some extent in the health system.

Few trials measure how much contamination occurred. In the same review only 27 studies (12%) tried to quantify it, and in the 23 trials that measured binary receipt in the control arm the median proportion contaminated was 13% (interquartile range 5% to 33%).

Why contamination biases the estimate towards no difference

Contamination makes the control arm more like the intervention arm and so dilutes the contrast between randomised groups. Magill and colleagues note that the intention-to-treat estimator is then biased for the effect of actually receiving the treatment. Torgerson adds that the reduced point estimate may lead to a type II error, in which the trial misses a real effect.

A simple model shows the size of the problem when a proportion of control participants receive the intervention and gain its full effect, while everyone in the intervention arm receives it.

$$\Delta_{ITT} = (1 - c)\,\Delta$$

where $\Delta_{ITT}$ is the expected difference between arms in an intention-to-treat analysis, $\Delta$ is the true effect of receiving the intervention, and $c$ is the proportion of the control arm contaminated. Because the required sample size for a given power varies with the inverse square of the difference to be detected, contamination inflates it by a factor of $(1-c)^{-2}$, which is about 1.23 at 10% contamination, 1.56 at 20% and 2.04 at 30%. Torgerson's sample size table for a trial designed to detect a 25% effect follows this pattern closely, rising from 116 participants with no contamination to 144 at 10% and 236 at 30%. Hewitt and colleagues confirmed by simulation that statistical power declines as contamination increases.

The Cochrane RoB 2 tool places contamination under bias due to deviations from intended interventions, sometimes called performance bias. For the effect of assignment it counts only deviations that are inconsistent with the protocol, arise because of the experimental context and influence the outcome, such as unblinded participants who feel unlucky to be in the comparator group seeking the experimental intervention. Blinding, where it is possible, should prevent knowledge of assignment from driving contamination.

Cluster randomisation as the design response and its price

Cochrane lists avoiding contamination when participants are managed within the same setting, such as a trial of training clinicians in a clinic, as one of the main reasons for cluster designs, alongside measuring group effects such as herd immunity and simple convenience. Magill and colleagues add that this works only if clusters are built at the level at which contamination takes place.

The price is lost precision, because people in the same cluster tend to respond alike. The design effect measures how much a cluster design inflates the sample size.

$$D = 1 + (m - 1)\,\rho$$

where $D$ is the design effect, $m$ is the average cluster size and $\rho$ is the intraclass correlation coefficient. In Torgerson's table an intraclass correlation of 0.02 with clusters of 50 gives a design effect of 1.98, almost doubling the 116 participants an individually randomised trial needs. His conclusion was that, in terms of total sample size, cluster trials are only more efficient where contamination exceeds 30%. Hewitt and colleagues reached a similar threshold for an individually randomised trial analysed with a complier average causal effect method, with clusters of 10 and an intraclass correlation of 0.04.

Cluster designs bring bias of their own. Torgerson warns that cluster trials can be susceptible to recruitment bias, and Magill and colleagues found that cluster trials more often randomised before consent and before baseline measures were completed. The Keogh-Brown HTA monograph found by simulation that cluster trials might produce biases greater than or similar to those of individually randomised trials, concluding that cluster randomisation should not be uncritically assumed always to be a solution. Many trials instead limit contamination through conduct, for example by having each clinician deliver only one treatment or separating the arms in space or time.

Analysis choices when contamination has occurred

An intention-to-treat analysis estimates the effect of assignment, which Cochrane regards as the effect most relevant to a policy question such as whether to introduce a screening programme. Dropping contaminated participants in a per-protocol analysis, or analysing people by the treatment they received, breaks randomisation, because people who deviate may differ in prognosis from those who do not. Sussman and Hayward describe a contamination adjusted intention-to-treat analysis that uses random assignment as the instrument in an instrumental variable analysis.

$$\Delta_{CACE} = \frac{\Delta_{ITT}}{p_I - p_C}$$

where $\Delta_{CACE}$ is the complier average causal effect, $\Delta_{ITT}$ is the intention-to-treat difference, and $p_I$ and $p_C$ are the proportions of the intervention and control arms that actually received the intervention. The Keogh-Brown monograph found complier average causal effect analyses less biased than intention-to-treat or per-protocol analyses and judged them appropriate for adjusting for contamination if it can be measured. Sussman and Hayward note the main assumption, that the treatment would have had the same effect in those who deviated as in those who adhered.

Contamination in trial-based economic evaluation

A trial-based cost-effectiveness analysis inherits the diluted effect, but costs need not be diluted in the same way. When contamination travels through information or advice, control participants gain some of the benefit while the trial records none of the intervention's cost against them, so the incremental cost stays whole, the incremental health gain shrinks and the ICER rises. When control participants receive intervention sessions from shared staff, some cost is diluted along with the effect. Sussman and Hayward point to the related mismatch of an intention-to-treat benefit paired with a treatment cost that assumes full adherence, which underestimates cost-effectiveness. Resource use should therefore be measured as delivered in both arms.

A commissioner adopting a service compares a world with the programme against one without it, and contamination created by the trial setting, such as control clinicians in the same practice learning the new approach, would be absent from that counterfactual. In that case the trial understates the effect of adoption. Contamination reflecting routine practice, where the treatment is already available, belongs in that comparison.

When cluster randomisation is chosen to avoid contamination, the economic analysis must allow for clustering. Gomes and colleagues found that cost-effectiveness methods ignoring clustering underestimate statistical uncertainty and the value of information from further research, and recommended multilevel models or a two-stage bootstrap.

Worked example of a diluted cost-effectiveness estimate

All numbers in this example are illustrative. A nurse-led self-management programme costs GBP 400 per participant and gives a true gain of 0.05 QALYs. It is tested in a trial randomising individuals within general practices, and 20% of control participants pick up the programme materials from friends in the intervention arm, gaining the full effect at no cost to the health service.

1. Observed health gain.

$$\Delta Q_{ITT} = (1 - 0.20) \times 0.05 = 0.04$$

where $\Delta Q_{ITT}$ is the QALY difference between arms in the intention-to-treat analysis.

2. Cost-effectiveness. The incremental cost stays at GBP 400. The true ICER is 400 / 0.05 = GBP 8,000 per QALY and the observed ICER is 400 / 0.04 = GBP 10,000 per QALY. At an illustrative threshold of GBP 20,000 per QALY, true net monetary benefit is 20,000 × 0.05 − 400 = GBP 600 per participant and observed net monetary benefit is 20,000 × 0.04 − 400 = GBP 400.

3. Adjusting for contamination. With $p_I = 1.0$ and $p_C = 0.2$, the complier average causal effect is 0.04 / (1.0 − 0.2) = 0.05 QALYs, which recovers the true gain, provided the contaminated participants benefited as much as the treated ones.

4. Sample size. Contamination of 20% inflates the sample size by 1 / 0.8² = 1.56. A cluster design with 20 participants per practice and an intraclass correlation of 0.05 has a design effect of 1 + 19 × 0.05 = 1.95.

5. Interpretation. Contamination raised the ICER by 25% and cut the net benefit by a third, although the decision at this threshold does not change. Individual randomisation with about 56% more participants needs fewer people than a cluster design needing 95% more, matching Torgerson's point that substantial contamination can be tolerated before a cluster design becomes more efficient.

How contamination differs from treatment switching and cointervention

Treatment switching usually means that control patients in a drug trial start the experimental treatment during follow-up, often after disease progression and with the protocol's permission, and has its own adjustment methods. Contamination in the sense used here usually spreads between people through shared settings and is not planned by the protocol. Both dilute the intention-to-treat contrast, and Torgerson treats crossover between treatments in cancer trials using Zelen's randomised consent design as analogous to contamination. Terminology varies, since Sussman and Hayward use treatment contamination to cover any departure from the assigned treatment, including non-adherence in the intervention arm.

A cointervention is different again. RoB 2 treats non-protocol interventions received alongside the intended intervention as a separate deviation, and the question is whether they are balanced across groups. A cointervention is extra care such as additional monitoring or another therapy, while contamination is exposure to the trial's own intervention.

Where contamination analyses commonly go wrong

The dilution formula assumes contaminated participants receive the full intervention and its full effect, so partial exposure dilutes less than the formula suggests. A frequent error is to treat cluster randomisation as automatically correct, when it trades contamination for larger samples and new sources of bias. Adjustment methods need measured receipt of the intervention in the control arm, and Magill and colleagues found that such measurement was rare. Reporting the extent, nature and effects of contamination, as the Keogh-Brown monograph recommended, lets reviewers and modellers judge how far a trial's estimate understates the effect of adoption.

Sources

• Gomes M, Ng ES, Grieve R, Nixon R, Carpenter J, Thompson SG. Developing appropriate methods for cost-effectiveness analysis of cluster randomized trials. Medical Decision Making. 2012;32(2):350-361. https://doi.org/10.1177/0272989X11418372
• Hewitt CE, Torgerson DJ, Miles JNV. Individual allocation had an advantage over cluster randomization in statistical efficiency in some circumstances. Journal of Clinical Epidemiology. 2008;61(10):1004-1008. https://doi.org/10.1016/j.jclinepi.2007.12.002
• Higgins JPT, Eldridge S, Li T. Chapter 23: Including variants on randomized trials. In: Higgins JPT, Thomas J, Chandler J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-23
• Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial. In: Higgins JPT, Thomas J, Chandler J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08
• Keogh-Brown MR, Bachmann MO, Shepstone L, Hewitt C, Howe A, Ramsay CR, et al. Contamination in trials of educational interventions. Health Technology Assessment. 2007;11(43). https://doi.org/10.3310/hta11430
• Magill N, Knight R, McCrone P, Ismail K, Landau S. A scoping review of the problems and solutions associated with contamination in trials of complex interventions in mental health. BMC Medical Research Methodology. 2019;19:4. https://doi.org/10.1186/s12874-018-0646-z
• Sussman JB, Hayward RA. An IV for the RCT: using instrumental variables to adjust for treatment contamination in randomised controlled trials. BMJ. 2010;340:c2073. https://doi.org/10.1136/bmj.c2073
• Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ. 2001;322(7282):355-357. https://doi.org/10.1136/bmj.322.7282.355