Cluster Randomization

Cluster Randomization of Practices, Wards and Communities in Trials and Their Economic Evaluation

In a cluster randomised trial the unit given a random allocation is a group of people, such as a general practice, a hospital ward, a school or a village, and everyone in that group receives the same strategy. The price is precision, because people in the same group have similar outcomes and costs, so 600 patients in 30 practices carry much less information than 600 patients randomised one by one. For health economics the design matters because many service delivery and public health interventions act on whole practices or areas, and the cost-effectiveness analysis must respect the clustering as much as the clinical analysis does.

Why trials randomise groups instead of patients

Chapter 23 of the Cochrane Handbook gives three reasons for a cluster design. The intervention may have a group effect, such as the herd immunity of a vaccine; individual allocation within one setting may cause contamination, as when clinicians trained in a clinic treat patients from both arms; or the design may be more convenient. Billot and colleagues regard it as appropriate for cluster-level interventions or a risk of contamination, despite its loss of power against an individually randomised trial of the same size.

Gomes and colleagues add interventions that operate at group level, such as incentives for health providers, and note that agencies may use cost-effectiveness analyses of these trials when recommending public health interventions. It is one form of randomisation in a randomised controlled trial, chosen when the decision itself concerns practices, wards or areas.

Similar outcomes within clusters and the design effect

Patients in the same practice share clinicians, procedures and local habits, so their outcomes and costs are more alike than those of patients elsewhere. Gomes and colleagues summarise this with the intraclass correlation coefficient (ICC), the proportion of the overall variation that lies at the cluster level. The Cochrane Handbook reports that ICCs below 0.05 are typical, that even small values can substantially widen confidence intervals when clusters are large, and that larger natural clusters tend to have smaller ICCs.

The usual planning tool is the design effect, which the Cochrane Handbook gives approximately as follows.

$$DE = 1 + (M - 1) \times ICC$$

where $DE$ is the design effect, $M$ is the average cluster size and $ICC$ is the intracluster correlation coefficient. In the standard approach described by the NIH Collaboratory Biostatistics and Design Core, the sample size needed without clustering is inflated by the design effect. Dividing an arm's sample size by the design effect gives its effective sample size, and the Handbook treats multiplying a naive standard error by the square root of the design effect as equivalent.

The formula assumes clusters of equal size. Eldridge and colleagues expected many trials randomising UK general practices to have a coefficient of variation of cluster size of about 0.65. They found the adjustment negligible below 0.23 and concluded that most such trials should allow for variable cluster size in their sample size calculation.

Worked example of planning a practice-level medicines review trial

All numbers in this example are illustrative. A health board wants to test a pharmacist-led medicines review delivered by whole general practices. A power calculation for an individually randomised trial gives 300 patients per arm. Each practice can recruit 20 eligible patients, and the ICC for the primary outcome is taken as 0.05.

1. Design effect. For clusters of 20 the design effect is 1 + 19 × 0.05 = 1.95.

2. Patients and practices needed. The arm needs 300 × 1.95 = 585 patients, which is 585 / 20 = 29.25 practices, rounded up to 30 practices (600 patients) per arm.

3. More patients per practice. If each practice recruited 40 patients instead, $DE$ = 1 + 39 × 0.05 = 2.95, so the arm needs 300 × 2.95 = 885 patients, or 885 / 40 = 22.1, rounded up to 23 practices (920 patients). Seven fewer practices cost 320 more patients per arm, because extra patients within a cluster add less and less information.

4. A lower ICC. With an ICC of 0.01 and 20 patients per practice, $DE$ = 1.19, so the arm needs 300 × 1.19 = 357 patients, or 17.85, rounded up to 18 practices. Assuming an ICC of 0.05 instead of 0.01 makes the trial two thirds larger, 600 patients per arm against 360.

The interpretation for a funder is that the number of clusters, not the number of patients, sets the limit on what a cluster trial can show.

Recruitment after allocation and other sources of bias

The Cochrane Handbook explains that individuals are often recruited after their cluster has been allocated, so knowledge of the allocation can influence who is recruited or selected for analysis; it calls this identification or recruitment bias. Puffer and colleagues, reviewing 36 cluster trials published in the BMJ, the Lancet and the New England Journal of Medicine from 1997 to 2002, found some evidence of susceptibility to individual-level bias in 14 (39%).

With few clusters, chance imbalance is also more likely. The Cochrane Handbook advises that chance imbalances in cluster trials should not be read as evidence of bias, and Billot and colleagues point out that imbalance can still bias the estimate when the unbalanced variable is strongly related to the outcome, with any adjustment pre-specified. The Collaboratory authors describe stratification and constrained randomisation, which restricts allocation to schemes with acceptable baseline balance, and single out cluster size as worth balancing.

Analysing a cluster randomised trial

Analysing a cluster trial as if individuals had been randomised is a unit-of-analysis error. The Cochrane Handbook states that confidence intervals then become artificially narrow and P values artificially small, which can produce false-positive conclusions. Billot and colleagues describe cluster-level analysis, which compares cluster summaries and suits very few clusters but makes covariate adjustment harder, and individual-level analysis. The latter uses a mixed effects model with a random cluster effect, giving a cluster-specific effect, or generalized estimating equations, giving a population-averaged effect; the two agree for differences in means and risk ratios but not for odds ratios.

Billot and colleagues illustrate the cost of the error with UKGRIS, a trial of risk stratification in suspected acute coronary syndrome that randomised 38 hospitals. The odds ratio for guideline-recommended management was 1.16 (95% confidence interval 0.70 to 1.92, P = 0.56), whereas a naive chi-square test ignoring clustering would have given P < 0.001. When the number of clusters is small, typically under 30, they report that both mixed models and GEE can inflate the type I error rate and need small-sample corrections.

Cost-effectiveness analysis alongside a cluster randomised trial

An economic evaluation must allow for clustering in costs and outcomes and for the correlation between them. In a systematic review of 62 economic evaluations that used cluster trials, Gomes and colleagues found that more than 40% used statistical methods that completely ignored clustering, 75% disregarded any correlation between costs and outcomes, and only 4 studies used methods that allowed for both. Grieve and colleagues, in a postnatal depression trial across 70 primary care providers, found an ICC of 0.17 for costs, and their hierarchical models gave substantially more uncertainty around cost-effectiveness than models that ignored the hierarchy.

In a simulation study, Gomes and colleagues found that multilevel models and a two-stage bootstrap with a shrinkage correction, resampling clusters and then individuals, performed well in each scenario. Seemingly unrelated regression and GEE with robust standard errors performed poorly with few clusters and, the authors suggest, need 8 to 15 clusters per arm. In their case study, ignoring clustering underestimated statistical uncertainty and the expected value of further research, and they caution that a low estimated ICC is not a reason to ignore clustering. Flynn and Peters, by contrast, found poor coverage for cluster-adjusted bootstrap intervals for mean cost differences at typical numbers of clusters, and put 24 clusters per arm as about the minimum before the bootstrap is even considered over robust methods. Gomes and colleagues note that their own results differ from this earlier work.

Costs also arise at two levels. Training and facilitation are incurred once per cluster, while visits and medicines accrue per patient. A practice training cost of GBP 3,000 (illustrative) is GBP 150 per patient in a trial practice with 20 eligible patients but GBP 50 per patient in a roll-out practice with 60, so the cost-effectiveness found in the trial may not transfer to practices of a different size.

Worked example of clustering and the net benefit decision

Allowing for clustering mainly widens the uncertainty around cost-effectiveness, although hierarchical models can also shift the point estimate. Continuing the illustrative trial, suppose an analysis that treats patients as independent estimates the incremental net benefit at a willingness to pay of GBP 20,000 per QALY as GBP 600 per patient, with a standard error of GBP 250. As a rough approximation, the design effect of 1.95 is assumed to apply to net benefit.

$$SE_{c} = SE \times \sqrt{DE} = 250 \times \sqrt{1.95} = 250 \times 1.396 = 349$$

where $SE_{c}$ is the cluster-adjusted standard error, $SE$ is the standard error ignoring clustering and $DE$ is the design effect. The naive 95% interval is 600 ± 1.96 × 250, or GBP 110 to GBP 1,090, which excludes zero. The adjusted interval is 600 ± 1.96 × 349, or from minus GBP 84 to GBP 1,284, which includes zero. Under a normal approximation the probability that the intervention is cost-effective falls from 0.99 to 0.96.

Decision uncertainty, and with it the value of further research, rises. A real analysis would use a multilevel model or a corrected two-stage bootstrap, since costs and QALYs may have different ICCs.

Stepped-wedge and cluster crossover variants

Both variants randomise clusters but vary the timing of exposure. In a stepped-wedge trial, the Cochrane Handbook explains, all clusters start with the comparator and switch to the intervention in a randomised order, so the analysis must allow for time trends. In a cluster crossover trial, Billot and colleagues note, the intervention must be possible to switch off, and the analysis assumes no carryover effect.

Boundaries with nearby designs and methods

Several nearby ideas share vocabulary with cluster randomisation. The distinction turns on what was allocated and at which level the data are correlated.

• Multicentre trials. Gomes and colleagues distinguish cluster trials from multicentre trials in which patients within a centre are randomised to different treatments; both have clustering, but only in the cluster trial is the whole centre allocated.
• Block randomization. It controls the balance of the allocation sequence and can be used to allocate clusters.
• Cluster analysis. It is a method for finding groups in data and has nothing to do with trial allocation.

Common misreadings in cluster trials and their economic evaluations

Most errors come from treating the patient as the unit of information when the cluster was the unit of allocation. The result looks more certain than it is.

1. Counting patients, not clusters. Thousands of patients in a handful of practices give little replication at the level that was randomised.
2. Ignoring a small ICC. With large clusters, an ICC of 0.01 still inflates the variance.
3. Resampling patients in the bootstrap. A patient-level bootstrap understates uncertainty in costs and net benefit; resampling has to start with clusters.
4. Spreading cluster costs wrongly. Dividing a per-practice cost by trial cluster size and applying it to a different roll-out setting misstates the cost per patient.

Sources

• Billot L, Copas A, Leyrat C, Forbes A, Turner EL. How should a cluster randomized trial be analyzed? Journal of Epidemiology and Population Health. 2024;72(1):202196. https://doi.org/10.1016/j.jeph.2024.202196
• Cook AJ, Delong E, Murray DM, Vollmer WM, Heagerty PJ. Statistical lessons learned for designing cluster randomized pragmatic clinical trials from the NIH Health Care Systems Collaboratory Biostatistics and Design Core. Clinical Trials. 2016;13(5):504-512. https://doi.org/10.1177/1740774516646578
• Eldridge SM, Ashby D, Kerry S. Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method. International Journal of Epidemiology. 2006;35(5):1292-1300. https://doi.org/10.1093/ije/dyl129
• Flynn TN, Peters TJ. Use of the bootstrap in analysing cost data from cluster randomised trials: some simulation results. BMC Health Services Research. 2004;4:33. https://doi.org/10.1186/1472-6963-4-33
• Gomes M, Grieve R, Nixon R, Edmunds WJ. Statistical methods for cost-effectiveness analyses that use data from cluster randomized trials: a systematic review and checklist for critical appraisal. Medical Decision Making. 2012;32(1):209-220. https://doi.org/10.1177/0272989X11407341
• Gomes M, Ng ES, Grieve R, Nixon R, Carpenter J, Thompson SG. Developing appropriate methods for cost-effectiveness analysis of cluster randomized trials. Medical Decision Making. 2012;32(2):350-361. https://doi.org/10.1177/0272989X11418372
• Grieve R, Nixon R, Thompson SG. Bayesian hierarchical models for cost-effectiveness analyses that use data from cluster randomized trials. Medical Decision Making. 2010;30(2):163-175. https://doi.org/10.1177/0272989X09341752
• Higgins JPT, Eldridge S, Li T. Chapter 23: Including variants on randomized trials. In: Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-23
• Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ. 2003;327(7418):785-789. https://doi.org/10.1136/bmj.327.7418.785