Equivalence Trial

Equivalence Trial Design, Margins and Use in Cost-Minimisation

An equivalence trial asks whether a new treatment performs close enough to an existing one that any difference between them would not matter clinically. The question is settled by fixing a margin in advance and showing that the plausible range for the true difference sits inside it in both directions. In health economics the design matters because a credible finding of equivalence is the usual evidence offered before costs alone are compared, as in a cost-minimisation analysis. Biosimilars and generic medicines are typical cases. A trial that merely fails to find a significant difference shows none of this, and confusing the two is a common error.

Two one-sided tests and the confidence interval rule

A superiority analysis tests the null hypothesis of no difference. An equivalence analysis turns the question round. ICH E9 describes a composite null hypothesis that the treatment difference lies outside the equivalence margins, tested against the alternative that it lies within them.

$$H_0 \colon\; \Delta \le -\delta \;\text{ or }\; \Delta \ge \delta \qquad \text{versus} \qquad H_1 \colon\; -\delta < \Delta < \delta$$

where $\Delta = \mu_N-\mu_R$ is the true difference in mean outcome between the new treatment ($\mu_N$) and the reference treatment ($\mu_R$), and $\delta$ is the equivalence margin. ICH E9 notes that an equivalence trial needs both a lower and an upper margin, while a non-inferiority trial needs only the lower one.

The two one-sided tests (TOST) procedure rejects each half of the null hypothesis separately.

$$T_L = \frac{\hat{\Delta} + \delta}{SE(\hat{\Delta})} \qquad\qquad T_U = \frac{\delta - \hat{\Delta}}{SE(\hat{\Delta})}$$

where $\hat{\Delta}$ is the estimated difference and $SE(\hat{\Delta})$ its standard error. Equivalence is concluded only when both statistics exceed the one-sided critical value $z_{1-\alpha}$, where $\alpha$ is the one-sided type I error.

ICH E9 states that this is operationally the same as requiring the whole two-sided confidence interval to fall within the margins, and that the type I error is controlled because the two null hypotheses are disjoint. A TOST at level $\alpha$ on each side corresponds to a $(1-2\alpha) \times 100\%$ interval. ICH E9 prefers one-sided tests at half the conventional two-sided level in regulatory settings, which for clinical equivalence gives 2.5% on each side and a 95% interval. Pharmacokinetic bioequivalence studies of immediate-release solid oral products follow ICH M13A, which uses a 90% interval, equivalent to two one-sided tests at the 5% level.

Setting the equivalence margin

A wide enough margin makes almost any treatment look equivalent, so the margin carries much of the conclusion. ICH E9 requires it to be stated in the protocol as the largest difference judged clinically acceptable, to be smaller than differences seen in superiority trials of the active comparator, and to be justified clinically. It adds that this acceptable difference may be smaller than the clinically relevant difference used to power a superiority trial, so a minimal clinically important difference is not automatically a suitable margin.

ICH E10, on the choice of control group, adds a historical anchor. Against an active comparator, the margin cannot be greater than the smallest effect the active drug would reliably be expected to have against placebo in the planned setting, and it should be suitably conservative. ICH E10 also observes that most active control equivalence trials are really non-inferiority trials. Where only the lower bound matters, a non-inferiority trial is the right design.

Sample size for an equivalence comparison

Equivalence trials are often large, because the interval must be narrow enough to fit between two limits. For a continuous outcome with a true difference of zero, Flight and Julious give a direct formula.

$$n_A = \frac{(r+1)\,\left(Z_{1-\beta/2} + Z_{1-\alpha}\right)^2 \sigma^2}{r\,\delta^2}$$

where $n_A$ is the number in arm A, $r$ is the allocation ratio, $\sigma^2$ is the outcome variance, $\delta$ is the margin, $\beta$ is the type II error and $Z_p$ is the standard normal quantile at $p$. The power term uses $\beta/2$ because both margins must be cleared.

Their example, a 2.5 mm margin on a visual analogue scale with a standard deviation of 10 mm, 2.5% type I error and 90% statistical power, needs 416 patients per arm, rising to 530 if the true difference is 0.5 mm. ICH E9 warns that powering at a true difference of zero underestimates the size needed when the true difference is not zero.

Analysis populations, conduct and assay sensitivity

Poor conduct usually dilutes observed differences, which in a superiority trial hides a benefit but in an equivalence trial pushes towards the desired conclusion. ICH E9 states that such trials are not conservative in nature, so many flaws in design or conduct bias results towards equivalence, and it names non-compliance, withdrawals, losses to follow-up and missing data.

ICH E9 notes that withdrawn patients tend to show a lack of response, so the full analysis set may be biased towards equivalence. Piaggio and colleagues add that, in non-inferiority trials, intention-to-treat analysis will often increase the risk of a false claim, while analyses excluding non-adherent patients could bias the result in either direction. For confirmatory trials in general, ICH E9 recommends planning both a full analysis set and a per-protocol analysis, and confidence rises when they reach much the same conclusion.

ICH E10 defines assay sensitivity as the ability of a trial to distinguish an effective treatment from a less effective or ineffective one. A successful equivalence result contains no direct evidence of it, since two treatments would also look alike if neither worked in the trial. ICH E10 says it can be deduced from historical evidence that similar trials reliably separated effective treatments, together with appropriate conduct, and lists threats such as poor compliance and patients who lack the disease under study. Piaggio and colleagues describe the related assumption of assay constancy, that current and earlier trials are alike in all relevant respects.

Generics, biosimilars and bioequivalence

ICH E9 places bioequivalence trials in the equivalence category. For immediate-release oral products, ICH M13A requires the 90% confidence interval for the geometric mean ratio of the pharmacokinetic parameters to lie within 80.00 to 125.00%. The EMA states that ICH M13A came into effect on 25 January 2025, superseding the applicable parts of its earlier guideline.

For a biosimilar, the EMA guideline on similar biological medicinal products (EMEA/CHMP/BMWP/42832/2005 Rev1) notes that bioequivalence studies are harder to interpret than for small molecules, and that comparable clinical efficacy usually has to be shown in randomised trials when no surrogate marker exists. It states that in general an equivalence design should be used, with margins prespecified and justified on statistical and clinical grounds. The EMA guideline page now says the guideline needs revision after a reflection paper on a tailored clinical approach, which may allow comparative efficacy studies to be waived where analytical and pharmacokinetic or pharmacodynamic data suffice.

Reporting with the CONSORT extension

The 2006 CONSORT extension for non-inferiority and equivalence trials, by Piaggio and colleagues, cites a review of 88 studies claiming equivalence in which 67% made the claim inappropriately from non-significant superiority tests and only 23% had a preset margin. The checklist asks whether the sample size used an equivalence criterion and for the margin with its rationale. The 2012 update aligns the extension with the CONSORT 2010 statement.

Worked example with an illustrative biosimilar comparison

All numbers here are illustrative. A biosimilar costs GBP 6,000 a year against GBP 9,000 for the reference biologic. The outcome is the change in a disease activity score with a standard deviation of about 10 points, and the prespecified margin is 4 points either way.

1. Plan the size. With equal allocation, 2.5% one-sided type I error and 90% power, $Z_{0.95} = 1.645$ and $Z_{0.975} = 1.960$. Their sum is 3.605, which squared is 12.996, so the formula gives 2 × 12.996 × 100 / 16 = 162.45, or 163 per arm, and the trial recruits 165 to allow a little headroom.

2. Analyse the result. The biosimilar arm improves by 0.8 points less, so the estimated difference is minus 0.8, with a pooled standard deviation of 10.2. The standard error is $SE = 10.2 \times \sqrt{2/165} = 1.123$ and the 95% half-width is 1.96 × 1.123 = 2.201, giving an interval from minus 3.00 to 1.40.

3. Apply both tests. The lower statistic is (4 − 0.8) / 1.123 = 2.85 and the upper is (4 + 0.8) / 1.123 = 4.27. Both exceed 1.96, matching the interval lying wholly inside minus 4 to 4, so equivalence is shown.

4. Contrast a small trial. With 30 per arm and the same estimate, $SE = 10.2 \times \sqrt{2/30} = 2.634$ and the half-width is 1.96 × 2.634 = 5.162, giving minus 5.96 to 4.36. The test of no difference gives z = 0.8 / 2.634 = 0.30, a p-value of about 0.76. There is no significant difference, yet the interval crosses both margins, so equivalence is not shown.

5. Check the economics. Suppose each score point is worth 0.01 QALY a year.

$$NMB = \lambda \, \Delta E - \Delta C$$

where $NMB$ is net monetary benefit per patient-year, $\lambda$ is the value of a QALY, $\Delta E$ is the QALY difference (biosimilar minus reference) and $\Delta C$ is the cost difference, here minus GBP 3,000. At GBP 25,000 per QALY and the lower limit of minus 3.00 points, the QALY loss is worth 0.030 × 25000 = 750, so net monetary benefit is GBP 2,250. At the margin itself and GBP 35,000 per QALY, 0.04 × 35000 = 1400, leaving GBP 1,600.

1. Interpret. The saving outweighs any loss the trial leaves plausible, so a costs-only comparison would not mislead. With a price gap of GBP 500, the lower limit would give a loss of GBP 250 and a full cost-utility comparison would be needed despite the equivalence finding.

Why equivalence evidence supports cost-minimisation analysis

The NICE manual (PMG36) describes cost-comparison analysis as suited to technologies likely to give similar or greater health benefits at similar or lower cost than comparators recommended for the same population, with health effects captured in the clinical evidence and left out of the cost model. PMG36 adds that substantial differences in costs linked to outcomes, such as adverse events, suggest overall benefits may not be similar, and it values QALY gains at GBP 25,000 and GBP 35,000 when presenting net health benefit.

Briggs and O'Brien argued in 2001 that cost-minimisation analysis is appropriate only in rare circumstances and proposed estimating the joint distribution of cost and effect differences, presented through cost-effectiveness acceptability curves. Dakin and Wordsworth found that cost-minimisation analysis biases measures of uncertainty, including the value of information, although the bias is negligible in non-inferiority studies of treatments that differ enormously in cost, and cost and effect data are generally still needed to show this.

Limitations and common misreadings

The commonest mistake is reading absence of evidence as evidence of equivalence. ICH E9 calls it inappropriate to conclude equivalence from a non-significant test of no difference.

1. A margin chosen after seeing the data, or without clinical and historical justification, removes the protection the design gives.
2. A 95% and a 90% interval inside the margins are different claims, with 2.5% and 5% one-sided error rates.
3. Equivalence on the primary outcome does not establish equivalence on safety or adherence, which may carry different costs.
4. Equivalence within a margin still leaves uncertainty for the decision model to carry.

Sources

• Briggs AH, O'Brien BJ. The death of cost-minimization analysis? Health Economics. 2001;10(2):179-184. https://doi.org/10.1002/hec.584
• Dakin H, Wordsworth S. Cost-minimisation analysis versus cost-effectiveness analysis, revisited. Health Economics. 2013;22(1):22-34. https://doi.org/10.1002/hec.1812
• European Medicines Agency. Guideline on similar biological medicinal products containing biotechnology-derived proteins as active substance: non-clinical and clinical issues. EMEA/CHMP/BMWP/42832/2005 Rev1. In effect 1 July 2015; guideline page consulted 18 October 2026. https://www.ema.europa.eu/en/similar-biological-medicinal-products-containing-biotechnology-derived-proteins-active-substance-non-clinical-clinical-issues-scientific-guideline
• European Medicines Agency. ICH M13A Guideline on bioequivalence for immediate-release solid oral dosage forms. EMA/CHMP/ICH/953493/2022, Step 5, in effect 25 January 2025. https://www.ema.europa.eu/en/ich-guideline-m13a-bioequivalence-immediate-release-solid-oral-dosage-forms-scientific-guideline
• Flight L, Julious SA. Practical guide to sample size calculations: non-inferiority and equivalence trials. Pharmaceutical Statistics. 2016;15(1):80-89. https://doi.org/10.1002/pst.1716
• ICH. Topic E 9 Statistical Principles for Clinical Trials. CPMP/ICH/363/96. European Medicines Agency; September 1998. https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-9-statistical-principles-clinical-trials-step-5_en.pdf
• ICH. Topic E 10 Choice of Control Group in Clinical Trials. CPMP/ICH/364/96. European Medicines Agency; January 2001. https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-10-choice-control-group-clinical-trials-step-5_en.pdf
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 4.2.13, 4.2.16 and 4.2.20. https://www.nice.org.uk/process/pmg36
• Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJW. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295(10):1152-1160. https://doi.org/10.1001/jama.295.10.1152
• Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594-2604. https://doi.org/10.1001/jama.2012.87802