Non-Inferiority Trial

Non-Inferiority Trial Margins, Analysis and Economic Reading

In a non-inferiority trial, patients are randomised between a new treatment and an established effective one, and the analysis asks whether any loss of effect with the new treatment can be ruled out beyond a limit agreed before the data are seen. Piaggio and colleagues note that the design often rests on some other advantage, such as lower cost, fewer harms or easier administration. For health economists a non-inferiority claim is the usual gateway to a costs-only comparison, and a treatment that is slightly less effective but cheaper sits in the south-west quadrant of the cost-effectiveness plane. The clinical margin and the economic break-even are different quantities, and the gap between them often decides whether a non-inferior treatment is good value.

The one-sided hypothesis and the confidence interval rule

A superiority trial tests the null hypothesis of no difference. A non-inferiority trial moves the null hypothesis to the margin, so rejecting it means the new treatment is worse by less than the margin, or better.

$$H_0 \colon \Delta \le -\delta \qquad \text{versus} \qquad H_1 \colon \Delta > -\delta$$

where $\Delta = \mu_N - \mu_S$ is the true difference in mean outcome between the new treatment ($\mu_N$) and the standard active treatment ($\mu_S$), with higher values better, and $\delta$ is the positive non-inferiority margin.

ICH E9 states that a one-sided interval should be used for non-inferiority trials and prefers one-sided type I error rates at half the conventional two-sided rate, for consistency with two-sided confidence intervals. Flight and Julious give the convention as 2.5% one-sided, so non-inferiority is concluded when the lower limit of the two-sided 95% confidence interval lies above minus $\delta$. For an outcome where higher is worse, such as adverse events, the signs reverse.

Where the interval can fall

The CONSORT extension by Piaggio and colleagues sets out eight positions an interval can take relative to zero and the margin. They group into six readings, shown here for an outcome where higher is better.

In the third row, which Piaggio and colleagues call rare and linked to very large trials or too wide a margin, the treatment passes while a loss of effect is statistically established. They accept testing superiority once non-inferiority is shown. The reverse move usually rests on a margin that was not prespecified, and the Australian Pharmaceutical Benefits Advisory Committee (PBAC) guidelines call such post hoc margins difficult and not preferred.

What the margin protects, absolute and relative efficacy

A margin can answer two questions. The draft EMA guideline on non-inferiority and equivalence comparisons (EMA/301654/2025) calls the typical regulatory aim absolute efficacy, showing that the test treatment is superior to no treatment or placebo without necessarily being as effective as the active comparator. Relative efficacy means showing it is not worse than the comparator by more than a clinically acceptable amount.

ICH E10 caps any margin at the smallest effect the active drug would reliably be expected to have against placebo. The FDA guidance (2016) calls the entire control effect assumed present in the new trial M1, and the largest clinically acceptable loss M2, a clinical judgement that can never exceed M1. A margin near M1 protects only absolute efficacy. The derivation of M1, constancy and assay sensitivity are covered under active control trial.

Payers need relative efficacy. The PBAC guidelines (section 2.4.5) ask for a margin that assures the medicine is not inferior to the main comparator by an important difference, and note that an unimportant difference is commonly taken as less than a minimal clinically important difference. A margin designed only to retain superiority over placebo may still allow an important loss, in which case the guidelines ask for it to be redefined and non-inferiority retested.

The scale of the margin

The scale of a margin changes what a trial can conclude. The FDA guidance notes that relative measures such as the hazard ratio may be more stable than the absolute risk difference when background event rates differ, and the PBAC guidelines (section 2.5.1) ask for any conversion between absolute and relative margins to be explained. For illustration, a 10-point margin on failure set when the standard fails in 20% of patients equals a relative risk of 1.5. If the standard fails in only 10% in the new trial, the 10-point margin accepts a doubling to 20%, the relative margin only 15%.

Sample size and why these trials are large

Ruling out a small loss needs a narrow interval, so Piaggio and colleagues note that non-inferiority trials are usually larger than superiority trials. Flight and Julious give the sample size for a continuous outcome.

$$n_A = \frac{(r + 1)\,(Z_{1-\alpha} + Z_{1-\beta})^2\,\sigma^2}{r\,(\Delta_a + \delta)^2}$$

where $n_A$ is the number in arm A, $r$ the allocation ratio, $\alpha$ the one-sided type I error, $\beta$ the type II error, $Z_p$ the standard normal quantile at $p$, $\sigma^2$ the outcome variance, $\Delta_a$ the true difference assumed at design and $\delta$ the margin. With equal allocation, 90% statistical power and 2.5% one-sided error, the squared sum of quantiles is 10.51. Flight and Julious show that assuming a small advantage for the new treatment reduces the size needed, and by the same formula a small assumed disadvantage increases it.

Analysis populations and trial conduct

ICH E9 states that non-inferiority trials are not conservative in nature, so many flaws in design or conduct will tend to bias results towards equivalence. Piaggio and colleagues add that intention-to-treat analysis will often increase the risk of falsely claiming non-inferiority, though not always. The PBAC guidelines ask for the least favourable tail of the 95% interval to be compared with the margin under both intention-to-treat and per-protocol approaches. The 2025 EMA draft says the default use of per-protocol sets should generally be avoided.

Worked example with a non-inferior treatment in the south-west quadrant

All numbers are illustrative. A new oral treatment N is compared with a clinic-given standard S, with treatment success as the outcome, expected in 80% of patients on S. The prespecified margin is 10 percentage points.

1. Plan the size. Under the normal approximation the per-patient variance is 0.8 × 0.2 = 0.16. With no true difference assumed, the formula gives 2 × 10.51 × 0.16 / 0.01 = 336.2, so 337 per arm are needed. The trial randomises and analyses 400 per arm.

2. Estimate the difference. Success is 308 of 400 with N and 320 of 400 with S.

$$d = \hat{p}_N - \hat{p}_S = 0.770 - 0.800 = -0.030, \qquad SE = \sqrt{\frac{0.770 \times 0.230}{400} + \frac{0.800 \times 0.200}{400}} = \sqrt{0.000443 + 0.000400} = 0.0290$$

where $d$ is the estimated difference in success, $\hat{p}_N$ and $\hat{p}_S$ are the observed proportions and $SE$ is the standard error of $d$. The 95% interval is minus 0.030 plus or minus 1.96 × 0.0290 = 0.0569, running between minus 8.69 and plus 2.69 points.

3. Read the interval. The lower limit lies above minus 10, so N is non-inferior. The interval includes zero, so neither superiority nor a loss is statistically shown.

4. Cost the difference. N saves GBP 600 per patient in treatment costs. Each extra failure costs GBP 2,500 in retreatment and loses 0.2 QALYs. With $f = -d$ extra failures per patient, $\Delta C = -600 + 2500f$ and $\Delta E = -0.2f$. At $f = 0.03$, $\Delta C = -600 + 75 = -525$ and $\Delta E = -0.006$, so N lies in the south-west quadrant.

5. Apply the decision rule. Net monetary benefit avoids misreading ratios in this quadrant.

$$NMB = \lambda\,\Delta E - \Delta C = 25000 \times (-0.006) - (-525) = -150 + 525 = 375$$

where $NMB$ is the net monetary benefit of N against S in GBP per patient and $\lambda$ is the value of a QALY, here GBP 25,000. At GBP 35,000 it is minus 210 plus 525, or GBP 315. When $\Delta E$ is negative, a positive $NMB$ means the saving per QALY forgone, here 525 / 0.006 or GBP 87,500, exceeds $\lambda$.

6. Find the economic break-even. Setting $NMB$ to zero gives $f^{*} = 600 / (0.2\lambda + 2500)$, which is 600 / 7,500 = 0.080 at GBP 25,000 and 600 / 9,500 = 0.063 at GBP 35,000. The saving pays for at most 8.0 or 6.3 extra failures per 100 patients, while the clinical margin accepts 10.

7. Carry the uncertainty. At the lower limit $f = 0.0869$, so $\Delta C = -382.75$, $\Delta E = -0.01738$ and $NMB$ at GBP 25,000 is minus 434.50 plus 382.75, or minus GBP 51.75. Treating $f$ as normal (mean 0.030, standard error 0.0290) with per-failure cost and QALY loss fixed, the probability of positive $NMB$ is $\Phi(1.72) = 0.96$ at GBP 25,000 and $\Phi(1.14) = 0.87$ at GBP 35,000.

8. Interpret. The trial meets its clinical margin, yet plausible losses run beyond what the saving can buy. A costs-only analysis reporting a GBP 600 saving would overstate the gain and hide a 4% to 13% chance that N is not cost-effective.

Using a non-inferiority result in health technology assessment

Non-inferiority evidence often decides the form of economic evaluation. The NICE manual (PMG36) reserves cost-comparison analysis for technologies likely to provide similar or greater health benefits at similar or lower cost than the relevant comparators (section 4.2.13), and says a non-inferiority or equivalence study with appropriate margins should ideally be presented (section 6.2.9). For technologies with less benefit at lower cost, PMG36 asks for net health benefits at GBP 25,000 and GBP 35,000 per QALY (section 4.10.8) and for the usual cost-effectiveness levels of GBP 25,000 to GBP 35,000 per QALY to be considered (section 6.3.9).

The PBAC guidelines (section 3) state that a cost-minimisation approach is appropriate where there is a therapeutic claim of non-inferiority or superiority, safety is equivalent or better, and costs are expected to be equal or lower. A medicine that is therapeutically inferior but likely to cost less needs a full cost-effectiveness analysis.

Briggs and O'Brien, and later Dakin and Wordsworth, argued that a costs-only analysis is rarely justified, a debate set out under cost-minimisation analysis.

Boundaries with superiority and equivalence designs

A superiority trial asks whether the new treatment is better and an equivalence trial whether the difference lies inside a margin in both directions, while ICH E9 notes that a non-inferiority trial needs only the lower margin. Flight and Julious add that a result far above the margin is no failure here, though it would make equivalence harder to show.

Limitations and common misreadings

The design reduces a question about the size of a loss to a pass or fail against one number. Several misreadings follow from that.

1. Non-inferior read as equal. The example interval allows N to be 8.69 points worse, which a decision model should carry rather than assume a zero difference.
2. Clinical margin read as economic margin. The margin was 10 points against an economic break-even of 6.3 to 8.0.
3. No significant difference read as non-inferiority. The PBAC guidelines state that a lack of a statistically significant difference does not adequately establish non-inferiority.

Sources

• Briggs AH, O'Brien BJ. The death of cost-minimization analysis? Health Economics. 2001;10(2):179-184. https://doi.org/10.1002/hec.584
• Committee for Medicinal Products for Human Use. Guideline on non-inferiority and equivalence comparisons in clinical trials. Draft for consultation (EMA/301654/2025). European Medicines Agency; 2025. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-guideline-non-inferiority-equivalence-comparisons-clinical-trials_en.pdf
• Dakin H, Wordsworth S. Cost-minimisation analysis versus cost-effectiveness analysis, revisited. Health Economics. 2013;22(1):22-34. https://doi.org/10.1002/hec.1812
• Flight L, Julious SA. Practical guide to sample size calculations: non-inferiority and equivalence trials. Pharmaceutical Statistics. 2016;15(1):80-89. https://doi.org/10.1002/pst.1716
• International Conference on Harmonisation. ICH E9: Statistical principles for clinical trials. Step 4 guideline; 1998. https://database.ich.org/sites/default/files/E9_Guideline.pdf
• International Conference on Harmonisation. ICH E10: Choice of control group and related issues in clinical trials. Step 4 guideline, 20 July 2000. https://database.ich.org/sites/default/files/E10_Guideline.pdf
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 4.2.13, 4.10.8, 6.2.9 and 6.3.9. https://www.nice.org.uk/process/pmg36
• Pharmaceutical Benefits Advisory Committee. Guidelines for preparing a submission to the Pharmaceutical Benefits Advisory Committee, version 5.0. September 2016. Sections 2.4.5, 2.5.1 and 3. https://pbac.pbs.gov.au/content/information/files/pbac-guidelines-version-5.pdf
• Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJW. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295(10):1152-1160. https://doi.org/10.1001/jama.295.10.1152
• US Food and Drug Administration. Non-inferiority clinical trials to establish effectiveness: guidance for industry. November 2016. https://www.fda.gov/media/78504/download