Comparator

Comparator Selection in Economic Evaluation and HTA

Economic evaluation compares the costs and outcomes of alternative courses of action rather than examining an intervention in isolation. The comparator establishes the reference point for that comparison, so changing the comparator can change the incremental costs, incremental outcomes, incremental cost-effectiveness ratio (ICER) and net benefit of the intervention.

A decision-relevant comparator has to be distinguished from a clinical-trial control, multiple or mixed alternatives need careful handling, and the choice should be reported transparently. In England the NICE technology appraisal manual (PMG36) sets rules on comparators, and the CHEERS 2022 reporting standard asks authors to report how they were chosen. The same intervention can produce different incremental results against different comparators, which is why a fully incremental analysis is needed when there are several.

How to select a relevant comparator

A comparator should represent a feasible alternative for the population, setting and decision under consideration. It should reflect what would realistically happen if the evaluated intervention were not adopted, rather than simply using whichever control happened to be included in a clinical trial.

A structured selection process helps prevent relevant alternatives from being overlooked. The process should be completed before the economic model and evidence strategy are finalised.

1. Define the population, decision-maker and healthcare setting.
2. Describe the current clinical or service pathway.
3. Identify all feasible alternatives available to the population.
4. Determine what the intervention would replace, supplement or displace.
5. Examine whether current practice varies across subgroups, providers or jurisdictions.
6. Identify emerging alternatives likely to affect the decision.
7. Compare the decision-relevant alternatives with the controls used in the clinical evidence.
8. Include every relevant mutually exclusive option in the economic analysis.
9. Define any composite or treatment-mix comparator transparently.
10. Test plausible alternative comparators when the base-case choice is uncertain.

Comparator selection should reflect actual or expected practice, not only what a guideline states should happen. Administrative data, registries, prescribing records, clinical audits, treatment pathways and expert elicitation can help establish the relevant alternatives.

Comparators in NICE technology appraisal

In England, the NICE manual (PMG36, "NICE technology appraisal and highly specialised technologies guidance: the manual") ties the comparator to the scope and to established NHS practice. The scope identifies all potentially relevant comparators that are established practice in the NHS, and at the scoping stage the identification of comparators is meant to be inclusive (section 2.2.12). The reference case then uses the comparators as listed in the scope (table 4.1 and section 4.2.5).

The manual sets several further rules about which technologies can serve as comparators.

• Branded medicines, non-proprietary (generic) medicines and biosimilars can all be comparators (section 2.2.13).
• A technology without regulatory approval for the population in the scope can be a comparator if it is established clinical practice for that population in the NHS (sections 2.2.13 and 6.2.4).
• When the technology and the comparator are both part of a sequence in the care pathway, the evaluation may compare alternative sequences (section 2.2.14).
• Technologies recommended with managed access are not considered established NHS practice and are not suitable comparators (section 2.2.15).
• In a cost-comparison appraisal of a medicine, the relevant comparators are those recommended in published NICE guidance for the same population (section 4.2.13).

When selecting the most appropriate comparators, the appraisal committee considers established NHS practice in England, the natural history of the condition without suitable treatment, existing NICE guidance, the cost effectiveness of the comparator and its licensing or regulatory status (section 6.2.2). The committee is normally guided by established NHS practice. If an established practice appears not to be a good use of NHS resources relative to another available treatment, the committee decides whether to keep it as a comparator after reviewing an incremental economic analysis, guided by whether it is recommended in other NICE guidance or is so embedded in clinical practice that it will continue unless a new technology replaces it (section 6.2.3).

Results are presented as a fully incremental analysis, with dominated and extendedly dominated technologies removed (section 4.10.8). Pairwise comparisons may also be presented when relevant and justified, for example when the technology is expected to displace specific comparators.

Common types of comparator

Different decision problems require different types of comparator. Labels such as “usual care”, “standard management” and “no treatment” are not sufficiently precise unless the services, treatments and resource use included within them are described.

The following table summarises common comparator types and the clarification each one requires.

An active comparator may be appropriate when an established treatment already addresses the same clinical need. A no-intervention comparator may be appropriate only when withholding the active programme is a genuine option and any continuing background care is still represented. A “do nothing” option is rarely free of cost, because untreated disease continues to generate healthcare use and health losses that belong in the comparator arm. Including it can also show whether the treatment currently used is itself cost-effective, which matters when established practice might be a poor use of resources.

Why a clinical-trial control may not be the right comparator

The control arm of a clinical trial answers the trial’s research question, but it does not automatically represent the alternative relevant to an economic decision. A placebo-controlled trial may establish efficacy even though the practical decision concerns whether a new treatment should replace an existing active treatment.

The reverse problem can also occur. A head-to-head trial may use an active control that is unavailable, rarely used or inappropriate in the jurisdiction where the economic evaluation will inform a decision.

Aligning the clinical evidence with the economic comparator may require additional analytical work. Each adjustment introduces assumptions that should be documented and tested.

• Direct head-to-head evidence can estimate relative effects against the required comparator.
• Network meta-analysis can compare several treatments through a connected evidence network.
• Indirect treatment comparison can estimate relative effects when direct evidence is unavailable.
• Population adjustment can address important differences between trial populations.
• Observational evidence can describe local treatment patterns, outcomes or baseline risks.
• Model calibration can align simulated outcomes with relevant external evidence.
• Clinical-pathway modelling can represent an alternative not observed directly in the trial.

The economic evaluation should clearly distinguish the clinical-evidence control from the decision comparator. Any method used to connect them should be reported with its assumptions and uncertainty.

When the intervention and the decision comparator have never been compared in the same trial, an adjusted indirect comparison estimates their relative effect through a common comparator, such as placebo, so that the randomised contrast within each trial is preserved. Pooling the single treatment arms of different trials abandons randomisation and is prone to the same biases as an observational comparison. The NICE manual (PMG36) asks for a network meta-analysis when the relevant technologies have not been evaluated within a single randomised controlled trial, requires trial randomisation to be preserved, and treats comparisons of single arms from different trials as observational evidence with increased uncertainty (sections 3.4.11 and 3.4.20). Its committee also takes account of the additional uncertainty when relative effects come from indirect sources only.

How current practice can vary

Current practice is not always a single, stable treatment. It may differ between patient groups, providers, geographical areas, healthcare sectors or stages of a treatment pathway.

The evaluation should investigate material variation instead of assuming that one national or historical average represents every decision setting. The following factors commonly affect the definition of current practice.

• Current practice may differ according to disease severity or previous treatment.
• Current practice may differ between hospitals, clinics or geographical areas.
• Current practice may be restricted by eligibility rules, capacity or access.
• Current practice may involve several treatments used in different proportions.
• Current practice may change following new guidance, price changes or market entry.
• Current practice may include background care even when no active treatment is provided.

The evidence used to define current practice should be recent enough for the decision. Historical market shares should not be treated as current practice without checking whether the pathway has changed.

How to compare several alternatives

Some decisions involve more than one relevant comparator. When alternatives are mutually exclusive, a fully incremental analysis should compare all feasible options together rather than comparing every intervention separately with one fixed baseline.

The alternatives are ordered by expected outcome before dominance and incremental results are assessed. This process identifies the efficient options that form the cost-effectiveness frontier.

1. Order the alternatives from the least effective to the most effective.
2. Remove any strictly dominated alternative that costs more and produces fewer outcomes than another option.
3. Calculate incremental costs and outcomes between adjacent alternatives.
4. Identify alternatives subject to extended dominance.
5. Remove the extendedly dominated alternatives.
6. Recalculate the incremental sequence after each exclusion.
7. Construct the cost-effectiveness frontier from the remaining alternatives.
8. Compare the remaining options using the applicable threshold or net-benefit rule.

A collection of pairwise comparisons against a common baseline can select an inefficient option because it does not show the additional cost and outcome of moving between adjacent alternatives. The fully incremental sequence should therefore be reported whenever all included options represent competing choices.

An ICER calculated against a dominated or extendedly dominated option looks more favourable than the ICER against the next less effective option that remains on the frontier. The inefficient option lies above the frontier on the cost-effectiveness plane, so the line from it to the new option is flatter. The ICER that informs adoption is the one calculated against the adjacent non-dominated option.

When a treatment-mix comparator is appropriate

A treatment mix can represent current practice when several alternatives are used across the target population and the decision concerns replacing that combined pattern of care. The component treatments and their weights must remain visible so that variation is not hidden inside a single average.

The expected cost and outcome of a mixed comparator are calculated as weighted averages. The weights should describe the target population, setting and relevant practice period and must sum to one.

Mixed comparator cost = Σ(wⱼ × Cⱼ)

Mixed comparator outcome = Σ(wⱼ × Eⱼ)

In these equations, wⱼ is the proportion receiving comparator j, Cⱼ is its expected cost, and Eⱼ is its expected outcome. The condition Σwⱼ = 1 confirms that the treatment shares represent the entire comparator population.

A weighted treatment mix is inappropriate when its components are mutually exclusive policy choices that should be compared separately. Sensitivity analysis should be used when the component weights are uncertain or changing.

Replacement, add-on treatment and displacement

Adoption does not always produce a simple one-for-one replacement of an existing treatment. A new intervention may supplement background care, displace several treatments, alter later treatment sequences or expand treatment to people who previously received no active intervention.

The comparator should reflect the change expected in practice. The initial adoption decision should also be distinguished from later switching, rescue treatment and subsequent-line care represented within the model.

• A replacement intervention should be compared with the treatment or treatments it is expected to displace.
• An add-on intervention should usually be compared as the intervention plus background care versus background care alone.
• A pathway-changing intervention should include relevant changes to subsequent tests, treatments and services.
• An intervention that expands eligibility should include the outcomes and costs of the newly treated population.
• An intervention that changes the care setting should represent displaced resources in both the original and new settings.

Comparing an add-on treatment with no care can incorrectly attribute the effects of background management to the new intervention. The model should therefore maintain consistent background care across the relevant comparison groups.

How the comparator changes the result

The same intervention can produce different incremental results when evaluated against different alternatives. The intervention’s own costs and outcomes stay the same, and the difference in the result comes entirely from the reference option.

The following illustrative example compares one intervention with current practice and an active alternative. The figures are invented for teaching.

The calculation against current practice is as follows.

Incremental cost = £18,000 − £12,000 = £6,000

Incremental QALYs = 4.50 − 4.20 = 0.30

ICER = £6,000 ÷ 0.30 = £20,000 per QALY

The calculation against the active alternative is as follows.

Incremental cost = £18,000 − £14,000 = £4,000

Incremental QALYs = 4.50 − 4.40 = 0.10

ICER = £4,000 ÷ 0.10 = £40,000 per QALY

The intervention appears less cost-effective against the active alternative because that comparator already produces more QALYs than current practice. These isolated calculations still do not replace a fully incremental analysis when all three options are mutually exclusive.

Fully incremental analysis of the three illustrative options

When the three options are competing choices, they are ordered by QALYs and each is compared with the previous option that remains in the sequence. The table uses the same illustrative figures.

No option is dominated, and the ICERs rise along the sequence from £10,000 to £40,000 per QALY, so no option is extendedly dominated and all three lie on the frontier. At an illustrative threshold of £30,000 per QALY, the active alternative is the preferred option, because moving from it to the new intervention costs £40,000 per additional QALY. The pairwise ICER of £20,000 per QALY against current practice alone would have suggested adopting the new intervention.

Net monetary benefit at the same illustrative threshold gives the same ranking. Current practice yields £114,000 (£126,000 minus £12,000), the active alternative £118,000 (£132,000 minus £14,000) and the new intervention £117,000 (£135,000 minus £18,000), so the active alternative has the highest net monetary benefit.

Keeping the comparison consistent

The comparator must be represented consistently throughout the economic evaluation. Misalignment between the stated comparator, clinical evidence, model structure, resource use and costs can make the incremental result difficult to interpret.

Consistency should be checked across every part of the analysis. Differences should be adjusted or clearly explained rather than left implicit.

• The decision problem should name the same comparator represented in the model.
• The clinical evidence should support the relative effects used for the required comparison.
• The treatment pathway should represent relevant dosing, administration, monitoring and duration.
• Resource use and costs should reflect both the intervention and comparator pathways.
• Outcomes should be measured over the same population and time horizon.
• Perspective should be applied consistently to the costs and outcomes of every option.
• Subgroup analyses should use comparators relevant to each subgroup.
• Subsequent treatment and switching should be modelled consistently across alternatives.

A comparator can also change during an evaluation because of evolving practice, new technologies or price changes. The practice date and evidence cutoff used to define the comparator set should therefore be reported.

Testing uncertainty about the comparator

Comparator uncertainty is different from ordinary parameter uncertainty because it concerns which alternative represents the decision. Scenario analysis is especially important when several plausible comparators could produce different conclusions.

The evaluation should report comparator-specific results rather than averaging away meaningful differences. Alternative-comparator scenarios are particularly useful in the following circumstances.

• Practice varies materially between settings or patient groups.
• Several active alternatives are feasible.
• Treatment-mix weights are uncertain.
• Market shares or clinical pathways are changing.
• The principal comparator differs by subgroup.
• A near-market technology may alter practice.
• The clinical evidence uses a different control.
• The preferred option changes when the comparator changes.

Where comparator selection remains contested, the evaluation should explain the competing interpretations and show their consequences. Selecting a comparator because it produces a preferred result is not methodologically acceptable.

Implementing comparator analysis in Excel

An Excel model can use validated option selectors to retrieve the selected comparator’s costs and outcomes. The workbook should preserve visible option-level inputs and prevent self-comparison, missing selections and invalid treatment-mix weights.

The following formulas illustrate common implementation checks. Named ranges can be replaced with structured table references when appropriate.

A model containing several mutually exclusive options should also construct a fully incremental table. It should not rely solely on pairwise comparisons with a fixed baseline.

What should be reported about the comparator

Transparent reporting allows readers to judge whether the comparator represents the real decision problem. It also helps future analysts understand why results may differ across studies, jurisdictions or time periods.

The CHEERS 2022 reporting standard (item 7, Comparators) asks authors to describe the interventions or strategies being compared and why they were chosen. A complete report should provide the following information, which extends that item to the detail needed to judge the comparator.

• The evaluation should identify every comparator that was considered.
• The evaluation should name the base-case comparator and explain why it was selected.
• The evaluation should define current practice or standard care operationally.
• The evaluation should explain which treatments, services or resources the intervention is expected to displace.
• The evaluation should distinguish the clinical-evidence control from the economic comparator.
• The evaluation should describe adjustments used to align the evidence with the required comparison.
• The evaluation should report the components, weights and sources of any treatment-mix comparator.
• The evaluation should explain material subgroup or jurisdictional differences.
• The evaluation should state the practice date and evidence cutoff.
• The evaluation should identify relevant near-market alternatives.
• The evaluation should report comparator-specific incremental results.
• The evaluation should present uncertainty and alternative-comparator scenarios.

Common comparator errors

Comparator errors can make a technically sophisticated model answer the wrong decision question. They often arise when an available evidence source is allowed to determine the comparison without first defining the real alternatives.

The most important errors to avoid are listed below.

• Treating the clinical-trial control as automatically appropriate for the economic evaluation.
• Selecting an obsolete, unavailable or artificially weak comparator.
• Leaving usual care or standard care undefined.
• Describing no intervention while omitting continuing background care.
• Omitting a relevant active alternative.
• Comparing all options only with a fixed baseline when fully incremental analysis is required.
• Averaging alternatives that should be evaluated as separate policy choices.
• Using outdated treatment-mix weights.
• Ignoring material subgroup or jurisdictional variation.
• Confusing initial displacement with later treatment switching.
• Comparing an add-on intervention with no care.
• Misaligning the comparator used for effectiveness, costs and outcomes.
• Ignoring a near-market alternative that could change the decision.
• Selecting the comparator because it produces a preferred conclusion.

Sources

• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford: Oxford University Press; 2006.
• Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. Journal of Clinical Epidemiology. 1997;50(6):683-691.
• Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford: Oxford University Press; 2015.
• Husereau D, Drummond M, Augustovski F, et al. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. BMJ. 2022;376:e067975.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; published 31 January 2022, last updated 31 March 2026. Sections 2.2.12 to 2.2.15, 3.4.11, 3.4.20, 4.2.5, 4.2.13, 4.10.8 and 6.2.2 to 6.2.4, and table 4.1.