Indirect Treatment Comparison

Indirect Treatment Comparison: Concept Architecture

Orientation and learning roadmap

An indirect treatment comparison connects separate bodies of comparative evidence when the treatments of interest have not been evaluated against each other in the same randomized trial. This page moves from the evidence network and core calculation to the assumptions that make the comparison credible, the choice between anchored and unanchored methods, and the way results should be assessed and used in health-economic models.

Why an indirect comparison may be needed

Health technology assessments often need a relative treatment effect for every relevant comparator, even when the available trials do not provide all required head-to-head comparisons. An indirect treatment comparison can fill a specific evidence gap by using a shared comparator, but it does not recreate randomization between the treatments being compared. Its validity therefore depends on whether the linked studies are sufficiently comparable for the chosen contrast.

• A direct comparison estimates the relative effect of two treatments within studies that randomized participants between those treatments.
• An indirect comparison estimates a missing treatment contrast by combining other comparisons that connect the treatments through a network.
• A mixed treatment comparison combines direct and indirect evidence for the same contrast within a connected evidence network.
• A network meta-analysis estimates multiple treatment contrasts jointly and may include direct, indirect, and mixed evidence; an indirect treatment comparison may be a simpler comparison within that broader family of methods.

How the evidence network creates the comparison

The simplest anchored comparison has three treatments: treatment A, treatment C, and a common comparator B. Trials provide direct evidence for A versus B and C versus B, while the relative effect of A versus C is obtained indirectly. The common comparator anchors the two trial sets to a shared randomized contrast.

A ── direct evidence ── B ── direct evidence ── C
└──────────── indirect A-versus-C estimate ────────────┘

The treatments, doses, outcome definitions, follow-up times, and analysis populations must be aligned closely enough for the path through B to answer the intended decision question. A connected diagram is necessary for an anchored estimate, but connectivity alone does not establish that the comparison is valid.

The adjusted indirect-comparison calculation

An adjusted indirect comparison preserves the randomized treatment contrasts within each set of trials. The calculation must use a compatible relative-effect scale, such as the log risk ratio, log odds ratio, log hazard ratio, or an additive scale such as a mean difference. Raw outcomes from separate treatment arms should not be compared as though the participants had been randomized across studies.

Let (d_{AB}) denote the relative effect of A versus B and (d_{CB}) the relative effect of C versus B on the same additive analysis scale. The indirect estimate of A versus C is:

[ d_{AC}=d_{AB}-d_{CB} ]

If the two direct estimates are statistically independent, the variance of the indirect estimate is:

[ \operatorname{Var}(d_{AC})= \operatorname{Var}(d_{AB})+ \operatorname{Var}(d_{CB}) ]

For ratio measures, the subtraction is performed on the logarithmic scale and then exponentiated. The orientation of every contrast must remain consistent; reversing one treatment comparison without changing its sign or taking the reciprocal produces an incorrect result.

Worked example using risk ratios

Suppose separate randomized-trial meta-analyses estimate a risk ratio of 0.70 for A versus placebo and 0.80 for C versus placebo for the same adverse outcome at the same follow-up time. Because both estimates use placebo as the common comparator and share the same orientation, the indirect risk ratio for A versus C is the ratio of those two effects. This example illustrates the arithmetic only; the clinical credibility of the result still depends on the study-comparability assumptions.

[ RR_{AC}=\frac{RR_{AB}}{RR_{CB}} =\frac{0.70}{0.80} =0.875 ]

The point estimate suggests that the risk under A is 12.5% lower than the risk under C on the relative scale used here. A confidence interval must be calculated from the standard error on the log scale before judging precision or statistical compatibility with no difference. The result should not be converted into an absolute risk difference without an appropriate baseline risk for the target population.

The transitivity assumption that supports the comparison

Transitivity is the central clinical and methodological assumption behind an indirect comparison. It requires that the included studies could, in principle, have randomized participants to any of the treatments in the network and that the distribution of important effect modifiers does not differ in a way that distorts the linked contrasts. The assumption cannot be proven by a statistical test alone and must be evaluated using clinical knowledge and study-level evidence.

• Eligibility criteria should identify sufficiently comparable target populations across the contributing comparisons.
• Prognostic factors may change baseline outcome risk, while effect modifiers change the relative treatment effect; the distinction matters because imbalance in effect modifiers directly threatens the indirect contrast.
• Treatment definitions should align in dose, schedule, co-interventions, treatment line, and implementation context.
• Outcome definitions, measurement methods, estimands, and follow-up times should be compatible.
• Study design and risk-of-bias patterns should not differ systematically across treatment comparisons in ways that could alter relative effects.
• The common comparator should represent a sufficiently similar intervention across studies rather than sharing only a convenient label.

Homogeneity, consistency, and uncertainty

Homogeneity concerns variation in treatment effects among studies making the same direct comparison, whereas consistency concerns agreement between direct and indirect evidence for the same contrast. Both matter because unexplained heterogeneity can weaken the components used to construct the indirect estimate, and inconsistency may indicate that transitivity does not hold somewhere in a closed network. A simple network without a closed loop cannot estimate disagreement between direct and indirect evidence for the missing contrast.

In a larger network, analysts should examine between-study heterogeneity, compare direct and indirect estimates where loops exist, and investigate clinically plausible sources of inconsistency. Statistical checks have limited power in sparse networks, so a non-significant inconsistency test is not evidence that the assumptions are satisfied. Uncertainty should reflect the synthesis model and should not be reduced to the confidence interval around one point estimate.

Anchored and unanchored comparisons answer different evidence problems

An anchored comparison retains a common randomized comparator and is generally preferred when a credible anchor exists. An unanchored comparison is attempted when the relevant studies have no common comparator, often using individual patient data from one trial and aggregate data from another. Removing the anchor requires much stronger assumptions because observed absolute outcomes must be made comparable across studies.

Matching-adjusted indirect comparison and simulated treatment comparison are population-adjustment approaches rather than automatic substitutes for a conventional anchored analysis. Their suitability depends on data access, overlap, the target population, and whether the necessary covariates have been measured consistently. Effective sample size, covariate balance, model specification, and extrapolation should be reported when these methods are used.

Choosing the effect measure and synthesis model

The analysis scale should fit the outcome and remain consistent across the evidence network. A hazard ratio may be used for time-to-event outcomes only when its interpretation and proportional-hazards assumptions are reasonable, while risk ratios, odds ratios, rate ratios, mean differences, and standardized mean differences each answer different questions. Apparent mathematical compatibility should not override differences in estimand or clinical meaning.

The analyst must also decide whether direct comparisons should be synthesized with fixed-effect or random-effects models before constructing or jointly estimating indirect effects. That decision should reflect the evidence structure, plausible heterogeneity, the number of studies, and the intended inference rather than a mechanical significance test. Multi-arm trials require methods that preserve the correlation between effect estimates sharing a trial arm.

A disciplined workflow for conducting an indirect comparison

A defensible analysis begins with the decision problem rather than with whichever trials are easiest to connect. Each step should be documented before the result is transferred into an economic model. Deviations from the planned approach should be explained because selective network construction can materially change the estimate.

1. Define the decision question. Specify the population, interventions, comparators, outcomes, time points, effect measures, and target estimand.
2. Identify the complete evidence base. Use a systematic and reproducible search rather than selecting only studies that create a convenient path.
3. Map the evidence network. Show every treatment node, direct comparison, study count, multi-arm trial, and disconnected component.
4. Harmonize treatment and outcome definitions. Confirm that doses, schedules, outcome measures, follow-up periods, and contrast orientations are compatible.
5. Assess transitivity before synthesis. Compare the distribution of plausible effect modifiers and important design features across treatment comparisons.
6. Select the analysis method. Justify a Bucher-type adjusted comparison, network meta-analysis, population-adjusted method, or decision not to pool.
7. Estimate effects and uncertainty. Preserve within-study randomization, use the correct covariance structure, and calculate intervals on the appropriate scale.
8. Evaluate heterogeneity and consistency. Use statistical diagnostics together with clinical assessment and sensitivity analyses.
9. Test influential choices. Examine alternative study sets, effect measures, heterogeneity assumptions, covariate specifications, and reasonable network definitions.
10. Report the result transparently. Present the network, data inputs, assumptions, diagnostics, limitations, and every transformation used by the economic model.

How indirect evidence enters a health-economic model

Economic models often use indirect estimates when the evaluated technology and a relevant comparator lack head-to-head evidence. The relative-effect estimate may determine event rates, response probabilities, progression, survival, adverse events, or treatment discontinuation and can therefore influence incremental costs and health outcomes. The evidence-synthesis uncertainty must remain connected to the model rather than being replaced by a single deterministic input.

For probabilistic sensitivity analysis, analysts should sample from an appropriate joint distribution when treatment effects are correlated. Baseline risk and relative treatment effect should be separated carefully, then combined on a scale consistent with the clinical model. Scenario analyses should examine credible alternative evidence networks and methods when those choices could change the cost-effectiveness conclusion.

Implementing and checking the calculation

A spreadsheet can audit a simple three-treatment adjusted comparison, but larger networks generally require validated statistical software. The implementation should preserve full precision, make the contrast direction visible, and separate transformed estimates from presentation values. Reproducible code and an analysis dataset are preferable when the model includes correlated effects, random effects, multi-arm trials, or population adjustment.

For a simple log-ratio comparison, an auditable spreadsheet can use:

Log effect A vs B       = LN(effect_A_vs_B)
Log effect C vs B       = LN(effect_C_vs_B)
Indirect log effect     = log_effect_A_vs_B - log_effect_C_vs_B
Indirect standard error = SQRT(SE_AB^2 + SE_CB^2)
Lower confidence limit  = EXP(indirect_log_effect - 1.96*indirect_SE)
Point estimate          = EXP(indirect_log_effect)
Upper confidence limit  = EXP(indirect_log_effect + 1.96*indirect_SE)

The variance formula above assumes independent component estimates. Shared studies, multi-arm trials, or jointly estimated network effects require the relevant covariance terms and should not be forced into this simplified spreadsheet structure.

Validation checks that should precede interpretation

Validation must cover the evidence, the statistical implementation, and the transfer of results into the decision model. A numerically reproducible estimate can still be clinically invalid if its assumptions are not credible. The final assessment should distinguish a calculation that is correct from a comparison that is fit for the intended decision.

• The network should include all eligible comparators and should not omit inconvenient evidence without a documented reason.
• Every treatment contrast should use a consistent direction and effect-measure definition.
• Recalculation from reported inputs should reproduce the indirect point estimate and interval.
• Trial characteristics and candidate effect modifiers should be compared by treatment contrast, not only summarized across the whole evidence set.
• Multi-arm trial correlations and repeated use of shared evidence should be handled correctly.
• Direct and indirect evidence should be compared where the network permits that assessment.
• Sensitivity analyses should reveal whether reasonable study, model, or covariate choices alter the decision.
• Economic-model inputs should match the clinical-analysis population, outcome, time horizon, and uncertainty structure.

Common errors and safeguards

Indirect comparisons can appear precise because the final output is a familiar effect estimate, even when the underlying evidence path is weak. The main safeguards are a complete evidence network, explicit assumptions, clinically informed assessment of effect modifiers, transparent diagnostics, and sensitivity analysis. The method should not be used to manufacture comparative certainty that the available studies cannot support.

• Comparing raw event rates from unrelated single treatment arms breaks the protection created by within-trial randomization.
• Treating a shared label such as “standard care” as an identical comparator can conceal material differences in care pathways.
• Assuming that similar mean age or baseline risk proves transitivity overlooks other effect modifiers and distributional differences.
• Mixing outcome definitions, follow-up times, treatment doses, or estimands can make the mathematical contrast clinically incoherent.
• Selecting a network after viewing favorable results creates avoidable bias.
• Ignoring covariance from multi-arm trials overstates the amount of independent information.
• Interpreting treatment rankings without their uncertainty can exaggerate clinically trivial differences.
• Treating an unanchored population-adjusted result as equivalent to a randomized comparison understates its identifying assumptions.
• Passing only a point estimate to the economic model understates decision uncertainty.

Interpreting the result responsibly

An indirect estimate is conditional on the included evidence, network structure, effect scale, synthesis model, and cross-study comparability assumptions. It should be reported as an estimate supported by a specified evidence pathway, not as though A and C were randomized directly. Decision makers need both the numerical result and a clear account of how assumption failure could change its direction, magnitude, precision, and economic consequences.

The strongest conclusion may sometimes be that an indirect comparison is too uncertain or too poorly connected to support a reliable decision. That conclusion is analytically valuable because it identifies the evidence gap rather than hiding it behind a model. Where the comparison is used, its limitations should remain visible in the clinical evidence assessment and in the health-economic uncertainty analysis.