Matching-Adjusted Indirect Comparison

Matching-Adjusted Indirect Comparison of Treatments Across Trials

A matching-adjusted indirect comparison (MAIC) estimates how two treatments compare when they were tested in separate trials on different kinds of patients. Each patient in the trial with individual patient data gets a weight so that the weighted group resembles the other trial's published population, and outcomes are then compared. Health technology assessment meets the method most often when a company holds patient-level data for its own trial and only the journal article for a competitor's trial. The estimate applies to the competitor trial's population and adjusts only for the characteristics that trial reported. Within the family of indirect treatment comparison methods, MAIC is the reweighting approach, and simulated treatment comparison (STC) is its regression-based counterpart.

Why different trial populations bias an indirect comparison

A standard indirect comparison of treatment B with treatment C through a shared comparator A combines the effect of B against A from one trial with the effect of C against A from another. NICE Decision Support Unit Technical Support Document (TSD) 18 states that the key assumption behind these standard methods is that the trials do not differ in the distribution of effect-modifying variables. An effect modifier changes the relative effect of treatment on the chosen scale, as when prior therapy shrinks the gain a new drug offers over placebo. A purely prognostic variable changes the outcome whichever treatment is received, so it cancels out of a comparison anchored on the shared arm.

TSD 18 warns that networks with one or two trials per treatment, common in NICE submissions, are highly vulnerable to this bias. Phillippo and colleagues note that network meta-regression on aggregate data has low power and is open to ecological bias, and that with individual patient data on every trial it would be the gold-standard approach. MAIC was built for the commoner case in which only one side's patient-level data exist.

How the weights are estimated

Signorovitch and colleagues introduced MAIC in 2010. Their first application reweighted 1,025 patients from trials of adalimumab against placebo in psoriasis to match a trial of etanercept against placebo with 330 patients. TSD 18 describes the weight for each patient as the odds of being enrolled in the comparator (AC) trial rather than the company (AB) trial, modelled with a logistic regression. That makes MAIC a close relative of inverse propensity score weighting.

$$\log(w_{it}) = \alpha_0 + \alpha_1^{T} X_{it}$$

where $w_{it}$ is the weight for patient $i$ on treatment $t$ in the AB trial, $X_{it}$ is that patient's vector of baseline covariates, $\alpha_0$ is an intercept and $\alpha_1$ is a vector of coefficients.

The model cannot be fitted in the usual way, because the AC trial has no patient-level covariates. Signorovitch and colleagues used the method of moments instead, choosing $\alpha_1$ so that the weighted covariate means of the AB patients equal the means published for the AC trial. TSD 18 records that, with covariates centred on the AC means, this is equivalent to minimising the sum below.

$$\sum_{t \in \{A,B\}} \sum_{i=1}^{N_{t(AB)}} \exp\left(\alpha_1^{T} X_{it}\right)$$

where $N_{t(AB)}$ is the number of patients on treatment $t$ in the AB trial. Squared covariate terms can be added so that variances are matched as well as means. The intercept $\alpha_0$ cancels from every weighted average, so only the relative sizes of the weights matter.

The weighted mean outcome then predicts what the AC population would have experienced on each treatment.

$$\hat{Y}_{t(AC)} = \frac{\sum_{i=1}^{N_{t(AB)}} Y_{it(AB)}\, \hat{w}_{it}}{\sum_{i=1}^{N_{t(AB)}} \hat{w}_{it}}$$

where $Y_{it(AB)}$ is the observed outcome of patient $i$ on treatment $t$ in the AB trial, $\hat{w}_{it}$ is the estimated weight and $\hat{Y}_{t(AC)}$ is the predicted mean outcome on treatment $t$ in the AC population. For censored time-to-event outcomes, a weighted survival analysis gives a hazard ratio.

Anchored and unanchored comparisons

An anchored MAIC keeps the common comparator. It subtracts the reweighted effect of B against A from the published effect of C against A, on a link-function scale.

$$\hat{d}_{BC(AC)} = \left[ g(\bar{Y}_{C(AC)}) - g(\bar{Y}_{A(AC)}) \right] - \left[ g(\hat{Y}_{B(AC)}) - g(\hat{Y}_{A(AC)}) \right]$$

where $\hat{d}_{BC(AC)}$ is the effect of C relative to B in the AC population, $\bar{Y}_{A(AC)}$ and $\bar{Y}_{C(AC)}$ are the published outcomes in the AC trial, $\hat{Y}_{A(AC)}$ and $\hat{Y}_{B(AC)}$ are the reweighted AB predictions, and $g$ is a link function such as the logit. An unanchored MAIC drops both A terms and sets the reweighted outcome on B directly against the published outcome on C.

TSD 18 says an unanchored MAIC effectively assumes that absolute outcomes can be predicted from the covariates, an assumption largely considered impossible to meet. Its first recommendation allows the unanchored form only without a connected network of randomised studies or where single-arm studies are involved. Its fourth asks an anchored MAIC to adjust for all effect modifiers but no purely prognostic variables, since these only inflate the standard error, while an unanchored MAIC must adjust for both. In the TSD 18 review of ten published MAICs, 40% involved single-arm trials or no common comparator, leaving the unanchored form as the only option.

Choosing the covariates and the scale

TSD 18 Recommendation 2 asks a submission using anchored adjustment to show grounds for believing that at least one covariate is an effect modifier and that the imbalance is large enough to cause material bias. The review found little such evidence in 9 of 11 published MAIC and STC applications, and the MAICs matched on between 2 and 17 variables. It also rejected the post hoc argument that a change after weighting justifies the weighted result, because NICE analyses should be prespecified.

The scale of comparison matters because effect modification is scale specific. Recommendation 5 asks for indirect comparisons on the usual linear predictor scale for that outcome, such as log odds for a response probability, whereas TSD 18 found almost all published MAICs compared outcomes on the natural scale. The weighting model needs no outcome scale, yet the comparison that follows still assumes additivity on one, so TSD 18 cautions against calling MAIC scale-free.

Trials usually publish only marginal summaries, so MAIC assumes the AC trial's covariate correlations equal those in the weighted AB population, which for an anchored comparison matters only among effect modifiers.

Effective sample size and population overlap

Weighting concentrates information on the patients who most resemble the AC population, so the reweighted sample carries less information. TSD 18 gives the approximate effective sample size (ESS) proposed by Signorovitch and colleagues.

$$ESS = \frac{\left(\sum_{t \in \{A,B\}} \sum_{i=1}^{N_{t(AB)}} \hat{w}_{it}\right)^{2}}{\sum_{t \in \{A,B\}} \sum_{i=1}^{N_{t(AB)}} \hat{w}_{it}^{2}}$$

where $\hat{w}_{it}$ are the estimated weights over both arms of the AB trial. TSD 18 notes that this approximation is likely to underestimate the true ESS, because the weights are estimated and related to outcome, but a small ESS still warns of poor population overlap and inferences resting on a few patients.

In the TSD 18 review, the three MAICs that reported ESS saw an average fall of 80% against the original sample size. The psoriasis application first applied exclusion criteria, cutting 1,359 patients to 1,025, and weighting then gave an ESS of 591. Weighting cannot create patients the AB trial never enrolled, so TSD 18 asks for the weight distribution and the number of zero weights.

Standard errors and sampling uncertainty

Treating the estimated weights as fixed would understate uncertainty. TSD 18 reports that MAIC standard errors typically come from a sandwich estimator, and that bootstrapping or a Bayesian framework are alternatives that capture all sources of uncertainty. Bootstrap replicates of the whole analysis can also feed probabilistic sensitivity analysis. TSD 18 also notes that MAIC estimates outcomes in the AC trial sample rather than its parent population, so the sampling uncertainty of that sample is ignored.

Worked example with one effect modifier

All numbers in this example are illustrative. A company trial randomises 200 patients to A (placebo) and 200 to B. In each arm, 30% (60 patients) have had prior therapy. The competitor trial of C against A reports that 50% of its patients had prior therapy, with response rates of 25% on A and 40% on C. Prior therapy is taken, on evidence, to modify the effect of B on the log odds scale.

With one binary covariate, every patient with prior therapy receives the same weight $w_1$ and every other patient receives weight 1. Across both arms there are 120 patients with prior therapy and 280 without, and the weighted share with prior therapy must equal 50%.

$$w_1 = \frac{0.5 \times 280}{0.5 \times 120} = 2.33$$

where the 0.5 in the numerator is the target share with prior therapy and the 0.5 in the denominator the target share without it. The weights sum to 120 × 7/3 + 280 = 560, the patients with prior therapy hold 280 of that total, which is half, and $\alpha_1 = \log(7/3) = 0.847$.

$$ESS = \frac{560^{2}}{120 \times (7/3)^{2} + 280} = \frac{313600}{933.33} = 336$$

where 560 is the sum of the weights and the denominator is the sum of their squares, so 400 patients become an effective 336.

Within the company trial, response without prior therapy is 30% on A and 50% on B, and with prior therapy it is 20% on A and 25% on B. Unweighted, the arms respond at 0.7 × 0.30 + 0.3 × 0.20 = 0.27 on A and 0.7 × 0.50 + 0.3 × 0.25 = 0.425 on B, an odds ratio of $(0.425/0.575)/(0.27/0.73) = 2.00$. Weighted to a 50% share with prior therapy, the predictions are 0.5 × 0.30 + 0.5 × 0.20 = 0.25 on A and 0.5 × 0.50 + 0.5 × 0.25 = 0.375 on B, an odds ratio of $(0.375/0.625)/(0.25/0.75) = 1.80$. The competitor trial's published odds ratio for C against A is $(0.40/0.60)/(0.25/0.75) = 2.00$. On the log odds scale the anchored contrast is a difference, which becomes a ratio of odds ratios.

$$OR_{CB}^{MAIC} = \frac{OR_{CA}}{OR_{BA}^{MAIC}} = \frac{2.00}{1.80} = 1.11$$

where $OR_{CB}^{MAIC}$ is the odds ratio for C against B in the competitor trial's population, $OR_{CA}$ is the published odds ratio for C against A and $OR_{BA}^{MAIC}$ is the weighted odds ratio for B against A. The standard anchored comparison gives 2.00/2.00 = 1.00, no difference, while MAIC finds odds of response about 11% higher on C. The shift arises because B works less well after prior therapy and the competitor trial enrolled more such patients. The result describes that population only, and a submission would show both estimates with intervals.

Whose population the estimate describes

TSD 18 points out that a company running MAIC implicitly treats the competitor's trial as closer to the decision population than its own, whereas the target population is likely to be a UK cohort or registry that differs from both. It describes Novartis and AbbVie presenting apparently conflicting MAICs of the same ankylosing spondylitis trials of secukinumab and adalimumab, each estimating effects in the other company's trial population. Recommendation 6 therefore asks for the target population to be stated and for adjusted relative effects to be produced for it, which TSD 18 shows is possible under an added shared effect modifier assumption.

How NICE and decision models use the estimate

The NICE manual, PMG36 (section 3.4.14), says population adjustment in connected networks can be considered when effect modifiers may be imbalanced, needs individual patient data from at least one trial, and should have its limitations recognised and, if possible, the likely size of any systematic bias reported, pointing to TSD 18. TSD 18 asks for the standard indirect comparison to be presented alongside an anchored MAIC, and for a crude unadjusted difference alongside an unanchored one, so that the effect of the adjustment is visible.

In a cost-effectiveness analysis the MAIC estimate usually supplies the relative effect that drives the health economic model. Where the choice of adjustment method could change the decision, that choice is a source of structural uncertainty for scenario analysis.

Frequent misreadings of MAIC results

Matching on reported means balances only the covariates that were reported and matched, and leaves differences in the treatments themselves untouched. Three misreadings recur.

• Treating matched means as full balance. Signorovitch and colleagues state that MAIC, like any comparison of non-randomised groups, assumes no unobserved cross-trial differences that could confound outcomes.
• Expecting MAIC to fix treatment differences. TSD 18 states that population adjustment cannot correct differences in dosing, administration, co-treatments, titration or treatment switching.
• Using placebo tests as validation. TSD 18 notes that such tests detect imbalance only in prognostic variables and cannot detect imbalance in effect modifiers.

Sources

• Signorovitch JE, Wu EQ, Yu AP, Gerrits CM, Kantor E, Bao Y, Gupta SR, Mulani PM. Comparative effectiveness without head-to-head trials: a method for matching-adjusted indirect comparisons applied to psoriasis treatment with adalimumab or etanercept. PharmacoEconomics. 2010;28(10):935-945.
• Signorovitch JE, Sikirica V, Erder MH, Xie J, Lu M, Hodgkins PS, Betts KA, Wu EQ. Matching-adjusted indirect comparisons: a new tool for timely comparative effectiveness research. Value in Health. 2012;15(6):940-947.
• Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. NICE DSU Technical Support Document 18: Methods for population-adjusted indirect comparisons in submissions to NICE. Decision Support Unit, ScHARR, University of Sheffield; December 2016.
• Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. Methods for population-adjusted indirect comparisons in health technology appraisal. Medical Decision Making. 2018;38(2):200-211.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Section 3.4.14.