Network Meta-Analysis

Network Meta-Analysis: NMA Methods, Assumptions and Use in HTA

A network meta-analysis (NMA) can estimate relative effects for treatment pairs that have never been compared in the same trial, and it produces one coherent set of estimates across a connected evidence network. Health technology assessment needs exactly this, because a decision usually involves several competing treatments and the decision model needs a relative effect for each of them. This page explains the evidence geometry, the transitivity and consistency assumptions, fixed- and random-effects models in Bayesian and frequentist form, inconsistency checks, ranking measures, certainty and reporting frameworks, and the use of NMA results in economic evaluation.

NMA extends pairwise meta-analysis; it does not remove the need for comparable studies. Its conclusions depend on the network being connected and on the clinical and methodological conditions needed for valid indirect comparison.

A network links treatments through study comparisons

The evidence network is represented by treatments as nodes and direct randomised comparisons as edges. Node size may show the number of participants receiving a treatment, and edge width may show the number of studies making a comparison. The diagram is descriptive and should not be interpreted as evidence quality by itself.

Every node should have a precise intervention definition, including dose, route, schedule and co-intervention when these can modify relative effects. Combining meaningfully different regimens under one label can create apparent precision while obscuring heterogeneity.

Direct and indirect evidence answer the same contrast

Direct evidence for treatment $A$ versus $C$ comes from studies that randomise participants between $A$ and $C$. Indirect comparison can estimate the same contrast through a common comparator $B$ when studies compare $A$ with $B$ and $B$ with $C$. The estimates can be combined when the relevant effect modifiers are sufficiently comparable across those studies.

On an additive effect scale, the indirect estimate is:

$$ d_{AC}^{\text{indirect}}=d_{AB}+d_{BC} $$

where $d_{XY}$ denotes the relative effect of treatment $X$ compared with treatment $Y$ on the chosen scale. This is the adjusted indirect comparison described by Bucher and colleagues. Notation differs between sources: the NICE Decision Support Unit technical support documents write $d_{XY}$ for the effect of $Y$ relative to $X$, the reverse of the convention used here, so the sign of each contrast should be read in the convention of the source being used. Because it combines relative effects estimated within each set of trials, it preserves randomisation, unlike a naive comparison of the active arms taken from different trials.

Equivalently, if both effects are expressed relative to $B$:

$$ d_{AC}^{\text{indirect}}=d_{AB}-d_{CB} $$

Here $d_{CB}$ is the effect of $C$ relative to $B$, so $d_{BC}=-d_{CB}$ on an additive scale. The sign depends on the direction used for every contrast. For ratio measures, the relationship is additive on the log scale and multiplicative after exponentiation.

A worked indirect comparison

The numbers in this example are illustrative. Suppose trials estimate that treatment $A$ reduces a continuous outcome by 4 units compared with $B$, and treatment $C$ reduces it by 1 unit compared with $B$. Define negative values as favourable reductions relative to the comparator. The indirect comparison asks how $A$ differs from $C$.

Let $d_{AB}=-4$ and $d_{CB}=-1$, each a mean difference relative to $B$.

Then:

$$ d_{AC}^{\text{indirect}} =d_{AB}-d_{CB} =-4-(-1) =-3 $$

Here treatment $A$ is estimated to reduce the outcome by 3 units more than treatment $C$. The variance of the indirect estimate is the sum of the component variances only when the two estimates are independent; shared studies or multi-arm structures require their covariance to be represented.

Transitivity is the central clinical assumption

Transitivity means that the studies could, in principle, have randomised participants to any of the treatments being compared, after accounting for relevant effect modifiers. It supports the use of one set of studies to infer a comparison in another part of the network. This assumption cannot be established by a statistical test alone.

Potential effect modifiers include:

• Baseline disease severity or risk.
• Previous treatment and treatment line.
• Age, comorbidity and other prognostic characteristics.
• Dose, route, duration and background therapy.
• Outcome definition and assessment time.
• Study setting, calendar period and standard of care.
• Risk of bias features that can modify relative effects.

The analyst should compare the distribution of important effect modifiers across each treatment comparison. A network can be mathematically connected but clinically intransitive.

Consistency is the statistical expression of transitivity

Consistency means that direct and indirect evidence estimate the same underlying relative effect, within sampling and heterogeneity variation. In a consistent network, different evidence paths agree sufficiently to be represented by one coherent set of treatment effects. Inconsistency, also called incoherence, is disagreement beyond what the model expects.

For a three-treatment loop, consistency implies:

$$ d_{AC}=d_{AB}+d_{BC} $$

where each $d_{XY}$ is the underlying relative effect of $X$ compared with $Y$. A loop inconsistency factor can be written as:

$$ IF=d_{AC}^{\text{direct}}-d_{AC}^{\text{indirect}} $$

where $IF$ is the difference between the direct estimate and the indirect estimate of the same contrast. An inconsistency estimate near zero is compatible with agreement, but a wide interval may reflect low power rather than confirmed consistency. Statistical checks should be interpreted alongside the clinical assessment of transitivity.

Contrast-based models estimate relative treatment effects

A contrast-based NMA models study-level relative effects while preserving randomisation within each trial. Treatment effects are expressed relative to a reference treatment or through a set of basic parameters. All other pairwise contrasts are derived from those parameters.

For a two-arm study $i$ comparing treatments $a$ and $b$, an observed effect estimate may be represented as:

$$ y_{i,ab}\sim \operatorname{Normal}(\delta_{i,ab},s_{i,ab}^2) $$

where $y_{i,ab}$ is the observed relative effect of $b$ versus $a$ in study $i$, $\delta_{i,ab}$ is the underlying relative effect in that study and $s_{i,ab}^2$ is its within-study variance, treated as known. This subscript order follows the NICE Decision Support Unit convention, in which $\delta_{i,ab}$ is the effect of $b$ relative to $a$, the reverse of the $d_{XY}$ notation used for indirect comparisons above.

Under a fixed-effect consistency model:

$$ \delta_{i,ab}=d_b-d_a $$

where $d_k$ is the effect of treatment $k$ relative to a chosen network reference with $d_{ref}=0$. The effect of $b$ versus $a$ is then $d_b-d_a$.

The model must use an effect scale appropriate to the outcome, such as mean difference, log odds ratio, log risk ratio or log hazard ratio. Mixing effect scales or directions within one analysis invalidates the network relationships.

The same core model can be estimated in a Bayesian or a frequentist framework. NICE DSU Technical Support Document (TSD) 2 sets out a generalised linear modelling framework estimated by Bayesian Markov chain Monte Carlo simulation, which fits naturally with probabilistic decision models because posterior samples keep the correlation between treatment effects that arises from their joint estimation. Frequentist estimation is equally valid; for probabilistic modelling, its maximum likelihood estimates and their covariance matrix can define a multivariate normal distribution for sampling, or the data can be bootstrapped.

Random-effects models allow treatment effects to vary

A random-effects NMA allows the true relative effect to vary across studies. A common model assumes one heterogeneity variance across treatment comparisons, although alternative structures may be considered when supported by evidence. Sparse networks often provide limited information about heterogeneity, making prior or external assumptions influential in Bayesian analysis.

A simple random-effects formulation is:

$$ \delta_{i,ab}\sim \operatorname{Normal}(d_b-d_a,\tau^2) $$

where $\tau$ is the between-study standard deviation on the chosen effect scale. The resulting interval around the mean relative effect reflects uncertainty in the network mean; a predictive interval also represents the expected variation in a new study setting.

The NICE manual (PMG36) asks for external information to help estimate between-study heterogeneity, and notes that informative prior distributions for the heterogeneity parameter may be preferable in networks with few studies.

Choosing fixed or random effects should not be based only on which model gives a preferred result. The decision should consider clinical heterogeneity, network size, model fit, predictive purpose and the plausibility of the heterogeneity structure. In the TSD 2 framework, fit is assessed with the residual deviance and competing models are compared with the deviance information criterion (DIC).

Arm-based likelihoods can model outcomes directly

Some NMA models use arm-level outcomes with a likelihood appropriate to the observed data, such as the binomial likelihood with a logit link in TSD 2. This remains a contrast-based model, because treatment effects are still estimated within each study relative to a study-specific baseline. For a binary outcome in study $i$, events $r_{ik}$ among $n_{ik}$ participants assigned to treatment $k$ may be modelled as:

$$ r_{ik}\sim\operatorname{Binomial}(n_{ik},p_{ik}) $$

where $p_{ik}$ is the event probability in arm $k$ of study $i$, with:

$$ \operatorname{logit}(p_{ik})=\mu_i+\delta_{ik} $$

where $\mu_i$ is a study-specific baseline and $\delta_{ik}$ is the relative effect for arm $k$ within that study. Count, continuous and time-to-event outcomes require corresponding likelihoods and link functions.

Arm-level data do not permit the analyst to ignore within-study randomisation. Study-specific baselines or equivalent structures are needed so that between-study differences in absolute risk do not become treatment effects. A separate class of fully arm-based models treats study baselines as random rather than as unrelated study parameters; these models make additional assumptions about how baselines vary across studies and differ from the approach in TSD 2.

Multi-arm trials create correlated comparisons

A three-arm or larger trial contributes several pairwise comparisons, but those contrasts share participants and a common randomisation. Treating them as independent double counts information and underestimates uncertainty. The statistical model must include the induced covariance or use a likelihood formulation that preserves the multi-arm structure.

For a three-arm study with treatments $A$, $B$ and $C$, the contrasts $B-A$ and $C-A$ share arm $A$. Their covariance is not zero. A correctly specified NMA handles the study as one multi-arm randomised experiment rather than as several unrelated two-arm studies.

Connectedness determines which treatments can be estimated together

Standard NMA requires a connected network. If treatments fall into separate components without a common comparator path, relative effects between those components are not identified from the randomised evidence. A statistical model cannot create the missing link without additional assumptions or external evidence.

Analysts should:

• Display every connected component before modelling.
• Confirm that an apparent connection is not created by an incorrectly merged treatment node.
• Avoid presenting comparisons between disconnected components as ordinary NMA estimates.
• Separate analyses or use a clearly justified alternative evidence-synthesis framework when no defensible connection exists.

A single small or high-risk study can act as the only bridge between large parts of a network. Its influence should be investigated through sensitivity and contribution analyses.

When treatments cannot be connected, or when effect modifiers are imbalanced between trials and individual patient data are available for at least one trial, population-adjusted indirect comparisons such as matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC) are sometimes used. NICE DSU TSD 18 distinguishes anchored comparisons, which retain a common comparator, from unanchored comparisons without one. Unanchored comparisons rely on much stronger assumptions that are rarely defensible, so they do not provide a routine substitute for a connected network.

Heterogeneity and inconsistency are different

Heterogeneity is variation in true treatment effects among studies making the same comparison. Inconsistency is disagreement between different evidence paths for the same comparison. Both can occur together, and high heterogeneity can reduce the ability to detect inconsistency.

An analysis should not describe a non-significant inconsistency test as proof that the network is consistent. Sparse networks often have little power to detect disagreement.

Local and global methods examine inconsistency

Local methods assess particular loops or comparisons, while global methods assess the network as a whole. Different methods answer different diagnostic questions and may identify different problems. No single test replaces examination of the studies and effect modifiers.

Common approaches include:

• Node splitting separates direct and indirect evidence for a selected comparison.
• Loop-specific methods estimate inconsistency within closed loops.
• Design-by-treatment interaction models test inconsistency across the network while recognising study designs.
• Unrelated mean-effects models relax consistency constraints and compare fit with the consistency model.
• Residual and influence diagnostics identify studies or comparisons that contribute strongly to lack of fit.

The design-by-treatment interaction model handles multi-arm trials, which complicate loop-based checks, and the earlier Lu and Ades inconsistency model is a restricted version of it. NICE DSU TSD 4 stresses that inconsistency tests are inherently underpowered and will often fail to detect inconsistency that is present.

When inconsistency is found, the response should investigate data errors, treatment definitions, outcome timing, effect modifiers and risk of bias. Removing evidence solely to obtain statistical consistency is not a defensible default.

Meta-regression can adjust for effect modifiers

Network meta-regression adds study-level covariates to explain variation in relative effects and address identified differences across comparisons. It may improve transportability when the effect modifier is measured consistently and there is sufficient variation and overlap. With aggregate study data, it remains vulnerable to ecological bias and cannot recover individual-level interactions reliably.

A general interaction model is:

$$ \delta_{i,ab}= d_b-d_a+ (\beta_b-\beta_a)(x_i-\bar x) $$

where $x_i$ is a study-level covariate, $\bar x$ is its mean across studies and $\beta_k$ is the interaction for treatment $k$ relative to the network reference, with $\beta_{ref}=0$. A common-interaction model constrains every non-reference $\beta_k$ to a single value $\beta$, and NICE DSU TSD 3 considers this the version most likely to be useful for decision making. Exchangeable or independent treatment-specific interaction terms require substantially more information than a common interaction and can be weakly identified in sparse networks.

Covariate adjustment should be prespecified where possible and supported by a causal rationale. Searching many covariates for a model that removes inconsistency can create unstable and selective results.

Rankings summarise uncertainty but do not replace effects

NMA can estimate the probability that each treatment occupies each rank. Ranking probabilities, mean rank and the surface under the cumulative ranking curve (SUCRA) can summarise the distribution. They should be interpreted with the estimated effect sizes, uncertainty and outcome importance.

If treatment $k$ has cumulative ranking probabilities $F_k(r)=P(Rank_k\le r)$ among $K$ treatments, one expression for SUCRA is:

$$ SUCRA_k= \frac{1}{K-1} \sum_{r=1}^{K-1}F_k(r) $$

Here SUCRA is the mean of the first $K-1$ cumulative ranking probabilities. It ranges from zero to one, with larger values indicating more favourable ranks under the chosen outcome direction. A high SUCRA does not establish a clinically meaningful advantage, cost-effectiveness or high-certainty evidence.

Small and uncertain differences can still create a complete ranking. Presenting rank alone can therefore exaggerate distinctions between treatments whose effect distributions substantially overlap. In frequentist NMA, the P-score is an analogue of SUCRA calculated from point estimates and standard errors; the two give nearly identical values, and neither offers a major advantage over examining the confidence or credible intervals themselves.

League tables show every pairwise estimate

A league table displays the estimated relative effect for each treatment pair, usually with confidence or credible intervals. It provides a compact view of the coherent network estimates. The direction, effect measure, row-versus-column convention and favourable outcome direction must be stated clearly.

The table should identify whether each entry is:

• The network estimate combining direct and indirect evidence.
• A direct-only estimate.
• An indirect-only estimate.
• An estimate unavailable because the network is disconnected.
• Based on a subgroup, time point or alternative model.

Reversing the row and column changes the sign of a difference measure and takes the reciprocal of a ratio measure. Ambiguous orientation is a common source of incorrect interpretation and economic-model inputs.

Baseline risk is needed for absolute outcomes

NMA usually estimates relative treatment effects, while economic models often require absolute event probabilities, survival or mean outcomes. A baseline-risk model is needed to apply the relative effects to the target population. The baseline source should be relevant to the decision setting and should not be double counted as additional relative-effect evidence. NICE DSU TSD 5 covers evidence synthesis for this baseline natural history model.

For a log-odds-ratio NMA with baseline event probability $p_0$ on the reference treatment and treatment effect $d_k$:

$$ \operatorname{logit}(p_k)= \operatorname{logit}(p_0)+d_k $$

where $p_k$ is the event probability on treatment $k$ and $d_k$ is the log odds ratio of $k$ relative to the reference, so:

$$ p_k= \frac{\exp\left[\operatorname{logit}(p_0)+d_k\right]} {1+\exp\left[\operatorname{logit}(p_0)+d_k\right]} $$

Here the relative effect is added on the logit scale before converting back to a probability. Applying an odds ratio directly as if it were a risk ratio produces incorrect absolute risks. Joint uncertainty in baseline risk and relative effects should be propagated into the economic model.

Economic models need coherent joint treatment effects

NMA can provide a consistent set of relative effects for several interventions in a cost-effectiveness model. Because all basic parameters are estimated jointly, their covariance matters. Sampling each pairwise estimate independently can create internally inconsistent treatment effects and misstate decision uncertainty.

In NICE appraisals, the manual (PMG36) states that when technologies have not been compared within a single randomised controlled trial, the pairwise head-to-head trials should be presented together with a network meta-analysis if appropriate. It also states a preference for the network meta-analysis methods in the NICE DSU evidence synthesis series of technical support documents.

Economic-model implementation should preserve:

• The posterior or variance-covariance structure of the basic treatment effects.
• The chosen reference treatment and effect direction.
• Correlation across outcomes when treatment effects are modelled jointly or linked.
• Baseline-risk uncertainty and any relationship with relative effects.
• Heterogeneity or predictive uncertainty when the decision requires a new-setting effect.
• Treatment-specific discontinuation, safety and other outcomes that are not captured by one efficacy network.

If efficacy and safety are synthesised in separate networks, their treatment nodes and study populations should be reconciled before they enter one model.

Risk of bias and certainty remain comparison specific

The quality of an NMA is not a single network-wide property. Risk of bias, indirectness, imprecision, heterogeneity, incoherence and publication bias can differ across pairwise estimates. A treatment ranking may depend on comparisons supported by weak or indirect evidence.

Assessment should consider:

• Risk of bias within studies contributing to each estimate.
• The contribution of direct and indirect evidence paths.
• Indirectness from differences in population, intervention, comparator or outcome.
• Imprecision of the relative effect and clinically relevant thresholds.
• Heterogeneity and inconsistency.
• Small-study effects and selective publication.
• Dependence on a bridge study or a particular treatment-node definition.

Two structured frameworks apply this comparison-specific view. The GRADE Working Group approach for NMA rates the certainty of the direct, indirect and network estimate for each comparison. CINeMA (Confidence in Network Meta-Analysis) judges each network estimate across six domains: within-study bias, reporting bias, indirectness, imprecision, heterogeneity and incoherence, and it uses a contribution matrix showing how much each study informs each estimate.

The evidence certainty attached to a comparison should travel with that estimate into interpretation and decision modelling.

Common analysis and interpretation errors

NMA can generate an estimate for every treatment pair in a connected network, but numerical completeness is not evidence of validity. Most serious errors involve transitivity, node definition, multi-arm correlation, ranking and transfer into economic models. The following failures can materially change conclusions.

• Treating network connectedness as proof of transitivity ignores effect-modifier differences across comparisons.
• Combining different doses or regimens without justification can conceal clinically important heterogeneity.
• Counting multi-arm contrasts as independent studies underestimates uncertainty.
• Interpreting a non-significant inconsistency test as confirmed agreement ignores low power.
• Selecting a model only by statistical fit can overlook clinical plausibility and predictive purpose.
• Presenting treatment ranks without effect sizes and intervals can exaggerate trivial or uncertain differences.
• Reversing contrast direction in a league table can make an effective treatment appear harmful or vice versa.
• Applying odds ratios as risk ratios produces incorrect absolute outcomes.
• Sampling pairwise NMA estimates independently in a probabilistic model breaks network coherence.
• Using one baseline risk for a different target population without adjustment can misrepresent absolute benefit and cost-effectiveness.

A practical validation sequence

Validation should begin with study eligibility and treatment definitions before statistical modelling. Network calculations should then be reconciled with direct pairwise evidence and transferred into the economic model using coherent joint uncertainty. The following sequence provides an auditable minimum.

1. Define the decision population, interventions, comparators, outcomes and time points before constructing the network.
2. Verify every treatment node and study arm against dose, route, schedule and background therapy.
3. Draw the network and identify disconnected components, sparse links and bridge studies.
4. Assess the distribution of prespecified effect modifiers across each direct comparison.
5. Reproduce conventional pairwise meta-analyses before fitting the network model.
6. Represent multi-arm study covariance correctly and confirm contrast directions and effect scales.
7. Fit clinically plausible fixed- and random-effects structures and compare fit, residuals and predictive implications.
8. Assess inconsistency with local and global methods while recognising limited power.
9. Test sensitivity to risk of bias, treatment-node definitions, priors, heterogeneity and influential studies.
10. Report treatment effects with intervals and evidence certainty before presenting ranking measures.
11. Validate the baseline-risk transformation and absolute outcomes used in the economic model.
12. Preserve the joint covariance or posterior draws of treatment effects in probabilistic decision analysis.
13. Report the review against the PRISMA extension for network meta-analyses (PRISMA-NMA), which sets out a 32-item checklist for systematic reviews that incorporate an NMA.

Sources

• Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. Journal of Clinical Epidemiology. 1997;50(6):683-691.
• Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. Journal of the American Statistical Association. 2006;101(474):447-459.
• Dias S, Welton NJ, Caldwell DM, Ades AE. Checking consistency in mixed treatment comparison meta-analysis. Statistics in Medicine. 2010;29(7-8):932-944.
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. Decision Support Unit, ScHARR, University of Sheffield; 2011 (last updated September 2016).
• Dias S, Sutton AJ, Welton NJ, Ades AE. NICE DSU Technical Support Document 3: Heterogeneity: subgroups, meta-regression, bias and bias-adjustment. Decision Support Unit, ScHARR, University of Sheffield; 2011 (last updated April 2012).
• Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. NICE DSU Technical Support Document 4: Inconsistency in networks of evidence based on randomised controlled trials. Decision Support Unit, ScHARR, University of Sheffield; 2011 (last updated April 2014).
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 5: Evidence synthesis in the baseline natural history model. Decision Support Unit, ScHARR, University of Sheffield; 2011 (last updated April 2012).
• Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. NICE DSU Technical Support Document 18: Methods for population-adjusted indirect comparisons in submissions to NICE. Decision Support Unit, ScHARR, University of Sheffield; 2016.
• Salanti G, Ades AE, Ioannidis JPA. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. Journal of Clinical Epidemiology. 2011;64(2):163-171.
• Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Research Synthesis Methods. 2012;3(2):80-97.
• Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Research Synthesis Methods. 2012;3(2):98-110.
• Puhan MA, Schünemann HJ, Murad MH, Li T, Brignardello-Petersen R, Singh JA, Kessels AG, Guyatt GH; GRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ. 2014;349:g5630.
• Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Annals of Internal Medicine. 2015;162(11):777-784.
• Rücker G, Schwarzer G. Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Medical Research Methodology. 2015;15:58.
• Nikolakopoulou A, Higgins JPT, Papakonstantinou T, Chaimani A, Del Giovane C, Egger M, Salanti G. CINeMA: an approach for assessing confidence in the results of a network meta-analysis. PLoS Medicine. 2020;17(4):e1003082.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, updated 31 March 2026.