Node-Splitting

Node-Splitting in Network Meta-Analysis

Node-splitting takes one treatment comparison in a network meta-analysis, estimates it twice, once from the trials that compared the two treatments head to head and once from the rest of the network, and tests whether the two answers differ by more than chance would allow. The difference between them is the inconsistency for that comparison, reported with a confidence or credible interval and a p value. In health technology assessment this matters because one set of network estimates usually supplies the relative effects in a cost-effectiveness analysis, and that set is only coherent if direct and indirect evidence agree. The NICE manual (PMG36, section 3.4.15) asks for inconsistencies between direct and indirect evidence to be reported and, where found, for an attempt to explain and resolve them. A non-significant split is weak reassurance, because such tests have little power.

Why direct and indirect evidence can disagree

A network meta-analysis assumes consistency, meaning that every relative effect can be written in terms of a smaller set of basic parameters. With treatment A as the reference, the effect of C relative to B is fixed by the effects of B and C relative to A.

$$d_{BC} = d_{AC} - d_{AB}$$

where $d_{AB}$ and $d_{AC}$ are the relative effects of B and of C against A, and $d_{BC}$ is the effect of C against B, all on an additive scale such as the log odds ratio.

NICE Decision Support Unit Technical Support Document (TSD) 4 explains that inconsistency, like heterogeneity, is caused by effect modifiers, and specifically by an imbalance in their distribution between the direct and the indirect evidence. If the BC trials enrolled patients who could not take A, and that changed how well B and C worked, the BC trials and the AB and AC trials would be estimating different things. TSD 4 calls inconsistency a special case of between-study heterogeneity, in which the effect modifier is associated with the set of comparisons a trial made.

Inconsistency can only arise in loops of evidence. For a network of two-arm trials, TSD 4 counts the independent loops as the inconsistency degrees of freedom.

$$\text{ICDF} = N - (n_t - 1)$$

where $\text{ICDF}$ is the inconsistency degrees of freedom, $N$ is the number of pairwise comparisons with direct evidence and $n_t$ is the number of treatments, so $n_t - 1$ is the number of basic parameters. A star-shaped network has an ICDF of zero, and no test can detect inconsistency in it.

Splitting one comparison into direct and indirect parameters

The name confuses readers, since a network diagram draws a comparison as an edge. Van Valkenhoef and colleagues explain that the node being split is a parameter node in the directed acyclic graph of the Bayesian hierarchical model, so splitting the node for $d_{XY}$ replaces one parameter with two. TSD 4 describes the direct component as based on all the XY data, which may come from XY, XYZ or WXY trials, and the indirect component as based on all the remaining evidence.

$$\hat{\omega}_{XY} = \hat{d}^{Dir}_{XY} - \hat{d}^{Ind}_{XY}$$

where $\hat{d}^{Dir}_{XY}$ is the estimate of the XY effect from the trials that compared X and Y, $\hat{d}^{Ind}_{XY}$ is the estimate from a network meta-analysis of everything else, and $\hat{\omega}_{XY}$ is the estimated inconsistency. The null hypothesis is that $\omega_{XY}$ equals zero.

The two estimates come from separate sets of trials, so they are independent and their variances add. As in the Bucher method set out in TSD 4, the ratio of the estimate to its standard error is referred to the standard normal distribution.

$$\text{Var}(\hat{\omega}_{XY}) = \text{Var}(\hat{d}^{Dir}_{XY}) + \text{Var}(\hat{d}^{Ind}_{XY}), \qquad z = \frac{\hat{\omega}_{XY}}{\sqrt{\text{Var}(\hat{\omega}_{XY})}}$$

where $\text{Var}$ denotes a variance and $z$ is the test statistic, whose two-sided p value comes from the standard normal distribution. A 95% interval for the inconsistency is $\hat{\omega}_{XY} \pm 1.96 \sqrt{\text{Var}(\hat{\omega}_{XY})}$.

Bayesian and frequentist versions

Dias and colleagues introduced node-splitting in 2010 within a hierarchical Bayesian approach, describing it as more general but computationally intensive and suited to networks with trial-level data. Fitted by Markov chain Monte Carlo, the posterior of $\omega_{XY}$ is summarised by its credible interval. In a random-effects model the between-trial variance is shared by the two parts, which TSD 4 notes solves the problem of comparisons supported by only one or two trials.

The Cochrane Handbook calls the same idea SIDE (separating indirect from direct evidence), recommends it as a local approach and notes that it is implemented in Stata (network) and R (netmeta). Unlike the loop-specific approach, the indirect estimate uses the entire network once the direct evidence is removed, a point the EU HTA Coordination Group practical guideline also makes.

Which comparisons to split

Only a comparison that sits in a loop has indirect evidence to compare with. Multi-arm trials complicate the choice, since TSD 4 notes there can be no inconsistency within a three-arm trial, and warns that node-split models are not easy to parameterise when multi-arm trials are present because more than one parameterisation is possible.

Van Valkenhoef and colleagues proposed a decision rule, implemented in the R package gemtc. Comparison XY is split if and only if the studies that do not include both X and Y still connect X and Y. They prove that the rule investigates every potentially inconsistent loop and splits only comparisons in such loops, and note that it tends to split more comparisons than there are such loops, so some splits test the same thing.

Worked example with four treatments for preventing exacerbations

All numbers in this example are illustrative. A submission compares standard care (A) with add-on treatments B, C and D for preventing exacerbations of a chronic lung disease. All trials have two arms, and each direct comparison has been pooled in a fixed-effect pairwise meta-analysis. Effects are log odds ratios of an exacerbation for the second-named treatment against the first, so a negative value favours the second treatment.

Four treatments and five compared pairs give $\text{ICDF} = 5 - (4 - 1) = 2$, the loops ABC and ACD. The decision is between B and C, so BC is split first.

1. Indirect estimate of A against C. Without the BC trials, the AC effect is informed directly and through D. The route through D gives $d = -0.30 - 0.45 = -0.75$ with $\text{Var} = 0.0225 + 0.0400 = 0.0625$. The inverse-variance weights are $w = 1/0.0144 = 69.44$ and $w = 1/0.0625 = 16$, so the pooled AC effect is $d = (69.44 \times -0.60 + 16 \times -0.75)/85.44 = -0.6281$ with $\text{Var} = 1/85.44 = 0.0117$.

2. Indirect estimate of B against C. By consistency, $d = -0.6281 - (-0.40) = -0.2281$, with $\text{Var} = 0.0117 + 0.0100 = 0.0217$, a standard error of 0.147.

3. Inconsistency. The direct estimate is 0.10, so $\hat{\omega}_{BC} = 0.10 - (-0.2281) = 0.3281$ with $\text{Var} = 0.0225 + 0.0217 = 0.0442$, a standard error of 0.210. Then $z = 0.3281/0.2102 = 1.56$, a two-sided p value of 0.12, and the 95% interval is -0.084 to 0.740.

4. What the two sources say. The direct trials suggest C is slightly worse than B (odds ratio 1.11), while the indirect evidence suggests C is better (0.80). Their ratio is 1.39, with a 95% interval of 0.92 to 2.10.

5. Combined estimate. The consistency model pools the sources with weights $w = 1/0.0225 = 44.44$ and $w = 1/0.0217 = 46.07$, giving $d = (44.44 \times 0.10 + 46.07 \times -0.2281)/90.51 = -0.067$, an odds ratio of 0.94 (95% interval 0.76 to 1.15). Direct evidence carries 49% of the weight.

6. Effect on the decision model. Suppose the model's annual exacerbation probability on B is 0.30, odds of 0.4286. The probability on C is then 0.321 using direct evidence only, 0.254 using indirect evidence only and 0.286 using the combined estimate. Per 1,000 patients, C causes 21 more exacerbations than B on the direct evidence, 46 fewer on the indirect evidence and 14 fewer combined. The test did not reject consistency, yet the evidence chosen changes the sign of the effect that drives costs and QALYs.

Splitting every comparison in turn gives these results.

The AB and BC splits give the same result, as do AD and CD with the sign reversed, because AB and BC lie only in loop ABC and AD and CD only in loop ACD. Five tests were run on a network with two independent inconsistencies. A Bayesian model of the arm-level counts, or a random-effects model, would give somewhat different numbers.

Reading a node-split in an HTA submission

The commonest error is reading a large p value as proof of consistency. TSD 4 states that tests for inconsistency are inherently underpowered and will often fail to detect it, likening a null result to concluding no difference from a small, underpowered trial. The Cochrane Handbook makes the same point and advises checking whether the interval for the inconsistency includes clinically important values, as the interval of 0.92 to 2.10 for the ratio of odds ratios does above.

Heterogeneity can hide inconsistency. TSD 4 notes that the greater the between-trial heterogeneity, the less detectable inconsistency becomes, and that vague priors with sparse data can mask all but the most obvious signs. Van Valkenhoef and colleagues add that a smaller heterogeneity estimate in a split model than in the consistency model is a reason to suspect inconsistency. PMG36 (section 3.4.16) allows informative priors for between-study heterogeneity in networks with few studies.

The p value has no agreed cut-off. The EU HTA Coordination Group practical guideline says its interpretation is context dependent and that no formal framework for the required significance level exists. With five independent tests at the 5% level, the chance of at least one false alarm would be one minus 0.95 to the fifth power, or 0.226, but node-split tests are not independent, and TSD 4 notes for repeated loop tests that the right adjustment is difficult to specify.

Responding when a split shows conflict

A significant split does not identify the faulty trials. TSD 4 stresses that inconsistency is a property of loops, not of individual contrasts, and that in a triangle it is not possible to say which contrast is deviant, as the identical AB and BC results show. It advises reconsidering the entire network, warns that removing or adjusting different trials may each restore consistency yet give very different estimates, and calls for sensitivity analysis.

TSD 4 concludes that decisions should be based on coherent models that fit the data, and that a thorough review of the evidence by clinical epidemiologists is required, looking for trials that differ in effect modifiers such as dose, baseline severity or previous treatment. Van Valkenhoef and colleagues warn that unexplained inconsistency may mean abandoning the synthesis or interpreting it with extreme caution. The probabilistic sensitivity analysis of a consistency model does not capture unexplained inconsistency, so scenarios using the direct or the indirect estimate show whether the decision depends on it.

Node-splitting beside other inconsistency checks

Node-splitting is a local method. For a single triangle of two-arm trials with fixed effects it reproduces the Bucher comparison set out on the indirect comparison page, and TSD 4 prefers the Bucher method and its chi-square extension wherever independent tests can be constructed. Global methods test the whole network, for example by comparing the consistency model with an unrelated mean effects model on residual deviance and the deviance information criterion, as TSD 4 describes. The design-by-treatment interaction model gives a global test and, unlike node-splitting, treats a difference between three-arm and two-arm trials as a design inconsistency. Dias and colleagues also proposed back-calculation, which infers the indirect contribution from the direct evidence and the network output when only pooled summaries are available.

Sources

• Chaimani A, Caldwell DM, Li T, Higgins JPT, Salanti G. Chapter 11: Undertaking network meta-analyses (last updated October 2019). In: Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-11
• Dias S, Welton NJ, Caldwell DM, Ades AE. Checking consistency in mixed treatment comparison meta-analysis. Statistics in Medicine. 2010;29(7-8):932-944. https://doi.org/10.1002/sim.3767
• Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. NICE DSU Technical Support Document 4: Inconsistency in networks of evidence based on randomised controlled trials. Decision Support Unit, ScHARR, University of Sheffield; May 2011, last updated April 2014. https://doi.org/10.15131/shef.data.33737323.v1
• Health Technology Assessment Coordination Group. Practical Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons. Adopted 8 March 2024 under Regulation (EU) 2021/2282. https://health.ec.europa.eu/publications/practical-guideline-quantitative-evidence-synthesis-direct-and-indirect-comparisons_en
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 3.4.15 and 3.4.16. https://www.nice.org.uk/process/pmg36/chapter/evidence-2
• van Valkenhoef G, Dias S, Ades AE, Welton NJ. Automated generation of node-splitting models for assessment of inconsistency in network meta-analysis. Research Synthesis Methods. 2016;7(1):80-93. https://doi.org/10.1002/jrsm.1167