Signature
omega = d_BC_dir - d_BC_ind; V_omega = V_BC_dir + V_BC_ind; z = (d_BC_dir - d_BC_ind) / sqrt(V_BC_dir + V_BC_ind)
| Inputs | Definition | Unit |
|---|---|---|
d_BC_dir | Direct estimate of the effect of C compared with B from head-to-head trials | effect scale, for example log odds ratio |
d_BC_ind | Indirect estimate d_AC minus d_AB from the Bucher comparison on this page | effect scale, for example log odds ratio |
V_BC_dir | Variance of d_BC_dir | squared units of the effect scale |
V_BC_ind | Variance of d_BC_ind, the sum of the two direct variances | squared units of the effect scale |
omega | Direct minus indirect estimate of the effect of C relative to B | effect scale, for example log odds ratio |
|---|---|---|
V_omega | Variance of omega | squared units of the effect scale |
z | Inconsistency estimate divided by its standard error, referred to the standard normal distribution | no unit |
Function
Network consistency function
Expresses every pairwise relative effect in a connected network through effects relative to a common reference treatment A, on a scale where effects add, such as the log odds ratio. Here d_XY is the effect of Y relative to X, the NICE Decision Support Unit convention. Consistency is the assumption that direct and indirect evidence estimate the same d_XY.
Implementations
Excel
Inconsistency, variance, z and two-sided p value
With the direct and indirect estimates and variances in named cells, the four formulas return omega, its variance, z and the two-sided p value.
=dBCdir-dBCind; =VBCdir+VBCind; =Omega/SQRT(VarOmega); =2*(1-NORM.S.DIST(ABS(Zstat),TRUE))
Assumptions
Three independent sources of evidence
The three direct estimates come from separate sets of trials. Three-arm trials are excluded from the test because they are internally consistent and would dilute any disagreement.
Worked examples
Inconsistency in the TSD 4 HIV loop
A direct log odds ratio of 0.47 (standard error 0.10) for C against B, compared with the indirect estimate of minus 1.37 (variance 0.4292), gives an inconsistency of 1.84 with variance 0.4392 and z of about 2.78, evidence of inconsistency with p below 0.01, as in Box 1 of TSD 4.
d_BC_dir = 0.47; d_BC_ind = -1.37; V_BC_dir = 0.01; V_BC_ind = 0.4292; omega = 1.84; V_omega = 0.4392; z = 2.78
Common errors
Reading a non-significant test as proof of consistency
TSD 4 stresses that inconsistency tests are underpowered and will often miss inconsistency that is present. A z value below 1.96 is compatible with agreement but does not establish it, and the clinical assessment of transitivity still applies.
Sources
Bucher method for testing inconsistency in a loop
Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. NICE DSU Technical Support Document 4: Inconsistency in networks of evidence based on randomised controlled trials. Sheffield: Decision Support Unit, ScHARR, University of Sheffield; May 2011, last updated April 2014. Section 3.1 Bucher method for single loops of evidence (the inconsistency estimate omega as direct minus indirect, its variance and the approximate z test), section 3.2 (loops with more edges) and Box 1 worked example with the HIV virologic suppression network.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0