Stinnett and Mullahy confidence interval for incremental net health benefit from two trial arms

Estimates incremental net health benefit with the analogy estimator, the difference in arm mean effects less the difference in arm mean costs divided by lambda, and estimates its variance from each arm's sample variances of effect and cost, their within-arm correlation and the arm size (Stinnett and Mullahy, equations 6 and 7). The estimator is linear in the four sample means, so its variance is a direct sum of sample terms with no Taylor series approximation, and the interval is a normal (Wald) interval as in HE-FM-AN-001 (their equation 8). The statistic z_INHB, the estimate divided by its standard error, converts to the probability that net health benefit is positive through the standard normal cumulative distribution function, which the calculator lacks, so Excel NORM.S.DIST or R pnorm does that step. Multiplying every term by lambda gives the money-scale result of HE-FM-AN-004.

Signature

INHB = E_1 - E_0 - (C_1 - C_0) / lambda; Var_INHB = (s_E1^2 + s_C1^2 / lambda^2 - 2 * r_1 * s_E1 * s_C1 / lambda) / n_1 + (s_E0^2 + s_C0^2 / lambda^2 - 2 * r_0 * s_E0 * s_C0 / lambda) / n_0; SE_INHB = sqrt(Var_INHB); L_INHB = INHB - z * SE_INHB; U_INHB = INHB + z * SE_INHB; z_INHB = INHB / SE_INHB
Inputs
InputsDefinitionUnit
E_1Sample mean effect per patient in the arm given the new optionQALYs per patient
E_0Sample mean effect per patient in the comparator armQALYs per patient
C_1Sample mean cost per patient in the arm given the new optioncurrency per patient, for example GBP
C_0Sample mean cost per patient in the comparator armcurrency per patient
lambdaCost-effectiveness threshold, the cost at which one QALY is forgone elsewhere, fixed by the decision makercurrency per QALY
s_E1Sample standard deviation of patient effects in the new-option armQALYs
s_C1Sample standard deviation of patient costs in the new-option armcurrency
r_1Sample correlation between patient costs and patient effects in the new-option armnone, a correlation coefficient
n_1Number of patients in the new-option armpatients
s_E0Sample standard deviation of patient effects in the comparator armQALYs
s_C0Sample standard deviation of patient costs in the comparator armcurrency
r_0Sample correlation between patient costs and patient effects in the comparator armnone, a correlation coefficient
n_0Number of patients in the comparator armpatients
zStandard normal critical value, 1.96 for a two-sided 95% intervalnone
Output
INHBAnalogy estimate of the incremental net health benefit of the new option against the comparator at threshold lambdaQALYs per patient
Var_INHBSum of the two arms' variance terms in Stinnett and Mullahy's equation 7QALYs squared
SE_INHBSquare root of Var_INHBQALYs per patient
L_INHBLower limit of the normal interval for INHBQALYs per patient
U_INHBUpper limit of the normal interval for INHBQALYs per patient
z_INHBINHB divided by its standard error, the input to the standard normal cumulative distribution function for the probability that INHB is positivenone

Function

Net benefit approach to a two-option cost-effectiveness decision and its sampling uncertainty

Maps a threshold lambda and the differences in mean effect and mean cost between a new option and its comparator, or the per-arm trial data behind them, to incremental net benefit, whose sign gives the adoption decision, and to the sampling uncertainty around it. Option-level and incremental net monetary benefit, the maximum net monetary benefit rule and expected net benefit from simulations are on the Net Monetary Benefit and Expected Net Benefit pages (HE-FM-NMB-001 to HE-FM-NMB-003, HE-FM-ENB-001 and HE-FM-ENB-002). Incremental net health benefit at a threshold is HE-FM-CET-002, the ICER is HE-FM-ICER-001 and dominance is HE-FM-DOM-001. A normal interval for incremental net benefit built from the standard errors of the two differences is HE-FM-AN-004 with HE-CF-AN-001, its bootstrap variance is HE-FM-BSE-003, the acceptability curve and frontier are HE-FM-CEAC-001 to HE-FM-CEAC-003 and EVPI is HE-FM-VOI-001. This package adds Stinnett and Mullahy's interval for incremental net health benefit from two independent trial arms, the identity that links the sign of incremental net benefit to the ICER in each quadrant of the cost-effectiveness plane, and net-benefit regression. Notation follows the Net Benefit Approach article.

Computational function

  • Computational function for incremental net benefit on the money and health scales from per-arm trial summaries

    Takes the per-arm summaries a trial report gives, the mean effect and cost in each arm, their standard deviations, the within-arm correlation and the arm size, together with a threshold, and returns incremental net benefit on the money and health scales with its standard error, interval and z statistic, plus the ICER. It first builds the variances and covariance of the two mean differences from the arm summaries, the inputs that HE-FM-AN-004 and HE-CF-AN-001 start from, then forms incremental net monetary benefit and divides by lambda to return the net health benefit of HE-FM-NBA-001, Stinnett and Mullahy's equation 7. The ICER is returned for reading with the quadrant rule of HE-FM-NBA-002, and the Excel, R and Python versions add the probability that net benefit is positive.

    Inputs and outputs: E_1, E_0 are the arm mean effects in QALYs per patient. C_1, C_0 are the arm mean costs in currency per patient. s_E1, s_E0, s_C1, s_C0 are the patient-level standard deviations of effect and cost in each arm. r_1, r_0 are the within-arm correlations of cost and effect. n_1, n_0 are the arm sizes. lambda is the threshold in currency per QALY, above zero. z is the critical value, 1.96 for 95%. Delta_E, Delta_C are the differences in means and R the ICER. V_E, V_C and Cov_EC are the variances and covariance of the two differences. INMB, SE_INMB, M_L and M_U are incremental net monetary benefit, its standard error and limits in currency per patient. INHB, SE_INHB, H_L and H_U are the same in QALYs per patient. z_NB is the z statistic, the same on both scales. P_pos is the probability that net benefit is positive, returned by the Excel, R and Python versions only.

    Assumption: The arms are independent samples with costs and effects correlated within each arm, the normal approximation holds, and lambda is fixed.

    Worked example (Illustrative trial at GBP 25,000 per QALY): The trial of HE-EX-NBA-001 gives INMB of GBP 1,000 per patient with a 95% interval of about minus GBP 394 to GBP 2,394, INHB of 0.04 QALYs with about minus 0.0158 to 0.0958, and a probability of about 0.920 that net benefit is positive. E_1 = 0.80; E_0 = 0.70; C_1 = 6000; C_0 = 4500; s_E1 = 0.25; s_C1 = 3000; r_1 = 0.3; n_1 = 150; s_E0 = 0.25; s_C0 = 2500; r_0 = 0.2; n_0 = 150; lambda = 25000; z = 1.96; Delta_E = 0.10; Delta_C = 1500; R = 15000; V_E = 0.00083333; V_C = 101666.67; Cov_EC = 2.3333; INMB = 1000; SE_INMB = 711.22; M_L = -393.99; M_U = 2393.99; INHB = 0.04; SE_INHB = 0.028449; H_L = -0.015760; H_U = 0.095760; z_NB = 1.4060

    Worked example (Same trial at GBP 35,000 per QALY): INMB rises to GBP 2,000 with an interval of about GBP 80 to GBP 3,920, INHB is about 0.0571 QALYs with about 0.0023 to 0.1120, and the probability that net benefit is positive is about 0.979. E_1 = 0.80; E_0 = 0.70; C_1 = 6000; C_0 = 4500; s_E1 = 0.25; s_C1 = 3000; r_1 = 0.3; n_1 = 150; s_E0 = 0.25; s_C0 = 2500; r_0 = 0.2; n_0 = 150; lambda = 35000; z = 1.96; Delta_E = 0.10; Delta_C = 1500; R = 15000; V_E = 0.00083333; V_C = 101666.67; Cov_EC = 2.3333; INMB = 2000; SE_INMB = 979.37; M_L = 80.43; M_U = 3919.57; INHB = 0.057143; SE_INHB = 0.027982; H_L = 0.002298; H_U = 0.111988; z_NB = 2.0421

    Excel: =NORM.S.DIST((Threshold*(MeanEffect1-MeanEffect0)-(MeanCost1-MeanCost0))/SQRT(Threshold^2*(SdEffect1^2/ArmSize1+SdEffect0^2/ArmSize0)+SdCost1^2/ArmSize1+SdCost0^2/ArmSize0-2*Threshold*(Corr1*SdEffect1*SdCost1/ArmSize1+Corr0*SdEffect0*SdCost0/ArmSize0)),TRUE) Returns the probability that net benefit is positive from the named cells of HE-IM-NBA-001. The other outputs use the formulas of HE-IM-NBA-001 and HE-FM-AN-004.

    R: nb_arms <- function(e1, e0, c1, c0, sdE1, sdC1, r1, n1, sdE0, sdC0, r0, n0, lambda, z = 1.96) { dE <- e1-e0; dC <- c1-c0; vE <- sdE1^2/n1+sdE0^2/n0; vC <- sdC1^2/n1+sdC0^2/n0; cv <- r1*sdE1*sdC1/n1+r0*sdE0*sdC0/n0; inmb <- lambda*dE-dC; se <- sqrt(lambda^2*vE+vC-2*lambda*cv); list(R = dC/dE, INMB = inmb, SE_INMB = se, M = inmb+c(-1,1)*z*se, INHB = inmb/lambda, SE_INHB = se/lambda, H = (inmb+c(-1,1)*z*se)/lambda, z_NB = inmb/se, P_pos = pnorm(inmb/se)) } Returns a named list with both scales.

    Python: def nb_arms(e1, e0, c1, c0, sdE1, sdC1, r1, n1, sdE0, sdC0, r0, n0, lam, z=1.96): dE = e1-e0; dC = c1-c0; vE = sdE1**2/n1+sdE0**2/n0; vC = sdC1**2/n1+sdC0**2/n0; cv = r1*sdE1*sdC1/n1+r0*sdE0*sdC0/n0; inmb = lam*dE-dC; se = math.sqrt(lam**2*vE+vC-2*lam*cv); return {'R': dC/dE, 'INMB': inmb, 'SE_INMB': se, 'M': (inmb-z*se, inmb+z*se), 'INHB': inmb/lam, 'SE_INHB': se/lam, 'H': ((inmb-z*se)/lam, (inmb+z*se)/lam), 'z_NB': inmb/se, 'P_pos': statistics.NormalDist().cdf(inmb/se)} Requires the math and statistics modules.

    Test (Money-scale standard error over lambda equals equation 7): SE_INMB divided by lambda equals the standard error of HE-FM-NBA-001 from the same arm summaries. Expected result TRUE. Excel check =ABS(SQRT(Threshold^2*(SdEffect1^2/ArmSize1+SdEffect0^2/ArmSize0)+SdCost1^2/ArmSize1+SdCost0^2/ArmSize0-2*Threshold*(Corr1*SdEffect1*SdCost1/ArmSize1+Corr0*SdEffect0*SdCost0/ArmSize0))/Threshold-SeINHB)<1E-9

    Test (Probability at GBP 35,000 per QALY): The z statistic of the second worked example converts to a probability of 0.9794 that net benefit is positive. Expected result TRUE. Excel check =ABS(NORM.S.DIST(2000/979.3705,TRUE)-0.9794)<0.0001

    Common error (Covariance of the differences set to zero): At GBP 35,000 per QALY, dropping Cov_EC raises SE_INMB from about GBP 979 to GBP 1,059 and moves the interval to about minus GBP 77 to GBP 4,077, which now includes zero, with the probability falling to about 0.970 (computed here for illustration). Resampling or summarising costs and effects separately loses the within-arm correlation that the trial data carry.

    Source: Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Medical Decision Making. 1998;18(2 Suppl):S68-S80. Equations 6 to 8, checked in NBER Technical Working Paper 227 (1998). Tambour M, Zethraeus N, Johannesson M. A note on confidence intervals in cost-effectiveness analysis. International Journal of Technology Assessment in Health Care. 1998;14(3):467-471. Abstract, which multiplies effects by the price per unit of effectiveness so that standard statistical techniques give a confidence interval for net benefit in money.

    Delta_E = E_1 - E_0; Delta_C = C_1 - C_0; R = Delta_C / Delta_E; V_E = s_E1^2 / n_1 + s_E0^2 / n_0; V_C = s_C1^2 / n_1 + s_C0^2 / n_0; Cov_EC = r_1 * s_E1 * s_C1 / n_1 + r_0 * s_E0 * s_C0 / n_0; INMB = lambda * Delta_E - Delta_C; SE_INMB = sqrt(lambda^2 * V_E + V_C - 2 * lambda * Cov_EC); M_L = INMB - z * SE_INMB; M_U = INMB + z * SE_INMB; INHB = INMB / lambda; SE_INHB = SE_INMB / lambda; H_L = INHB - z * SE_INHB; H_U = INHB + z * SE_INHB; z_NB = INMB / SE_INMB

Try this function

Implementations

  • Excel

    Interval for incremental net health benefit from per-arm named cells

    With the arm means in MeanEffect1, MeanEffect0, MeanCost1 and MeanCost0, the standard deviations in SdEffect1, SdCost1, SdEffect0 and SdCost0, the correlations in Corr1 and Corr0, the arm sizes in ArmSize1 and ArmSize0, the threshold in Threshold and the critical value in ZValue, the first formula gives the estimate (cell INHB) and the second its standard error (cell SeINHB). The third and fourth give the limits, and the last gives the probability that net health benefit is positive.

    =MeanEffect1-MeanEffect0-(MeanCost1-MeanCost0)/Threshold; =SQRT((SdEffect1^2+SdCost1^2/Threshold^2-2*Corr1*SdEffect1*SdCost1/Threshold)/ArmSize1+(SdEffect0^2+SdCost0^2/Threshold^2-2*Corr0*SdEffect0*SdCost0/Threshold)/ArmSize0); =INHB-ZValue*SeINHB; =INHB+ZValue*SeINHB; =NORM.S.DIST(INHB/SeINHB,TRUE)

Assumptions

  • Independent samples for the two arms of a net health benefit interval

    The two arms are separate samples of patients, so costs and effects are uncorrelated across arms, while each arm's own costs and effects may be correlated, which r_1 and r_0 carry. Stinnett and Mullahy state that this is reasonable when the two interventions are observed in different samples. A paired or crossover design needs the covariance of the paired differences instead.

  • Normal approximation for the net health benefit estimator

    The interval relies on the Central Limit Theorem. Stinnett and Mullahy give a rule of thumb of 30 or more observations, and the estimator is exactly normal in finite samples when costs and effects are jointly multivariate normal. They describe a nonparametric bootstrap interval as the alternative, the safer choice when arms are small and costs heavily skewed, and the bootstrap variance of incremental net monetary benefit is HE-FM-BSE-003.

  • Threshold fixed for each net health benefit interval

    Lambda is a stated policy value, not estimated from the trial, so it adds no sampling variance. Stinnett and Mullahy recommend repeating the analysis over a range of lambda values and reporting net health benefit as a function of lambda.

Worked examples

  • Illustrative two-arm trial judged at GBP 25,000 per QALY

    Illustrative arms of 150 patients each have mean QALYs of 0.80 and 0.70, mean costs of GBP 6,000 and GBP 4,500, standard deviations of 0.25 QALYs in both arms and of GBP 3,000 and GBP 2,500 for cost, and within-arm correlations of 0.3 and 0.2. At GBP 25,000 per QALY, one of the two values PMG36 section 4.2.16 asks for, the estimate is 0.04 QALYs per patient with a 95% interval of about minus 0.0158 to 0.0958, so the interval includes zero. The z statistic of 1.406 corresponds to a probability of about 0.92 that net health benefit is positive.

    lambda = 25000; E_1 = 0.80; E_0 = 0.70; C_1 = 6000; C_0 = 4500; s_E1 = 0.25; s_C1 = 3000; r_1 = 0.3; n_1 = 150; s_E0 = 0.25; s_C0 = 2500; r_0 = 0.2; n_0 = 150; z = 1.96; INHB = 0.04; Var_INHB = 0.00080933; SE_INHB = 0.028449; L_INHB = -0.015760; U_INHB = 0.095760; z_INHB = 1.4060

  • Illustrative two-arm trial judged at GBP 35,000 per QALY

    The same illustrative trial at GBP 35,000 per QALY, the second PMG36 value, gives an estimate of about 0.0571 QALYs per patient with a 95% interval of about 0.0023 to 0.1120, which now excludes zero. The standard error falls slightly because the cost terms are divided by a larger lambda.

    lambda = 35000; E_1 = 0.80; E_0 = 0.70; C_1 = 6000; C_0 = 4500; s_E1 = 0.25; s_C1 = 3000; r_1 = 0.3; n_1 = 150; s_E0 = 0.25; s_C0 = 2500; r_0 = 0.2; n_0 = 150; z = 1.96; INHB = 0.057143; Var_INHB = 0.00078299; SE_INHB = 0.027982; L_INHB = 0.002298; U_INHB = 0.111988; z_INHB = 2.0421

Common errors

  • Within-arm correlation left out of the net health benefit variance

    Setting r_1 and r_0 to zero in the illustrative trial at GBP 25,000 per QALY raises the standard error from about 0.0284 to 0.0316 QALYs and widens the interval to about minus 0.0219 to 0.1019 (computed here for illustration). With positive correlations, as here, dropping them overstates the standard error, and with negative correlations it understates it.

  • Patient standard deviations not divided by the arm sizes

    Leaving out the division by n_1 and n_0 treats patient-level standard deviations as standard errors. In the illustrative trial at GBP 25,000 per QALY the standard error becomes about 0.348 instead of 0.0284, about 12.2 times larger (the square root of 150), and the interval spans about minus 0.643 to 0.723 QALYs (computed here for illustration).

Sources

  • Stinnett and Mullahy variance and interval for net health benefit

    Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Medical Decision Making. 1998;18(2 Suppl):S68-S80. Section on confidence intervals for net health benefits, equations 6 to 8, checked in the NBER Technical Working Paper 227 version (1998). Equation 6 is the analogy estimator, equation 7 the variance as a sum over the two arms of the effect variance plus the cost variance over lambda squared minus twice the correlation times both standard deviations over lambda, each divided by the arm size, and equation 8 the normal interval, with no correlation assumed across arms and a rule of thumb of 30 or more observations.

    View source →

  • NICE manual on net health benefits at GBP 25,000 and GBP 35,000 per QALY

    National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance, the manual (PMG36). Chapter 4, economic evaluation. Published 31 January 2022, last updated 13 October 2026. Section 4.2.16, which asks for expected net health benefits alongside ICERs, presented using values placed on a QALY gain of GBP 25,000 and GBP 35,000.

    View source →

Canonical Identity

Persistent URI
https://healtheconomics.wiki/formula/stinnett-and-mullahy-confidence-interval-for-incremental-net-health-benefit-from-two-trial-arms

Stable URI · Machine-readable · Resolvable · CC BY 4.0

Stinnett and Mullahy confidence interval for incremental net health benefit from two trial arms | HealthEconomics.wiki