Wald large-sample confidence interval for an estimate

Signature

L = theta_hat - z * SE; U = theta_hat + z * SE

Function

Large-sample normal confidence interval function

Computational function

• Computational function: delta-method, Fieller and net benefit intervals from trial summary statistics

  Takes the summary statistics a trial-based analysis produces, the incremental mean cost and effect, their standard errors and covariance, and a threshold, and returns three large-sample intervals in one pass: the delta-method interval for the ICER (HE-FM-AN-002), Fieller's interval (HE-FM-AN-003) and the incremental net benefit interval with the probability that net benefit is positive (HE-FM-AN-004). The delta-method and net benefit intervals are Wald intervals (HE-FM-AN-001) applied to the ratio and to a linear combination of the differences; Fieller's method applies the normal approximation to the two differences themselves. The procedure branches on the sign of the Fieller coefficient a, so it reports whether Fieller's set is a bounded interval, two unbounded pieces or the whole real line, rather than returning two roots that may bound nothing.

  Inputs and outputs: Delta_C: Incremental mean cost, intervention minus comparator; required, non-zero. Unit: currency.; Delta_E: Incremental mean effect; required, non-zero. Unit: effect, for example QALYs.; SE_C: Standard error of Delta_C; required, zero or above. Unit: currency.; SE_E: Standard error of Delta_E; required, zero or above. Unit: effect.; Cov_CE: Covariance of the two differences; optional, default 0. Unit: currency times effect.; lambda: Threshold for net benefit; required. Unit: currency per unit of effect.; z: Standard normal critical value; optional, default 1.96 for a 95% interval. Unit: none.; R, SE_R, a, b, c, INB, SE_INB: intermediate outputs as defined in HE-FM-AN-002 to HE-FM-AN-004.; D_L, D_U: Delta-method limits for the ICER. Unit: currency per unit of effect.; F_L, F_U: Fieller limits, a bounded interval only when a is positive. Unit: currency per unit of effect.; N_L, N_U: Limits for incremental net benefit at lambda. Unit: currency.; z_INB: INB divided by its standard error. Unit: none.; P_pos: Probability that incremental net benefit is positive, the standard normal cumulative distribution function at z_INB, returned by the Excel, R and Python versions. Unit: probability from 0 to 1.

  Assumption: The cost and effect differences are approximately jointly normal and come from the same patients, so their covariance is estimated from the trial, and the threshold is fixed. The three intervals describe sampling uncertainty in one dataset only.

  Worked example (Article trial at £30,000 per QALY): With the article's figures the function returns £400 to £39,600 per QALY for the delta method, about £7,017 to £96,786 for Fieller's method, and about -£1,630 to £3,630 for incremental net benefit, with a probability of about 0.77 that net benefit is positive. Delta_C = 2000; Delta_E = 0.10; SE_C = 600; SE_E = 0.04; Cov_CE = 0; lambda = 30000; z = 1.96; D_L = 400; D_U = 39600; F_L = 7016.89; F_U = 96786.47; N_L = -1629.62; N_U = 3629.62; z_INB = 0.7454

  Worked example (QALY standard error of 0.06): With the QALY standard error raised to 0.06, a is negative, so the function reports Fieller's set as every value below about -£110,622 together with every value above about £6,177. The delta method still returns about -£6,296 to £46,296, and the net benefit interval at £30,000 per QALY widens to about -£2,719 to £4,719 with a probability of about 0.70 that net benefit is positive. Delta_C = 2000; Delta_E = 0.10; SE_C = 600; SE_E = 0.06; Cov_CE = 0; lambda = 30000; z = 1.96; D_L = -6296.16; D_U = 46296.16; N_L = -2718.84; N_U = 4718.84; z_INB = 0.5270

  Excel: =IF(QuadA>0,(-QuadB-SQRT(QuadB^2-4*QuadA*QuadC))/(2*QuadA),IF(QuadB^2-4*QuadA*QuadC>0,"two unbounded pieces","whole line")) Returns the lower Fieller limit from the helper cells QuadA, QuadB and QuadC of HE-FM-AN-003, or a description of the confidence set when QuadA is not positive. The other outputs use the Excel formulas under HE-FM-AN-002 and HE-FM-AN-004.

  R: an_intervals <- function(dC, dE, seC, seE, lambda, covCE=0, z=1.96) { r <- dC/dE; seR <- abs(r)*sqrt(seC^2/dC^2+seE^2/dE^2-2*covCE/(dC*dE)); a <- dE^2-z^2*seE^2; b <- -2*(dC*dE-z^2*covCE); cc <- dC^2-z^2*seC^2; disc <- b^2-4*a*cc; roots <- if (disc >= 0) sort((-b+c(-1,1)*sqrt(disc))/(2*a)) else c(NA,NA); fieller <- if (a > 0) roots else if (disc > 0) c(-Inf,roots[1],roots[2],Inf) else c(-Inf,Inf); inb <- lambda*dE-dC; seN <- sqrt(lambda^2*seE^2+seC^2-2*lambda*covCE); list(delta=r+c(-1,1)*z*seR, fieller=fieller, inb=inb+c(-1,1)*z*seN, p_pos=pnorm(inb/seN)) } Returns the delta-method and net benefit limits, the probability that net benefit is positive and the Fieller set: two limits when it is bounded, four values marking two unbounded pieces, or minus to plus infinity for the whole line.

  Python: def an_intervals(dC, dE, seC, seE, lam, covCE=0.0, z=1.96): r = dC/dE; seR = abs(r)*math.sqrt(seC**2/dC**2+seE**2/dE**2-2*covCE/(dC*dE)); a = dE**2-z**2*seE**2; b = -2*(dC*dE-z**2*covCE); c = dC**2-z**2*seC**2; disc = b*b-4*a*c; roots = sorted([(-b-math.sqrt(disc))/(2*a), (-b+math.sqrt(disc))/(2*a)]) if disc >= 0 else None; inb = lam*dE-dC; seN = math.sqrt(lam**2*seE**2+seC**2-2*lam*covCE); return {'delta': (r-z*seR, r+z*seR), 'fieller': roots if a > 0 else (('outside', roots) if roots else 'whole line'), 'inb': (inb-z*seN, inb+z*seN), 'p_pos': statistics.NormalDist().cdf(inb/seN)} Requires the math and statistics modules; the Fieller entry is the bounded interval, the two roots outside which the set lies, or the whole line.

  Test (Fieller limits are the zero crossings of the net benefit interval): Running the function with lambda set to the lower Fieller limit returns an upper net benefit limit of zero. Expected result: TRUE. Excel check: =ABS(FiellerLower*DeltaEffect-DeltaCost+ZValue*SQRT(FiellerLower^2*SeEffect^2+SeCost^2-2*FiellerLower*CovCostEffect))<1E-6

  Test (A bounded Fieller interval contains the ICER): When a is positive the ICER lies strictly between the two Fieller limits, because the Fieller quadratic is negative at the point estimate. Expected result: TRUE. Excel check: =AND(FiellerLower<DeltaCost/DeltaEffect,DeltaCost/DeltaEffect<FiellerUpper)

  Common error (Passing patient-level standard deviations instead of standard errors): Entering the per-patient standard deviations of £6,708 and 0.447 as SE_C and SE_E makes a and the discriminant negative, so Fieller's set becomes the whole real line, and the net benefit interval at £30,000 per QALY widens to about -£28,389 to £30,389. The function needs the standard errors of the two mean differences.

  Source: Hoch JS, Hay A, Isaranuwatchai W, Thavorn K, Leighl NB, Tu D, Trenaman L, Dewa CS, O'Callaghan C, Pater J, Jonker D, Chen BE, Mittmann N. Advantages of the net benefit regression framework for trial-based economic evaluations of cancer treatments: an example from the Canadian Cancer Trials Group CO.17 trial. BMC Cancer. 2019;19:552. Methods section on reading Fieller limits from the net benefit confidence limits. Polsky D, Glick HA, Willke R, Schulman K. Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Economics. 1997;6(3):243-252. Comparison of the Taylor series and Fieller methods.

  R = Delta_C / Delta_E; SE_R = abs(R) * sqrt(SE_C^2 / Delta_C^2 + SE_E^2 / Delta_E^2 - 2 * Cov_CE / (Delta_C * Delta_E)); D_L = R - z * SE_R; D_U = R + z * SE_R; a = Delta_E^2 - z^2 * SE_E^2; b = -2 * (Delta_C * Delta_E - z^2 * Cov_CE); c = Delta_C^2 - z^2 * SE_C^2; F_L = (-b - sqrt(b^2 - 4 * a * c)) / (2 * a); F_U = (-b + sqrt(b^2 - 4 * a * c)) / (2 * a); INB = lambda * Delta_E - Delta_C; SE_INB = sqrt(lambda^2 * SE_E^2 + SE_C^2 - 2 * lambda * Cov_CE); N_L = INB - z * SE_INB; N_U = INB + z * SE_INB; z_INB = INB / SE_INB

Try this function

Implementations

• Excel

  Wald interval limits from an estimate and its standard error

  With the estimate in a cell named Estimate and its standard error in StdError, the two formulas return the lower and upper limits. NORM.S.INV(0.975) returns 1.959964, which rounds to 1.96.

  =Estimate-NORM.S.INV(0.975)*StdError; =Estimate+NORM.S.INV(0.975)*StdError

Assumptions

• Asymptotically normal estimator at the sample size used

  The sampling distribution of the estimator is close to normal at the sample size available. For a difference in arithmetic means this follows from the central limit theorem whatever the shape of patient-level costs, provided observations are independent with finite variance. A long right tail in a small trial slows the approach to normality.

• Standard error estimated on the right basis for the Wald interval

  The standard error reflects how the data were generated: clustered observations, as in a cluster-randomised trial, need cluster-robust standard errors, and a model-based standard error from a misspecified model is unreliable. The interval describes sampling uncertainty in one dataset only, not the choice of data source, model structure or extrapolation.

Worked examples

• Difference in mean cost of £2,000 with a standard error of £600

  In the article's illustrative trial, 250 patients per arm and a per-patient cost standard deviation of £6,708 in each arm give a standard error of about £600 for the cost difference. The approximate 95% interval for the £2,000 difference runs from £824 to £3,176.

  theta_hat = 2000; SE = 600; z = 1.96; L = 824; U = 3176

• Incremental net benefit of £1,000 with a standard error of about £1,342

  The same rule applied to the article's incremental net benefit at £30,000 per QALY, with a standard error of about £1,342, gives an interval from about -£1,630 to £3,630, as in HE-FM-AN-004.

  theta_hat = 1000; SE = 1341.6408; z = 1.96; L = -1629.62; U = 3629.62

• Zero standard error collapses the Wald interval

  With a standard error of zero both limits equal the estimate, a limiting case that checks the implementation.

  theta_hat = 2000; SE = 0; z = 1.96; L = 2000; U = 2000

Common errors

• Requiring normally distributed patient-level costs

  Rejecting a normal-approximation interval because individual costs are skewed misreads the requirement: the approximation concerns the sampling distribution of the mean difference, not the data. The real concern is a modest trial with a few very expensive patients, which can leave the mean far from its normal limit.

• Using the patient-level standard deviation as the standard error

  Entering the per-patient cost standard deviation of £6,708 in place of the standard error of about £600 gives an interval of about -£11,148 to £15,148 around the £2,000 difference instead of £824 to £3,176, so a clear cost difference appears uncertain even in sign.

Sources

• ISPOR-SMDM task force on normal distributions for well-informed parameters

  Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value in Health. 2012;15(6):835-842. Section on estimation and choice of distribution for PSA and interval estimation, which states that by the central limit theorem a parameter informed by plenty of data can be given a normal distribution in PSA and a standard confidence interval in deterministic sensitivity analysis.

  View source →

• Arithmetic mean costs and normal-theory methods in pragmatic trials

  Thompson SG, Barber JA. How should cost data in pragmatic randomised trials be analysed? BMJ. 2000;320(7243):1197-1200. Argues that the arithmetic mean is the informative measure for cost data in pragmatic trials and that normal-theory methods are fairly robust to non-normality, especially in large samples.

  View source →

Canonical Identity