Delta Method

Delta Method for Standard Errors of Derived Health Economic Estimates

The delta method gives a standard error for a quantity that is calculated from estimates instead of being estimated directly, by treating the function as a straight line close to the point estimate. In health economics it supplies standard errors for mean survival from a fitted hazard, for transition probabilities converted from rates, for log risk ratios and, in early trial-based work, for the incremental cost-effectiveness ratio. It needs only the estimates, their covariance matrix and the function's derivatives, so it is quick and transparent. Its weakness is that the straight-line view can be poor when the function bends sharply over the plausible range, which is why ratios with an uncertain denominator are usually handled by other methods.

How a first-order expansion carries uncertainty through a function

The Stata FAQ written by Feiveson describes the method as expanding a function of a random variable about its mean, usually with a one-step Taylor approximation, and then taking the variance. Near the estimate the function behaves like its tangent line, so the spread of the output is the spread of the input multiplied by the slope.

$$\operatorname{Var}\left[g(X)\right] \approx \left[g'(\mu)\right]^{2} \operatorname{Var}(X)$$

where $X$ is the estimated input, $\mu$ is its mean, $g$ is the differentiable function that produces the decision quantity and $g'(\mu)$ is its first derivative, evaluated in practice at the point estimate. The same FAQ warns that this is a good approximation only if $X$ has a high probability of being close enough to its mean for the Taylor approximation to hold.

With several inputs the slope becomes a gradient and the input variance becomes a variance-covariance matrix.

$$\operatorname{Var}\left[g(\hat{\theta})\right] \approx \nabla g(\hat{\theta})^{\mathsf{T}} \, \Sigma \, \nabla g(\hat{\theta})$$

where $\hat{\theta}$ is the vector of estimated parameters, $\Sigma$ is their variance-covariance matrix and $\nabla g(\hat{\theta})$ is the vector of partial derivatives of $g$ at the estimates. As Zepeda-Tello and colleagues explain in a tutorial for medical statisticians, a smooth function of an asymptotically normal estimator is itself asymptotically normal, so the delta-method variance yields a Wald-type interval of the estimate plus or minus 1.96 standard errors at the 95% level. This ties the method to asymptotic normality and to maximum likelihood estimation, which returns exactly such a covariance matrix.

Two inputs and the covariance term

For a function of two estimates the general formula expands into three terms. The third carries the covariance, and its sign depends on the sign of the covariance and on the signs of both partial derivatives.

$$\operatorname{Var}\left[g(\hat{X}, \hat{Y})\right] \approx g_X^{2} \operatorname{Var}(\hat{X}) + g_Y^{2} \operatorname{Var}(\hat{Y}) + 2 g_X g_Y \operatorname{Cov}(\hat{X}, \hat{Y})$$

where $g_X$ and $g_Y$ are the partial derivatives of $g$ with respect to each input at the estimates. For the ratio $R = \Delta C / \Delta E$ the derivatives are $g_C = 1/\Delta E$ and $g_E = -\Delta C/\Delta E^{2}$, which gives the ratio-of-means variance set out by Zepeda-Tello and colleagues.

$$\operatorname{Var}(\hat{R}) \approx \frac{\operatorname{Var}(\Delta C)}{\Delta E^{2}} + \frac{\Delta C^{2} \operatorname{Var}(\Delta E)}{\Delta E^{4}} - \frac{2 \Delta C \operatorname{Cov}(\Delta C, \Delta E)}{\Delta E^{3}}$$

where $\Delta C$ and $\Delta E$ are the estimated differences in mean cost and mean effect between two arms. Because $\Delta E$ appears in every denominator, the approximation degrades as the effect difference approaches zero. Trial costs and effects are measured on the same patients, so dropping the covariance term is a substantive assumption.

Where health economic analyses use it

Kearns and colleagues, studying survival extrapolation, note that the delta method can be used whenever the derivative of the function can be calculated, which covers all of the commonly used parametric survival models. For the exponential model they derive mean survival as the reciprocal of the hazard and obtain its variance by the delta method. The same reasoning gives a standard error for a cycle probability, since rate-to-probability conversion is a smooth function of the rate. Kearns and colleagues note that Stata uses the delta method, while the flexsurv package in R uses multivariate normal simulation.

Worked example with an exponential survival model

The numbers in this example are illustrative. A trial arm records 100 deaths over 400 person-years, and an exponential model is fitted. Following Kearns and colleagues, the hazard is the number of events divided by total follow-up time, and its variance is the squared hazard divided by the number of events.

1. Estimate the hazard. The hazard is 100 / 400 = 0.25 per year. Its variance is 0.0625 / 100 = 0.000625, so its standard error is 0.025 per year.

2. Transform to mean survival. Mean survival is 1 / 0.25 = 4 years. The derivative of $\mu = \lambda^{-1}$ is $-\lambda^{-2}$, whose square is $\lambda^{-4}$.

$$\operatorname{Var}(\hat{\mu}) \approx \frac{\operatorname{Var}(\hat{\lambda})}{\hat{\lambda}^{4}} = \frac{0.000625}{0.00390625} = 0.16$$

where $\hat{\mu}$ is estimated mean survival in years and $\hat{\lambda}$ is the estimated annual hazard. The standard error is 0.4 years, and the Wald interval is 4 ± 1.96 × 0.4, or 3.216 to 4.784 years.

3. Repeat on the log scale. Since $\log \mu = -\log \lambda$, the standard error of the log hazard is 0.025 / 0.25 = 0.1. The interval for log mean survival is log 4 ± 0.196, and exponentiating gives 4 × 0.8220 = 3.288 to 4 × 1.2165 = 4.866 years. The interval is asymmetric, reflecting the skew of a reciprocal. Simulating 200,000 normal draws of the hazard gives 2.5th and 97.5th percentiles of about 3.35 and 4.98 years, closer to the log-scale interval.

4. Convert to a one-year probability. The annual probability of death is 1 − e^(−0.25) = 0.2212. The derivative of $p = 1 - e^{-\lambda}$ is $e^{-\lambda}$, here 0.7788, so the standard error is 0.7788 × 0.025 = 0.0195.

5. Interpretation. With 100 events the methods broadly agree, yet the choice of scale still moves the upper limit by about 0.08 years, and the gap widens with fewer events.

Worked example with an incremental cost-effectiveness ratio

The numbers in this example are illustrative. A trial estimates an incremental cost of GBP 2,000 (standard error GBP 500) and an incremental effect of 0.10 QALYs (standard error 0.04), with a correlation of 0.25, so the covariance is 0.25 × 500 × 0.04 = 5. The ICER is GBP 20,000 per QALY.

1. Apply the delta-method variance. The three terms are 250,000 / 0.01 = 25,000,000, then 4,000,000 × 0.0016 / 0.0001 = 64,000,000, then 2 × 2,000 × 5 / 0.001 = 20,000,000.

$$\operatorname{Var}(\hat{R}) \approx 25{,}000{,}000 + 64{,}000{,}000 - 20{,}000{,}000 = 69{,}000{,}000$$

where $\hat{R}$ is the estimated ICER in GBP per QALY. The standard error is about GBP 8,307 and the 95% interval is GBP 3,719 to GBP 36,281 per QALY.

2. Compare with Fieller's interval. The Fieller method, applied to ICERs by Willan and O'Brien, finds the values of $R$ for which $\Delta C - R\,\Delta E$ is not significantly different from zero. Setting the test statistic equal to 1.96 and squaring gives a quadratic in $R$.

$$(2000 - 0.1R)^{2} = 3.8416\,(250000 - 10R + 0.0016R^{2})$$

where 3.8416 is 1.96 squared and the bracket on the right is the variance of $\Delta C - R\,\Delta E$. Collecting terms gives a quadratic with coefficients 0.00385344, −361.584 and 3,039,600, whose roots are about GBP 9,335 and GBP 84,499 per QALY.

3. Check against net benefit. At an illustrative threshold of GBP 25,000 per QALY, incremental net monetary benefit is 25,000 × 0.10 − 2,000 = GBP 500. Because INMB is linear, its variance is exactly 625,000,000 × 0.0016 + 250,000 − 2 × 25,000 × 5 = 1,000,000, so the standard error is GBP 1,000 and the interval runs from GBP −1,460 to GBP 2,460. The interval includes zero and GBP 25,000 lies inside the Fieller set, as Heitjan showed must happen.

4. Interpretation. The delta-method interval is symmetric around GBP 20,000 and its upper limit is less than half of Fieller's. Polsky and colleagues found the same pattern in a Monte Carlo comparison, with Taylor series intervals that asymmetrically underestimated the upper limit.

Conditions that decide whether a delta-method interval holds

An analyst should check a few conditions before reporting a delta-method interval. Each can move the result.

1. Differentiability. Zepeda-Tello and colleagues note that the method does not work for non-differentiable functions such as indicators, maximums and absolute values. A quantity defined by choosing the option with the highest expected net benefit involves a maximum, which has no derivative where two options tie.
2. Closeness to the estimate. A large standard error relative to the estimate, or strong curvature in the function, breaks the tangent-line view.
3. Scale. Zepeda-Tello and colleagues build a risk ratio interval on the log scale and exponentiate the limits, and the log or logit scale likewise keeps intervals for rates and probabilities within their natural range.
4. Covariance. Costs and effects on the same patients, or correlated coefficients from one regression, need the full covariance matrix. Assuming independence can shrink or inflate the variance, depending on the signs of the derivatives and of the covariance.
5. Sample size. The normal limit is a large-sample result, so small trials and rare events weaken it.

Interpretation in economic evaluation and HTA

A delta-method standard error carries sampling uncertainty in the inputs through one function and says nothing about structural choices or bias. In a decision model with many parameters, probabilistic sensitivity analysis propagates parameter uncertainty by simulation and needs no derivatives.

For cost-effectiveness ratios the evidence points away from the delta method. Polsky and colleagues found nonparametric bootstrap and Fieller intervals more dependably accurate than Taylor series or box intervals. Briggs, Mooney and Wonderling found that methods assuming a normal sampling distribution performed poorly and advised against normal approximation where the sampling distribution is suspected to be skewed. Heitjan showed that the thresholds at which net health benefit does not differ significantly from zero form exactly the Fieller confidence set, so testing net benefit across thresholds conveys the same information without dividing by an uncertain effect. The joint estimates on the cost-effectiveness plane show that uncertainty without forming a ratio.

The method still serves well for smooth, well-estimated transformations inside models. Kearns and colleagues report that multivariate normal simulation gives similar estimates of uncertainty and is easier to implement because it avoids derivatives.

Boundaries with related methods

In the cost-effectiveness literature the same first-order calculation is often called the Taylor series method, the name Polsky and colleagues use. The Taylor series expansion is the underlying mathematics.

The Fieller method gives a ratio interval that can be asymmetric or unbounded, and so keeps the denominator uncertainty that the delta method smooths away. Bootstrapping resamples patients and recomputes the statistic, capturing skew without derivatives but needing the original data.

Misreadings of delta-method results

A serious error is to trust a delta-method interval for an ICER when the effect difference is small relative to its standard error, since the ratio can then change sign and the Fieller set may be unbounded. Zepeda-Tello and colleagues also show that a differentiable function can still defeat the method, with an attributable fraction whose derivative at zero is zero, so the first-order approximation is very poor near that point.

A second error is to forget that the method approximates variance only. It does not correct any bias that a nonlinear transformation introduces into the point estimate. A third is to quote a standard error without the scale on which it was computed, since natural-scale and log-scale intervals can differ noticeably in small samples.

Sources

• Briggs AH, Mooney CZ, Wonderling DE. Constructing confidence intervals for cost-effectiveness ratios: an evaluation of parametric and non-parametric techniques using Monte Carlo simulation. Statistics in Medicine. 1999;18(23):3245-3262. https://doi.org/10.1002/(SICI)1097-0258(19991215)18:23%3C3263::AID-SIM315%3E3.0.CO;2-3<3245::AID-SIM314>3.0.CO;2-2
• Feiveson AH. Explanation of the delta method. Stata FAQ. College Station, TX: StataCorp; 1999, updated June 2017. https://www.stata.com/support/faqs/statistics/delta-method/
• Heitjan DF. Fieller's method and net health benefits. Health Economics. 2000;9(4):327-335. https://doi.org/10.1002/1099-1050(200006)9:4%3C327::AID-HEC517%3E3.0.CO;2-S
• Kearns B, Stevens J, Ren S, Brennan A. How uncertain is the survival extrapolation? A study of the impact of different parametric survival models on extrapolated uncertainty about hazard functions, lifetime mean survival and cost effectiveness. PharmacoEconomics. 2020;38(2):193-204. https://doi.org/10.1007/s40273-019-00853-x
• Polsky D, Glick HA, Willke R, Schulman K. Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Economics. 1997;6(3):243-252. https://doi.org/10.1002/(SICI)1099-1050(199705)6:3%3C253::AID-HEC266%3E3.0.CO;2-6<243::AID-HEC269>3.0.CO;2-Z
• Willan AR, O'Brien BJ. Confidence intervals for cost-effectiveness ratios: an application of Fieller's theorem. Health Economics. 1996;5(4):297-305. https://doi.org/10.1002/(SICI)1099-1050(199607)5:4%3C297::AID-HEC216%3E3.0.CO;2-T
• Zepeda-Tello R, Schomaker M, Maringe C, Smith MJ, Belot A, Rachet B, Schnitzer ME, Luque-Fernandez MA. The delta-method and influence function in medical statistics: a reproducible tutorial. arXiv preprint 2206.15310; 2022. https://arxiv.org/abs/2206.15310