Net Benefit Approach

Net Benefit Approach and the Move Away from Cost-Effectiveness Ratios

The net benefit approach replaces the cost-per-QALY ratio as the statistic on which a cost-effectiveness decision turns. Each option's health gain is priced at the cost-effectiveness threshold, its cost is subtracted, and the option with the largest expected result is preferred. It underlies the way a modern analysis averages simulation results, compares many options at once and puts a value on further research. In the 1990s the incremental cost-effectiveness ratio ran into trouble once analysts tried to describe its uncertainty, and rewriting the threshold rule as a difference removed those problems.

The formulas have their own pages, with net benefit giving the overview and net monetary benefit and net health benefit giving the money and health forms.

Origins in a rearranged inequality

The algebra is older than the approach. Both Claxton and Stinnett and Mullahy trace the rearrangement of the cost-effectiveness decision rule into a net benefit to a 1991 paper by Phelps and Mushlin, who cited it as evidence that cost-effectiveness and cost-benefit analysis are nearly equivalent but did not propose it for practical use. What changed in the mid-1990s was patient-level cost and effect data from trials and observational studies, and with it a literature on confidence intervals for the ratio built on the delta method, Fieller's theorem and bootstrapping. Three lines of work then turned the rearranged rule into a method.

• Claxton and Posnett (1996) set out a decision-analytic approach to trial design and research priority-setting that took explicit account of the costs of sampling, the benefits of sample information and the decision rules of cost-effectiveness analysis. Claxton later described it as using incremental net benefit within a Bayesian framework.
• Stinnett and Mullahy (1998) named the "net health benefits" approach and offered it as an alternative to confidence intervals for ratios, one that needed no more data than ratio analysis.
• Tambour, Zethraeus and Johannesson (1998) multiplied effects by a price per unit of effectiveness so that standard statistical techniques could give a confidence interval for net benefit in money.

What went wrong with ratio-based decision rules

Four of Stinnett and Mullahy's objections to inference based on the ratio show how it can mislead even when the costs and effects behind it are estimated correctly. The first three concern a single comparison and the fourth concerns several options.

• Sign ambiguity. A negative ratio arises when the new option is cheaper and more effective, and also when it is dearer and less effective. A positive ratio below the threshold favours an option that costs more and gains health, but counts against one that costs less and loses health. A ratio estimator's distribution carries no record of the quadrant of the cost-effectiveness plane, so its confidence interval is ambiguous when results span quadrants.
• Meaningless magnitudes. Their example was a folic acid fortification analysis reporting a ratio of minus USD 13,000 per QALY. Saving USD 13,000 per QALY beats saving USD 6,500 if the larger figure comes from bigger savings, and loses if it comes from a smaller health gain.
• Averaging. The mean of a set of ratio estimates generally differs from the ratio of mean incremental cost to mean incremental effect, and if incremental costs and effects were independent unit normal variables the ratio would follow a Cauchy distribution, with no mean and infinite variance. The mean of net health benefit equals its value at mean costs and effects.
• Many options. Incremental ratios need dominated options removed and the rest ranked by cost, and in a bootstrap the ranking and comparator can change between replicates. Incremental net health benefit is the difference between two options' values, so the probability that an option is best is the share of replicates in which it has the highest value.

Behind them lie two properties. Net benefit is linear in costs and effects, and preferences rise steadily with it, while the direction in which a ratio moves when cost changes depends on the sign of the health difference.

The decision rule the approach puts in its place

At a stated threshold, a new option is worth adopting against its comparator when its incremental net benefit is positive, and among mutually exclusive options the one with the highest expected net benefit is chosen. No sequence of pairwise comparisons is needed.

Claxton argued that the choice should rest on mean net benefit alone, whatever its statistical significance, with the distribution of net benefit relevant only to whether more information should be acquired.

The approach also states opportunity cost openly. Stinnett and Mullahy read net health benefit as the health gained by funding an option instead of a marginally cost-effective programme, with the threshold as the reciprocal of the shadow price of the budget constraint, so a negative value means more health could be bought elsewhere.

Worked example with three options and four probabilistic draws

All numbers in this example are illustrative. A decision maker compares current care (A) with options B and C at £20,000 per QALY. Costs and QALYs are measured relative to A, so A's net monetary benefit (NMB) is zero in every draw, and four equally likely draws stand in for the thousands of a real probabilistic sensitivity analysis.

For B in draw 3, NMB is 20,000 × 0.09 − 1,200 = 600.

Averaging ratios misleads (step 1). B's ratios against A are £10,000, £12,500, £13,333 and minus £50,000 per QALY, the last from draw 4, where B costs more and loses health. Their mean, minus £3,542, reads as a saving although B is never cheaper than A. C's ratios of £30,000, £25,000, £21,429 and £7,500 average £20,982, above the threshold, wrongly suggesting C is not worth adopting.

Choose by expected NMB (step 2). Averaging net benefit across draws gives each option's expected value, shown here for C.

$$\overline{\text{NMB}}_C = \frac{1}{S}\sum_{s=1}^{S} \left( \lambda \Delta E_{C,s} - \Delta C_{C,s} \right) = \frac{3{,}200}{4} = 800$$

where $\overline{\text{NMB}}_C$ is C's expected net monetary benefit against A in pounds, $S = 4$ is the number of draws, $\lambda$ is £20,000 per QALY, $\Delta E_{C,s}$ and $\Delta C_{C,s}$ are C's extra QALYs and cost in draw $s$, and 3,200 is the sum of C's four NMB values. B's expected NMB is 800 / 4 = 200, so C (£800) is chosen over B (£200) and A (£0). Mean values agree, since C's £3,000 and 0.19 QALYs give about £15,789 per QALY, and B is extendedly dominated, since the step from B to C (1,950 / 0.1275, about £15,294 per QALY) is cheaper than the step from A to B (£16,800 per QALY).

Most often best is not best on average (step 3). B has the highest NMB in three of four draws and C in one. Reading the cost-effectiveness acceptability curve as a ranking would pick B, yet C wins on expected value because its good draw is large. The cost-effectiveness acceptability frontier at £20,000 per QALY shows 0.25, the probability for C.

The value of removing the uncertainty (step 4). With perfect information the best option would be chosen in each draw, and the expected gain from that is the EVPI.

$$\text{EVPI} = \frac{1}{S}\sum_{s=1}^{S} \max_{j} \text{NMB}_{j,s} - \max_{j} \overline{\text{NMB}}_j = 1{,}800 - 800 = 1{,}000$$

where EVPI is the expected value of perfect information per person in pounds, $\text{NMB}_{j,s}$ is option $j$'s NMB in draw $s$, 1,800 is the mean of 1,000, 600, 600 and 5,000, and 800 is C's expected NMB.

Interpretation (step 5). At £20,000 per QALY the framework selects C despite its 0.25 chance of being best, with £1,000 per person at stake from the remaining uncertainty. At £10,000 per QALY the expected NMB is minus £425 for B and minus £1,100 for C, so current care would be kept.

The framework in current practice

Once results are expressed as net benefit at a given threshold, most outputs of a modern evaluation follow from the same calculation. Each item below rests on it.

• Acceptability curves. Stinnett and Mullahy noted that the share of replicates with positive net health benefit, plotted against the threshold, reproduces the acceptability curve of van Hout and colleagues.
• The acceptability frontier. Fenwick, Claxton and Sculpher proposed a frontier as a concept more relevant to decision making, and the NICE manual (PMG36, section 4.7.15) asks for it to be shown and explained as the probability that the option with the highest expected net benefit is cost-effective.
• Value of information. Claxton described the expected value of perfect information as the expected opportunity loss of deciding on current evidence, and the expected net benefit of sampling as the value of sample information minus the cost of the sample, maximised to set the size of a trial.
• Regression on trial data. Hoch, Briggs and Willan proposed net-benefit regression, which brings in established econometric techniques, adjustment for imperfect randomisation and subgroup analysis. The quantity it models is covered on the incremental net benefit page.
• Health technology assessment. The NICE manual (PMG36, section 4.2.16) asks for expected net health benefits alongside ICERs at £25,000 and £35,000 per QALY.

Assumptions and limits

Every net benefit result is conditional on the threshold, which is the main limit of the approach. Stinnett and Mullahy recommended reporting net benefit as a function of the threshold over a range of values, argued that this explicitness is an advantage because a ratio cannot be judged without a threshold either, and advised keeping any probability distribution for the threshold out of the primary analysis.

• Marginal change. Reading the threshold as a shadow price assumes the option is small; Stinnett and Mullahy note it should not depend on one programme unless that programme uses a large share of resources. Affordability belongs to budget impact analysis.
• Attitude to risk. Choosing by the mean treats the decision maker as risk neutral. Stinnett and Mullahy noted that where one option stochastically dominates another, a risk-averse decision maker should prefer it, a check that ratios cannot support. Claxton noted that risk attitudes and equity concerns can be built into the outcome measure.
• Cost-effectiveness assumptions. Net health benefit keeps the assumptions of cost-effectiveness analysis, including equal value for every QALY, although its form resembles the net benefit of cost-benefit analysis, where health is valued by individual willingness to pay.

Common misreadings of the net benefit approach

Most errors carry habits from ratio analysis into the framework or read probability results as a decision rule. The worked example shows the first two.

• Averaging ICERs across simulations instead of averaging net benefit.
• Choosing the option most often best instead of the one with the highest expected net benefit.
• Rejecting an option because its incremental net benefit is not statistically significant, when on Claxton's argument significance bears only on whether to collect more evidence.
• Treating the approach as cost-benefit analysis, or reporting a net benefit figure without its threshold.

Sources

• Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Medical Decision Making. 1998;18(2 Suppl):S68-S80. doi:10.1177/0272989X98018002S09. Also issued as NBER Technical Working Paper 227, 1998.
• Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Economics. 1996;5(6):513-524. doi:10.1002/(SICI)1099-1050(199611)5:6<513::AID-HEC237>3.0.CO;2-9
• Tambour M, Zethraeus N, Johannesson M. A note on confidence intervals in cost-effectiveness analysis. International Journal of Technology Assessment in Health Care. 1998;14(3):467-471. doi:10.1017/S0266462300011442
• Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. Journal of Health Economics. 1999;18(3):341-364. doi:10.1016/S0167-6296(98)00039-3. Earlier version: CHE Technical Paper 8, University of York, 1997.
• Fenwick E, Claxton K, Sculpher M. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Economics. 2001;10(8):779-787. doi:10.1002/hec.635
• Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Economics. 2002;11(5):415-430. doi:10.1002/hec.678
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 4.2.16 and 4.7.15.