Signature
avoided = (NB_model - NB_all) * (1 - p_t) / p_t * 100
| Inputs | Definition | Unit |
|---|---|---|
NB_model | Decision curve net benefit of the model (HE-FM-DCA-001) | true positives per patient |
NB_all | Decision curve net benefit of intervening in every patient (HE-FM-DCA-002) | true positives per patient |
p_t | Threshold at which both net benefits are computed, above zero and below 1 | probability |
avoided | Reduction in unnecessary interventions per 100 patients with no more cases missed, compared with intervening in everyone | interventions per 100 patients |
|---|
Function
Decision curve net benefit of a risk model or test across threshold probabilities
Maps a risk model's or test's classifications in a validation sample, at a chosen threshold probability, to net benefit: true positives per patient minus false positives per patient weighted by the odds at the threshold, so that both are counted in units of true positives. Repeating the calculation over a preset range of thresholds, for the model and for intervening in everyone or no one, gives the decision curve. The notation follows the Decision Curve Analysis article; the economic net benefit of an option at a cost-effectiveness threshold is a different quantity (HE-FM-NMB-001).
Try this function
Implementations
Excel
Interventions avoided from named net benefits
With the two net benefits in NetBenefitModel and NetBenefitAll and the threshold in Threshold, the formula returns net interventions avoided per 100, held in AvoidedPer100.
=(NetBenefitModel-NetBenefitAll)*(1-Threshold)/Threshold*100
Assumptions
Both net benefits computed at the same threshold in the same sample
NB_model and NB_all come from the same patients at the same p_t; mixing thresholds or samples breaks the conversion.
Intervening in everyone is the reference practice
The conversion answers how many unnecessary interventions the model would save against intervening in everyone; against intervening in no one the net benefit itself is the comparison.
Worked examples
Model A against intervening in everyone at 15 per cent
Model A's net benefit of 0.1706 against 0.1294 for everyone, times 0.85 / 0.15 and 100, gives about 23.3 interventions avoided per 100: 46 fewer unnecessary interventions less 4 missed admissions weighted by about 5.67 each.
NB_model = 0.170588; NB_all = 0.129412; p_t = 0.15; avoided = 23.33
Model A against intervening in everyone at 40 per cent
At 40 per cent model A avoids 67 unnecessary interventions and misses 13 extra admissions, each weighted 1.5, a net 47.5 per 100 (computed here for illustration).
NB_model = 0.083333; NB_all = -0.233333; p_t = 0.4; avoided = 47.5
Interventions avoided by the seminal vesicle model against removal in every man
With 0.0443 for the model and minus 0.0039 for intervening in all at 10 per cent, the model avoids about 43.5 unnecessary operations per 100 men net of missed cases: 590 fewer false positives less 22 extra false negatives times 9, over 902 men (computed here for illustration from Vickers and Elkin's counts, against 43 in their Table 2).
NB_model = 0.044346; NB_all = -0.003942; p_t = 0.1; avoided = 43.46
Common errors
Multiplying by the false positive weight instead of dividing for interventions avoided
Multiplying the difference by p_t / (1 minus p_t) gives 0.73 per 100 for model A at 15 per cent instead of 23.3.
Reading avoided interventions as gross reductions
The figure is net of extra missed cases; model A at 15 per cent spares 46 unnecessary interventions but misses 4 admissions, and the 23.3 already deducts them.
Sources
Net reduction in false positives per 100 from the net benefit gain over treating all
Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making. 2006;26(6):565-574. doi:10.1177/0272989X06295361. Application section and Table 2 note: at a threshold of 5 per cent a net benefit 0.013 above treating all equals 0.013 x 100/(0.05/0.95) = 25 fewer false-positive results per 100 patients; the reduction is net of false negatives.
Avoided interventions recommended when intervention for all is the reference
Vickers AJ, van Calster B, Steyerberg EW. A simple, step-by-step guide to interpreting decision curve analysis. Diagnostic and Prognostic Research. 2019;3:18. doi:10.1186/s41512-019-0064-7. Step 5: expressing net benefit in terms of avoided unnecessary procedures or treatments is recommended if the reference strategy is intervention for all; doing so does not change conclusions about which model or test has the highest net benefit.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0