Signature
NB_all = pi - (1 - pi) * p_t / (1 - p_t)
| Inputs | Definition | Unit |
|---|---|---|
pi | Proportion of patients in the sample who have the event | proportion |
p_t | Threshold at which the line is evaluated, above zero and below 1 | probability |
NB_all | Net true positives per patient when every patient receives the intervention | true positives per patient |
|---|
Function
Decision curve net benefit of a risk model or test across threshold probabilities
Maps a risk model's or test's classifications in a validation sample, at a chosen threshold probability, to net benefit: true positives per patient minus false positives per patient weighted by the odds at the threshold, so that both are counted in units of true positives. Repeating the calculation over a preset range of thresholds, for the model and for intervening in everyone or no one, gives the decision curve. The notation follows the Decision Curve Analysis article; the economic net benefit of an option at a cost-effectiveness threshold is a different quantity (HE-FM-NMB-001).
Try this function
Implementations
Excel
Net benefit of intervening in everyone from named prevalence
With the prevalence in a cell named Prevalence and the threshold in Threshold, the formula returns the net benefit of intervening in everyone, held in NetBenefitAll.
=Prevalence-(1-Prevalence)*Threshold/(1-Threshold)
Assumptions
Intervene-in-all prevalence taken from the same validation sample
pi is the event rate in the sample used for the model's curve, so the defaults and the model are compared on the same patients.
Intervening in everyone uses the threshold only as an exchange rate
Kerr and colleagues note that the treat-all and treat-none policies use no individual risks, but the threshold is still used to value false positives against true positives.
Worked examples
Intervening in all 100 patients at 15 per cent
With 26 admissions among 100 patients the net benefit of intervening in everyone at 15 per cent is about 0.1294, the same as model B, which classes everyone positive.
pi = 0.26; p_t = 0.15; NB_all = 0.1294
Intervening in all 100 patients at 40 per cent
At 40 per cent the false positive weight is about 0.6667 and the net benefit of intervening in everyone is about minus 0.2333, as in the article's table.
pi = 0.26; p_t = 0.4; NB_all = -0.2333
Net benefit of removing the seminal vesicles in every man at 10 per cent
Vickers and Elkin report that with 87 of 902 men having seminal vesicle invasion, intervening in all at 10 per cent gives minus 0.0039, below the model's 0.0443.
pi = 0.096452; p_t = 0.1; NB_all = -0.0039
Common errors
Comparing a model only with intervening in no one
A positive net benefit shows only that a model beats no intervention. At 15 per cent model B's 0.1294 is positive but adds nothing over intervening in everyone.
Using a prevalence from another population for the intervene-in-all line
The defaults must use the validation sample's prevalence; a population figure moves the crossing point and can make a model appear to beat a default it does not beat.
Sources
Treat-all net benefit and its crossing with treat none at the prevalence
Kerr KF, Brown MD, Zhu K, Janes H. Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use. Journal of Clinical Oncology. 2016;34(21):2534-2540. doi:10.1200/JCO.2015.65.5654. Section on decision curves: under the treat-all policy TPR = FPR = 1, so NB = P minus R/(1 minus R) times (1 minus P); equating it with the treat-none NB of 0 shows the two curves cross at R = P; treat all and treat none use no individual risks but R still values the relative size of benefit and cost.
Treat-all and treat-none comparisons in the original decision curve paper
Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making. 2006;26(6):565-574. doi:10.1177/0272989X06295361. Application section: the true- and false-positive counts for treating all patients are the numbers with and without the disease, giving (87/902) minus (815/902) times (0.1/0.9) = minus 0.0039 for removing seminal vesicles in all men; treating none gives 0.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0