Signature
p_t = L / (G + L)
| Inputs | Definition | Unit |
|---|---|---|
L | Net loss from intervening for a patient who would not have the event, above zero | same unit as G |
G | Net value of intervening for a patient who would have the event, above zero | any unit shared with L, such as pounds or QALYs |
p_t | Risk above which intervening has the higher expected value | probability |
|---|
Function
Decision curve net benefit of a risk model or test across threshold probabilities
Maps a risk model's or test's classifications in a validation sample, at a chosen threshold probability, to net benefit: true positives per patient minus false positives per patient weighted by the odds at the threshold, so that both are counted in units of true positives. Repeating the calculation over a preset range of thresholds, for the model and for intervening in everyone or no one, gives the decision curve. The notation follows the Decision Curve Analysis article; the economic net benefit of an option at a cost-effectiveness threshold is a different quantity (HE-FM-NMB-001).
Try this function
Implementations
Excel
Threshold probability from named benefit and harm
With the benefit to a case in BenefitCase and the harm to a non-case in HarmNonCase, the formula returns the threshold, held in ThresholdImplied.
=HarmNonCase/(BenefitCase+HarmNonCase)
Assumptions
Threshold benefit and harm the same for every case and non-case
G and L do not vary with predicted risk; Kerr and colleagues state this as an assumption embedded in decision curve net benefit.
Threshold benefit and harm in a common unit
Only the ratio L / G matters, so both must be measured in the same unit; a ratio elicited directly (such as nine to one) can be used without either value.
Worked examples
Decision curve threshold for an admission-prevention programme valued by an economic model
If a decision model values reaching an admission-prone patient at 1,500 pounds and treating a patient unnecessarily at a loss of 500 pounds, the implied threshold is 500 / 2,000, or 25 per cent, as in the article's illustration.
G = 1500; L = 500; p_t = 0.25
Biopsy threshold when a missed high-grade cancer is nine times worse than an unnecessary biopsy
If missing a high-grade cancer is judged nine times worse than an unnecessary biopsy, so that at most 10 men would be biopsied to find one cancer, the threshold is 1 / 10, or 10 per cent.
G = 9; L = 1; p_t = 0.1
Common errors
Swapping benefit and harm in the threshold probability
Writing p_t = G / (G + L) turns the 25 per cent threshold of the first example into 75 per cent, so a model would be judged at the wrong end of the range.
Choosing a threshold to suit the model
The threshold comes from the benefits and harms of the intervention, not from the curve; picking the threshold where a model looks best overstates its value.
Sources
Rational risk threshold as cost over benefit plus cost
Kerr KF, Brown MD, Zhu K, Janes H. Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use. Journal of Clinical Oncology. 2016;34(21):2534-2540. doi:10.1200/JCO.2015.65.5654. Section on decision curves: a classic result in decision theory says a rationally chosen risk threshold is R = C/(B + C), equivalently R/(1 minus R) = C/B, where B is the expected benefit of intervention for a case and C the expected cost to a control, interpreted broadly (money, time, stress, negative health effects); every case is assumed to have the same B and every control the same C.
Threshold derived from a decision tree with outcome values
Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making. 2006;26(6):565-574. doi:10.1177/0272989X06295361. Introductory theory, Equation 1: p_t a + (1 minus p_t) b = p_t c + (1 minus p_t) d gives (a minus c)/(d minus b) = (1 minus p_t)/p_t, where a, b, c and d are the values of a true positive, false positive, false negative and true negative.
Ten biopsies per cancer found implies a decision curve harm ratio of nine
Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016;352:i6. doi:10.1136/bmj.i6. Net benefit and risk prediction: if no more than 10 men should undergo biopsy to find one man with high grade cancer, the harm of delaying diagnosis is nine times greater than that of an unnecessary biopsy.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0