Signature
P_pos = p * q + (1 - p) * (1 - q); pF_pos = p * q / P_pos; pF_neg = p * (1 - q) / (1 - P_pos); B_pos = pF_pos * NB_BF + (1 - pF_pos) * NB_BU; B_neg = pF_neg * NB_BF + (1 - pF_neg) * NB_BU; V_S = P_pos * max(NB_A, B_pos) + (1 - P_pos) * max(NB_A, B_neg); V_0 = max(NB_A, p * NB_BF + (1 - p) * NB_BU); EVSI = V_S - V_0
| Inputs | Definition | Unit |
|---|---|---|
p | Probability on current evidence that the new option B works as hoped (state F) | probability |
q | Probability that the study result points the right way, the same in both states, so a positive result has probability q in state F and 1 minus q in state U | probability |
NB_BF | Net benefit per patient of the new option B in state F, where it works as hoped | currency per patient |
NB_BU | Net benefit per patient of the new option B in state U, where it does not work as hoped | currency per patient |
NB_A | Net benefit per patient of standard care A, the same in both states | currency per patient |
P_pos | Probability on current evidence that the study gives a positive result | probability |
|---|---|---|
pF_pos | Probability of state F after a positive result, by Bayes theorem | probability |
pF_neg | Probability of state F after a negative result, by Bayes theorem | probability |
B_pos | Posterior expected net benefit per patient of option B after a positive result | currency per patient |
B_neg | Posterior expected net benefit per patient of option B after a negative result | currency per patient |
V_S | Expected net benefit per patient of choosing the better option after each result, averaged over the two results | currency per patient |
V_0 | Expected net benefit per patient of the option with the higher expected net benefit on current evidence | currency per patient |
EVSI | Expected value of sample information per patient for the study with a positive or negative result | currency per patient |
Function
Expected value of sample information function for a proposed study design
Maps the current uncertainty about a decision model's parameters, the sampling distribution of the data a proposed study would collect and the net benefit of each option to the expected gain from choosing after the study reports rather than now. EVSI equals the expectation, over the datasets X the study could produce, of the largest posterior expected net benefit among the options, less the largest expected net benefit on current evidence, the form given as equation 10 of the second ISPOR value of information report and equation 2 of Kunst and colleagues. It replaces the perfect knowledge of a parameter subset in EVPPI (HE-FM-VOI-003) with the data of a study of stated design and size, so it is never negative and does not exceed EVPPI for the parameters the study informs or per-person EVPI (HE-FM-VOI-001). This package gives the exact form for a study with two possible results, the normal preposterior variance, closed-form EVSI for two options and population EVSI over the decision's lifetime on the basis used for population EVPI (HE-FM-VOI-002). The expected net benefit of sampling, population EVSI less the cost of the study, has its own package. Notation follows the Expected Value of Sample Information article.
Try this function
Implementations
Excel
EVSI of a two-result study from named cells with LET
With the prior probability of state F in a cell named PriorProb, the accuracy in Accuracy and the three net benefits in NBStandard, NBNewWorks and NBNewFails, the formula returns EVSI per patient in Excel 2021 or Microsoft 365. The helper names pp, fp, fn, bp and bn hold P_pos, pF_pos, pF_neg, B_pos and B_neg.
=LET(pp,PriorProb*Accuracy+(1-PriorProb)*(1-Accuracy),fp,PriorProb*Accuracy/pp,fn,PriorProb*(1-Accuracy)/(1-pp),bp,fp*NBNewWorks+(1-fp)*NBNewFails,bn,fn*NBNewWorks+(1-fn)*NBNewFails,pp*MAX(NBStandard,bp)+(1-pp)*MAX(NBStandard,bn)-MAX(NBStandard,PriorProb*NBNewWorks+(1-PriorProb)*NBNewFails))
Assumptions
Decision maker free to act on or set aside a study result
After each result the option with the higher posterior expected net benefit is chosen, which may be the current choice. Because keeping the current choice is always open, V_S is at least V_0 and EVSI cannot be negative, although a particular result can still lead to a choice that later proves worse.
Risk-neutral choice on expected net benefit for EVSI
Options are ranked by expected net benefit at a stated cost-effectiveness threshold, the risk-neutral rule that Kunst and colleagues state for EVSI. Net benefit can be net monetary or net health benefit, with the three net benefit inputs on the same basis.
Two-result study with the same accuracy in both states
The study is summarised by one positive or negative result whose chance of pointing the right way is q in either state, an illustrative simplification of a study that usually returns a continuous estimate. Different accuracies in the two states change P_pos and the posterior probabilities but not the structure of the calculation.
Costless reversal, full implementation and no delay in per-patient EVSI
Per-patient EVSI here assumes that adoption can be reversed at no cost, that the chosen option is fully implemented and that neither the study nor adoption is delayed, the assumptions Strong and colleagues state for per-person EVSI within one jurisdiction. Departures from them need explicit treatment.
Worked examples
Smaller study S1 whose result points the right way seven times in ten
In the article's illustrative example, A is worth GBP 1,000 per patient, B GBP 1,600 if it works and GBP 600 if not, and state F has a prior probability of 0.45, so B is chosen now at GBP 1,050. Study S1 gives a positive result with probability 0.48, after which B is kept at GBP 1,256.25, and a negative result with probability 0.52, after which B falls to about GBP 859.62 and A is chosen. EVSI is GBP 73 per patient, a third of the EVPI of GBP 220.
p = 0.45; q = 0.7; NB_A = 1000; NB_BF = 1600; NB_BU = 600; P_pos = 0.48; pF_pos = 0.65625; pF_neg = 0.259615; B_pos = 1256.25; B_neg = 859.62; V_S = 1123; V_0 = 1050; EVSI = 73
Larger study S2 whose result points the right way nine times in ten
With q of 0.9 in the same illustrative example, a positive result (probability 0.46) raises the probability of state F to about 0.880 and a negative result (probability 0.54) lowers it to about 0.083, so the decision switches to A after a negative result. The expected net benefit with the study is GBP 1,221 and EVSI is GBP 171 per patient, about 78% of EVPI.
p = 0.45; q = 0.9; NB_A = 1000; NB_BF = 1600; NB_BU = 600; P_pos = 0.46; pF_pos = 0.880435; pF_neg = 0.083333; B_pos = 1480.43; B_neg = 683.33; V_S = 1221; V_0 = 1050; EVSI = 171
Perfectly accurate two-result study recovering EVPI
With q of 1 the result reveals the state, so B is chosen in state F and A in state U, and EVSI equals the EVPI of GBP 220 per patient that HE-FM-VOI-001 gives for the same two states, an upper limiting case that checks the implementation.
p = 0.45; q = 1; NB_A = 1000; NB_BF = 1600; NB_BU = 600; P_pos = 0.45; pF_pos = 1; pF_neg = 0; B_pos = 1600; B_neg = 600; V_S = 1270; V_0 = 1050; EVSI = 220
Uninformative two-result study with a coin-toss result
With q of 0.5 the result is unrelated to the state, both posterior probabilities equal the prior 0.45, B is chosen after either result and EVSI is zero, the lower limiting case.
p = 0.45; q = 0.5; NB_A = 1000; NB_BF = 1600; NB_BU = 600; P_pos = 0.5; pF_pos = 0.45; pF_neg = 0.45; B_pos = 1050; B_neg = 1050; V_S = 1050; V_0 = 1050; EVSI = 0
Common errors
Keeping the current choice after an unfavourable study result
Valuing S1 as if B were kept after every result gives 0.48 times GBP 1,256.25 plus 0.52 times about GBP 859.62, which is the GBP 1,050 of the current decision, so EVSI comes out as zero for any study. The value of S1 lies entirely in switching to A after a negative result, 0.52 times about GBP 140.38, or GBP 73 per patient.
Reading EVSI as a forecast that the study will favour the new option
A high EVSI does not predict a positive result. In the example the positive result of S1 leaves the decision unchanged and adds nothing, and all GBP 73 per patient comes from the negative result that reverses adoption.
Sources
ISPOR task force definition of EVSI and its Monte Carlo schemes
Rothery C, Strong M, Koffijberg HE, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of information analytical methods. Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(3):277-286. Equation 10 defines EVSI as the expectation over the new data X of the maximum over options of the posterior expected utility, less the maximum over options of the expected utility on current information. Algorithms 5 and 6 set out the double-loop and single-loop Monte Carlo schemes, which sample parameters from the prior, generate a plausible dataset, update to the posterior and average the value of the best option across datasets.
Kunst and colleagues on EVSI in net benefit terms under risk neutrality
Kunst N, Wilson ECF, Glynn D, Alarid-Escudero F, Baio G, Brennan A, Fairley M, Goldhaber-Fiebert JD, Jackson C, Jalal H, Menzies NA, Strong M, Thom H, Heath A. Computing the expected value of sample information efficiently. Practical guidance and recommendations for four model-based methods. Value in Health. 2020;23(6):734-742. Equation 2 writes EVSI as the expectation over the data X of the maximum over interventions of posterior expected net benefit, less the maximum of expected net benefit on current evidence, and the section on the decision-analytic model states that, assuming risk neutrality, the intervention with the highest expected net benefit is implemented.
Ades, Lu and Claxton on Monte Carlo EVSI for medical decision models
Ades AE, Lu G, Claxton K. Expected value of sample information calculations in medical decision modeling. Medical Decision Making. 2004;24(2):207-227. The abstract states that the article derives simple and nested Monte Carlo methods that extend EVSI calculations to medical decision applications with multiple sources of uncertainty, with approximate methods for relative measures such as risk differences, odds ratios, risk ratios and hazard ratios.
Strong and colleagues on the reversal, implementation and delay assumptions of per-person EVSI
Strong M, Oakley JE, Brennan A, Breeze P. Estimating the expected value of sample information using the probabilistic sensitivity analysis sample. A fast, nonparametric regression-based method. Medical Decision Making. 2015;35(5):570-583. The introduction lists the costs of delaying or reversing adoption, full implementation of the adoption decision and studies that extend across jurisdictions as factors that affect EVSI, and states that the article computes per-person EVSI within a single jurisdiction under the assumption of costless reversibility with perfect implementation and no delay in either the study or the adoption.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0