Per-person EVPI as the mean opportunity loss of the current decision

Signature

L_s = NBmax_s - NBcur_s; EVPI = sum_(s=1)^S [L_s] / S

Function

Expected value of perfect information function for a single adoption decision

Computational function

• Computational function for EVPI, error probability and population EVPI from a three-option table of draws

  Takes a table of net benefit for three options over five probabilistic draws, the size of each year's patient cohort, the number of years the decision stays relevant, the discount rate and the threshold, and returns per-person EVPI, the error probability and population EVPI in health and money units. It applies HE-FM-EVPI-001 across the draws and then scales the result by the discounted population as in HE-FM-VOI-002, with the first cohort undiscounted. The Math / Code line takes option A as the zero reference, and the R and Python versions take any number of draws and options.

  Inputs and outputs: B_1 to B_5 and C_1 to C_5 are the net benefits of options B and C relative to option A in each draw, required, in QALYs per person; I is the number of patients facing the decision each year, required, above zero; Y is the number of years the decision stays relevant, a whole number; r is the annual discount rate, required, above zero, as a proportion; lambda is the threshold in currency per QALY; M_1 to M_5 are the best net benefits in each draw; EVPI is the per-person EVPI in QALYs; N_disc is the discounted number of patients; EVPI_pop is population EVPI in QALYs and EVPI_popM its value in currency. The R and Python versions also return the error probability P_err.

  Assumption: The draws are equally likely, the option with the highest mean net benefit is chosen on current evidence, the same number of patients faces the decision in each year, counted at the start of each year, and per-person EVPI applies unchanged to every cohort.

  Worked example (Three options at GBP 25,000 per QALY over 10 years): The article's drug, with INHB values 0.10, minus 0.20, 0.30, minus 0.10 and 0.25 QALYs, is joined by an illustrative lower-dose regimen with 0.05, 0.05, 0.10, 0 and 0.10. The drug keeps the highest mean, 0.07 against 0.06, the mean of the best values is 0.14, and EVPI is 0.07 QALYs per patient, with an error probability of 0.4. For 2,000 patients a year over 10 years at 3.5% the discounted population is about 17,215, population EVPI about 1,205 QALYs and its money value about GBP 30.1 million. B_1 = 0.10; B_2 = -0.20; B_3 = 0.30; B_4 = -0.10; B_5 = 0.25; C_1 = 0.05; C_2 = 0.05; C_3 = 0.10; C_4 = 0; C_5 = 0.10; I = 2000; Y = 10; r = 0.035; lambda = 25000; M_1 = 0.10; M_2 = 0.05; M_3 = 0.30; M_4 = 0; M_5 = 0.25; EVPI = 0.07; N_disc = 17215.37; EVPI_pop = 1205.08; EVPI_popM = 30126902.78

  Worked example (Dominated third option reproduces the article): Giving option C a net benefit of minus 1 QALY in every draw leaves the two-option problem of the article, with EVPI of 0.06 QALYs per patient, population EVPI of about 1,033 QALYs and about GBP 25.8 million, a limiting case that checks the third option is handled correctly. B_1 = 0.10; B_2 = -0.20; B_3 = 0.30; B_4 = -0.10; B_5 = 0.25; C_1 = -1; C_2 = -1; C_3 = -1; C_4 = -1; C_5 = -1; I = 2000; Y = 10; r = 0.035; lambda = 25000; M_1 = 0.10; M_2 = 0; M_3 = 0.30; M_4 = 0; M_5 = 0.25; EVPI = 0.06; N_disc = 17215.37; EVPI_pop = 1032.92; EVPI_popM = 25823059.53

  Excel: =SUMPRODUCT(RowMax-INDEX(NB,0,MATCH(MAX(ColMeans),ColMeans,0)))/ROWS(NB) returns per-person EVPI from a table named NB with one column per option, a helper column RowMax and a helper row ColMeans. =SUMPRODUCT(--(RowMax>INDEX(NB,0,MATCH(MAX(ColMeans),ColMeans,0))))/ROWS(NB) returns the error probability, and =EVPI*Patients*(1-(1+Rate)^-Years)*(1+Rate)/Rate population EVPI with cells named EVPI, Patients, Rate and Years.

  R: evpi_table <- function(NB, I, Y, r, lambda) { d <- which.max(colMeans(NB)); L <- apply(NB, 1, max)-NB[, d]; N <- I*sum((1+r)^-(0:(Y-1))); c(EVPI = mean(L), P_err = mean(L > 0), EVPI_pop = mean(L)*N, EVPI_popM = lambda*mean(L)*N) } Takes a matrix with one row per draw and one column per option.

  Python: def evpi_table(NB, I, Y, r, lam): m = [sum(c)/len(NB) for c in zip(*NB)]; d = m.index(max(m)); L = [max(row)-row[d] for row in NB]; N = I*sum((1+r)**-t for t in range(Y)); e = sum(L)/len(L); return e, sum(x > 0 for x in L)/len(L), e*N, lam*e*N Takes a list of rows, one per draw, each with one net benefit per option.

  Test (Opportunity-loss and maxima forms agree): For any table the mean loss of the highest-mean option equals the mean of the row maxima less the largest column mean. Expected result TRUE. Excel check =ABS(SUMPRODUCT(RowMax-INDEX(NB,0,MATCH(MAX(ColMeans),ColMeans,0)))/ROWS(NB)-(AVERAGE(RowMax)-MAX(ColMeans)))<1E-9

  Test (Article's population EVPI reproduced): Per-person EVPI of 0.06 QALYs for 2,000 patients a year over 10 years at 3.5% gives 1,033 QALYs when rounded to the nearest QALY. Expected result TRUE. Excel check =ROUND(0.06*2000*(1-1.035^-10)*1.035/0.035,0)=1033

  Common error (Discounting the first cohort): Counting the 10 cohorts as falling in years 1 to 10 rather than 0 to 9 gives about 16,633 discounted patients and about 998 QALYs of population EVPI instead of 1,033, about 3.4% lower. The timing convention is stated and matched to the horizon.

  Source: Rothery C, Strong M, Koffijberg HE, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of information analytical methods. Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(3):277-286. Section on EVPI, Algorithm 1, which computes EVPI from the single set of probabilistic samples, and section on population VOI, equation 15, which multiplies per-person VOI by the sum over periods 0 to T of the incidence divided by one plus the discount rate raised to the period.

  M_1 = max(max(0, B_1), C_1); M_2 = max(max(0, B_2), C_2); M_3 = max(max(0, B_3), C_3); M_4 = max(max(0, B_4), C_4); M_5 = max(max(0, B_5), C_5); EVPI = (M_1 + M_2 + M_3 + M_4 + M_5) / 5 - max(max(0, (B_1 + B_2 + B_3 + B_4 + B_5) / 5), (C_1 + C_2 + C_3 + C_4 + C_5) / 5); N_disc = I * (1 - (1+r)^(-Y)) * (1+r) / r; EVPI_pop = EVPI * N_disc; EVPI_popM = lambda * EVPI_pop

Try this function

Implementations

• Excel

  EVPI as mean opportunity loss from a draw by option table in Excel

  With one row per draw and one column per option in a range named NB, a helper column RowMax holding the maximum of each row and a helper row ColMeans holding the mean of each column, INDEX and MATCH pick the column of the option with the highest mean and SUMPRODUCT averages the loss over the draws. The formula needs no array entry.

  =SUMPRODUCT(RowMax-INDEX(NB,0,MATCH(MAX(ColMeans),ColMeans,0)))/ROWS(NB)

Assumptions

• Option chosen on current evidence held fixed across all EVPI draws

  The option d* is the one with the highest mean net benefit over the same draws, and it is held fixed in every draw. NBcur_s is the net benefit of d* in draw s, not that of whichever option happens to be best in that draw, so the loss is never negative.

• EVPI draws taken from the probabilistic analysis without extra sampling

  The draws are equally likely samples from the joint distribution of the uncertain inputs, and each draw holds the net benefit of every option at the same threshold. The ISPOR methods report describes this as a single-loop Monte Carlo scheme that needs no sampling beyond the probabilistic analysis. A deterministic run at mean input values cannot replace the draws in a non-linear model.

• Risk-neutral choice by highest expected net benefit in EVPI

  The decision on current evidence is the option with the highest expected net benefit, the choice of a risk-neutral decision maker. A risk-averse decision rule can choose a different option, and the mean loss of that option is then larger than EVPI as defined here.

Worked examples

• Article's five draws at GBP 25,000 per QALY as opportunity losses

  In the article's illustrative example a new drug is compared with standard care, whose net benefit is set to zero, and the drug's incremental net health benefit in the five draws is 0.10, minus 0.20, 0.30, minus 0.10 and 0.25 QALYs. The drug has the higher mean, 0.07 QALYs, so it is chosen, and it loses 0.20 and 0.10 QALYs in draws 2 and 4 where standard care is better. The mean loss is 0.30 / 5 = 0.06 QALYs per patient, the same EVPI as the mean of the maxima less the maximum of the means. This example is for reading, as its draw-by-draw lists may not run in the site calculator.

  S = 5; NBmax_s = [0.10,0,0.30,0,0.25]; NBcur_s = [0.10,-0.20,0.30,-0.10,0.25]; L_s = [0,0.20,0,0.10,0]; EVPI = 0.06

• Article's five draws at GBP 35,000 per QALY as opportunity losses

  At GBP 35,000 per QALY the incremental cost of GBP 10,000 is worth about 0.2857 QALYs, so the drug's net benefit in the five draws is about 0.2143, minus 0.0857, 0.4143, 0.0143 and 0.3643 QALYs. Only draw 2 favours standard care, so the only loss is about 0.0857 QALYs and EVPI is about 0.0171 QALYs per patient, as in the article. This example is for reading, as its draw-by-draw lists may not run in the site calculator.

  S = 5; NBmax_s = [0.2143,0,0.4143,0.0143,0.3643]; NBcur_s = [0.2143,-0.0857,0.4143,0.0143,0.3643]; L_s = [0,0.0857,0,0,0]; EVPI = 0.0171

• Uncertain net benefit with no decision uncertainty gives zero EVPI

  In three illustrative draws the drug's net benefit lies between 0.10 and 0.30 QALYs and so is positive in every draw, so the drug is the best option in each draw and every loss is zero. This limiting case shows that uncertainty which never changes the preferred option has no value of information. This example is for reading, as its draw-by-draw lists may not run in the site calculator.

  S = 3; NBmax_s = [0.10,0.30,0.25]; NBcur_s = [0.10,0.30,0.25]; L_s = [0,0,0]; EVPI = 0

Common errors

• Holding an option other than the highest-mean option fixed in the loss

  If standard care is held fixed in the article's example although the drug has the higher mean, the mean loss is 0.65 / 5 = 0.13 QALYs, which is the EVPI of 0.06 plus the 0.07 QALYs of expected net benefit given up by the wrong current choice. The fixed option must be the one with the highest mean over the same draws.

• Computing EVPI from a single deterministic run

  With one set of inputs there is one best option and no loss, so an EVPI computed from a base case is zero by construction, whatever the true uncertainty. EVPI needs the full set of probabilistic draws.

Sources

• ISPOR introduction worked example of EVPI as mean opportunity loss

  Fenwick E, Steuten L, Knies S, Ghabri S, Basu A, Murray JF, Koffijberg HE, Strong M, Sanders Schmidler GD, Rothery C. Value of information analysis for research decisions, an introduction. Report 1 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(2):139-150. Box 4, which computes EVPI in net health benefit as the mean of the maximum NHB over iterations less the mean NHB of the option chosen on current evidence, and alternatively as the opportunity loss of the current decision in each iteration (column 7) averaged over all iterations, with no loss in the iterations where the current choice is correct.

  View source →

• ISPOR methods report on the EVPI definition and single-loop computation

  Rothery C, Strong M, Koffijberg HE, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of information analytical methods. Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(3):277-286. Section on the expected value of perfect information, which defines EVPI as the expected value of a decision with perfect information less the expected value of the decision on current knowledge, approximates the first term by the mean over samples of the maximum utility, and describes the calculation as a single-loop Monte Carlo scheme that needs no sampling beyond the probabilistic analysis.

  View source →

Canonical Identity