Closed-form EVSI for two options with normal incremental net benefit

Signature

s = sqrt(s2); z = abs(m0) / s; phi_z = exp(-z * z / 2) / sqrt(2 * 3.14159265); EVSI = s * (phi_z - z * Q_z)

Function

Expected value of sample information function for a proposed study design

Computational function

• Computational function for per-person and population EVSI of a two-arm trial under normal incremental net benefit

  Takes the inputs a trial planner usually holds, the prior mean and standard deviation of incremental net benefit from the probabilistic analysis, the per-patient standard deviation of net benefit in each arm, the number of patients per arm and the size and lifetime of the beneficiary population, and returns per-person EVSI, per-person EVPI under normality, the share of EVPI the trial captures and population EVSI. It chains HE-FM-EVSI-002 (preposterior variance), HE-FM-EVSI-003 (closed-form EVSI, applied also at the prior variance for EVPI) and HE-FM-EVSI-004 (population EVSI), so its inputs differ from the symbols of any one formula. The calculator version takes the two upper-tail probabilities Q_s and Q_0 as inputs because it has no normal distribution function, and the Excel, R and Python versions compute them. The same EVSI can be checked by the single-loop Monte Carlo scheme of the second ISPOR report, drawing a true incremental net benefit from the prior and a trial estimate around it, and averaging the amount by which the posterior mean falls on the other side of zero from the prior mean, the value of switching in those results.

  Inputs and outputs: m0 is the prior mean of incremental net benefit, required, in currency per patient; sd0 its prior standard deviation, required, above zero; sd_A and sd_B are the per-patient standard deviations of net benefit in the two arms, required, above zero; n is the number of patients per arm, above zero; N is the number of patients a year whose treatment the result informs; Y is the decision lifetime in whole years; r is the annual discount rate, above zero, as a proportion; Q_s and Q_0 are the standard normal upper-tail probabilities at z_s and z_0, calculator inputs only; v0 is the prior variance; sigma2 the per-patient sampling variance of the trial's estimate; n0 the prior effective sample size per arm; nu the variance fraction; s2 and s the preposterior variance and standard deviation; z_s, phi_s, z_0 and phi_0 the standardised prior means and normal densities at the preposterior and prior standard deviations; EVSI and EVPI are in currency per patient; share is EVSI divided by EVPI; D is the discounted number of years of beneficiaries; EVSI_pop is population EVSI in currency.

  Assumption: Incremental net benefit is normal a priori and the trial estimates it with normal error, the two arms are independent with equal numbers, and the choice is between the two arms' options. The trial informs incremental net benefit directly; a trial that informs only some parameters needs the variance explained by them, and its EVSI is bounded by their EVPPI (HE-FM-VOI-003).

  Worked example (Trial of 100 patients per arm): With the illustrative prior mean of GBP 500 and standard deviation of GBP 1,000, arm standard deviations of GBP 10,000, and 2,000 patients a year for five years at an illustrative 3.5%, the trial is worth about GBP 61.68 per patient, about 31% of EVPI, and about GBP 576,507 across the population. m0 = 500; sd0 = 1000; sd_A = 10000; sd_B = 10000; n = 100; N = 2000; Y = 5; r = 0.035; Q_s = 0.19323812; Q_0 = 0.30853754; v0 = 1000000; sigma2 = 200000000; n0 = 200; nu = 0.333333; s2 = 333333.33; s = 577.350269; z_s = 0.866025; phi_s = 0.274189; EVSI = 61.68; z_0 = 0.5; phi_0 = 0.352065; EVPI = 197.8; share = 0.3119; D = 4.673079; EVSI_pop = 576507.41

  Worked example (Trial of 400 patients per arm): Quadrupling the trial raises EVSI to about GBP 134.97 per patient, about 68% of EVPI, and population EVSI to about GBP 1.26 million, so value rises with n but less than in proportion. m0 = 500; sd0 = 1000; sd_A = 10000; sd_B = 10000; n = 400; N = 2000; Y = 5; r = 0.035; Q_s = 0.27014569; Q_0 = 0.30853754; v0 = 1000000; sigma2 = 200000000; n0 = 200; nu = 0.666667; s2 = 666666.67; s = 816.496581; z_s = 0.612372; phi_s = 0.330735; EVSI = 134.97; z_0 = 0.5; phi_0 = 0.352065; EVPI = 197.8; share = 0.6824; D = 4.673079; EVSI_pop = 1261460

  Worked example (Options equally preferred now): With a prior mean of zero, EVSI is about GBP 230.33 and EVPI about GBP 398.94 per patient, and the share equals the square root of the variance fraction, about 0.5774, a limiting case that checks the implementation. m0 = 0; sd0 = 1000; sd_A = 10000; sd_B = 10000; n = 100; N = 2000; Y = 5; r = 0.035; Q_s = 0.5; Q_0 = 0.5; v0 = 1000000; sigma2 = 200000000; n0 = 200; nu = 0.333333; s2 = 333333.33; s = 577.350269; z_s = 0; phi_s = 0.398942; EVSI = 230.33; z_0 = 0; phi_0 = 0.398942; EVPI = 398.94; share = 0.5774; D = 4.673079; EVSI_pop = 2152695.37

  Excel: =LET(pvar,PriorSDINB^2,nzero,(SDArmA^2+SDArmB^2)/pvar,spre,SQRT(pvar*PatientsPerArm/(PatientsPerArm+nzero)),zval,ABS(PriorMeanINB)/spre,spre*(NORM.S.DIST(zval,FALSE)-zval*(1-NORM.S.DIST(zval,TRUE)))) With named cells PriorMeanINB, PriorSDINB, SDArmA, SDArmB and PatientsPerArm, the formula returns per-person EVSI in Excel 2021 or Microsoft 365. Multiplying it by AnnualPop*PV(DiscountRate,DecisionYears,-1,0,1) gives population EVSI.

  R: evsi_normal <- function(m0, sd0, sd_A, sd_B, n, N, Y, r) { L <- function(s) { z <- abs(m0)/s; s*(dnorm(z)-z*(1-pnorm(z))) }; n0 <- (sd_A^2+sd_B^2)/sd0^2; s <- sqrt(sd0^2*n/(n+n0)); e <- L(s); data.frame(n = n, EVSI = e, EVPI = L(sd0), share = e/L(sd0), EVSI_pop = e*N*(1+r)*(1-(1+r)^(-Y))/r) } Vectorised over n, so one call traces EVSI against trial size.

  Python: def evsi_normal(m0, sd0, sd_A, sd_B, n, N, Y, r): nd = statistics.NormalDist(); L = lambda s: s*(nd.pdf(abs(m0)/s)-abs(m0)/s*(1-nd.cdf(abs(m0)/s))); n0 = (sd_A**2+sd_B**2)/sd0**2; s = math.sqrt(sd0**2*n/(n+n0)); e = L(s); return e, L(sd0), e/L(sd0), e*N*(1+r)*(1-(1+r)**(-Y))/r Needs the standard library modules math and statistics only, and returns EVSI, EVPI, the share and population EVSI.

  Test (Share of EVPI equals the square root of the variance fraction at a zero prior mean): With the options equally preferred now, EVSI divided by EVPI equals the square root of n over n plus n0, about 0.5774 for 100 per arm. Expected result TRUE. Excel check =ABS(SQRT(1E6*100/(100+2E8/1E6))*NORM.S.DIST(0,FALSE)/(1000*NORM.S.DIST(0,FALSE))-SQRT(100/300))<1E-9

  Test (Single-loop Monte Carlo agrees with the closed form): With 100,000 simulated trials of 100 per arm, the average shortfall of the posterior mean below zero is within GBP 3 of the closed-form GBP 61.68, a margin of more than five Monte Carlo standard errors. Expected result TRUE in Microsoft 365, recalculated with new random numbers each time. Excel check =LET(k,100000,t,500+1000*NORM.S.INV(RANDARRAY(k)),x,t+SQRT(2E8/100)*NORM.S.INV(RANDARRAY(k)),pm,(200*500+100*x)/300,ABS(AVERAGE(IF(pm<0,-pm,0))-61.68)<3)

  Common error (Total trial size entered as the number per arm): Entering a trial of 200 patients in total as n of 200 per arm doubles the information and gives about GBP 99.82 per patient instead of GBP 61.68 for 100 per arm.

  Source: Jalal H, Goldhaber-Fiebert JD, Kuntz KM. Computing expected value of partial sample information from probabilistic sensitivity analysis using linear regression metamodeling. Medical Decision Making. 2015;35(5):584-595. The EVSI section and Box 2, which compute EVSI from the unit normal loss integral at the prior mean of incremental net benefit and the preposterior variance, the prior variance times n over n plus n0. Population scaling follows equation 15 of Rothery C, Strong M, Koffijberg HE, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of information analytical methods. Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(3):277-286.

  v0 = sd0^2; sigma2 = sd_A^2 + sd_B^2; n0 = sigma2 / v0; nu = n / (n + n0); s2 = nu * v0; s = sqrt(s2); z_s = abs(m0) / s; phi_s = exp(-z_s * z_s / 2) / sqrt(2 * 3.14159265); EVSI = s * (phi_s - z_s * Q_s); z_0 = abs(m0) / sd0; phi_0 = exp(-z_0 * z_0 / 2) / sqrt(2 * 3.14159265); EVPI = sd0 * (phi_0 - z_0 * Q_0); share = EVSI / EVPI; D = (1 + r) * (1 - (1 + r)^(-Y)) / r; EVSI_pop = EVSI * N * D

Try this function

Implementations

• Excel

  Closed-form EVSI from the prior mean and preposterior variance in Excel

  With the prior mean of incremental net benefit in a cell named PriorMeanINB and the preposterior variance in PrepostVar, the formula returns EVSI per patient. NORM.S.DIST with FALSE gives the density and with TRUE the cumulative probability.

  =SQRT(PrepostVar)*NORM.S.DIST(ABS(PriorMeanINB)/SQRT(PrepostVar),FALSE)-ABS(PriorMeanINB)*(1-NORM.S.DIST(ABS(PriorMeanINB)/SQRT(PrepostVar),TRUE))

Assumptions

• Two options compared on incremental net benefit in the closed form

  The closed form holds for a choice between two options, where the decision depends only on the sign of incremental net benefit. With three or more options the expected maximum has no one-dimensional closed form and simulation is needed, although Lee and colleagues derive a two-dimensional closed form for three options and note that the sample information algorithms can be adapted to it.

• Normal preposterior distribution of the posterior mean

  The posterior mean of incremental net benefit is normal across possible results, which is exact for a normal prior and a normal study estimate and approximate otherwise. The second ISPOR report states that EVSI can be computed analytically under normality with a known reduction in variance, and that sampling-based methods are needed otherwise.

Worked examples

• Closed-form EVSI for 100 patients per arm with a prior mean of GBP 500

  For the illustrative prior with mean GBP 500 and standard deviation GBP 1,000 per patient, the preposterior variance of about 333,333 for 100 per arm gives a preposterior standard deviation of about GBP 577 and z of about 0.866. EVSI is about GBP 61.68 per patient, about 31% of the per-person EVPI of about GBP 197.80.

  m0 = 500; s2 = 333333.33; Q_z = 0.19323812; s = 577.350266; z = 0.866025; phi_z = 0.274189; EVSI = 61.68

• Closed-form EVPI recovered with the prior variance of incremental net benefit

  Entering the prior variance of 1,000,000 as s2 corresponds to an unlimited sample and returns the per-person EVPI of about GBP 197.80 under normality, the upper limiting case.

  m0 = 500; s2 = 1000000; Q_z = 0.30853754; s = 1000; z = 0.5; phi_z = 0.352065; EVSI = 197.8

• Closed-form EVSI with the two options equally preferred now

  With a prior mean of zero, z is zero, Q_z is 0.5 and EVSI is s times the density at zero, about 0.3989, or about GBP 230.33 per patient for 100 per arm. For a given s2 this is the largest EVSI any prior mean can give, because every result then decides the choice.

  m0 = 0; s2 = 333333.33; Q_z = 0.5; s = 577.350266; z = 0; phi_z = 0.398942; EVSI = 230.33

Common errors

• Prior variance used in place of the preposterior variance in the closed form

  Putting the prior variance of 1,000,000 into the formula returns per-person EVPI, about GBP 197.80, whatever the trial size, so a trial of 100 per arm is credited with more than three times its EVSI of about GBP 61.68.

• Cumulative normal probability used in place of the upper-tail probability

  Entering Phi(z), about 0.807 for 100 per arm, as Q_z gives about minus GBP 245 per patient, a negative EVSI that cannot occur.

Sources

• Jalal and colleagues on EVPI and EVSI from the unit normal loss integral

  Jalal H, Goldhaber-Fiebert JD, Kuntz KM. Computing expected value of partial sample information from probabilistic sensitivity analysis using linear regression metamodeling. Medical Decision Making. 2015;35(5):584-595. Equation 2 gives the unit normal loss integral for normal incremental net benefit with mean mu and variance sigma squared as mu times Phi(minus mu over sigma) less sigma over the square root of 2 pi times exp(minus mu squared over 2 sigma squared), with mu the incremental net benefit of the option chosen now, and Box 2 computes EVSI as minus that integral at the prior mean and the preposterior variance.

  View source →

• Lee, Gustafson and Sadatsafavi on the unit normal loss integral for two strategies

  Lee TY, Gustafson P, Sadatsafavi M. Closed-form solution of the unit normal loss integral in 2 dimensions, with application in value-of-information analysis. Medical Decision Making. 2023;43(5):621-626. The introduction defines the unit normal loss integral as the expectation of max(Y, 0) for normal Y with mean mu and standard deviation sigma, equal to mu times 1 minus Phi(minus mu over sigma) plus sigma times phi(minus mu over sigma). It states that the closed form applies to comparisons of 2 strategies on incremental net benefit and has been extended to EVSI (Willan and Pinto 2005), and the note derives a 2-dimensional form for 3 strategies.

  View source →

• Heath and colleagues on EVSI rising with sample size below EVPPI

  Heath A, Kunst NR, Jackson C, Strong M, Alarid-Escudero F, Goldhaber-Fiebert JD, Baio G, Menzies NA, Jalal H. Calculating the expected value of sample information in practice. Considerations from 3 case studies. Medical Decision Making. 2020;40(3):314-326. The results for the chronic pain case study and Figure 3 report EVSI estimates from all four methods increasing with the sample size of the future trial and remaining below EVPPI, which marks the upper limit.

  View source →

Canonical Identity