Signature
ibar_k = alpha_k + beta_k * mu_c; EVPPI_1L = sum_(k=1)^K [max(ibar_k, 0)] / K - max(sum_(k=1)^K [ibar_k] / K, 0)
| Inputs | Definition | Unit |
|---|---|---|
alpha_k | Part of the incremental net benefit at draw k that does not depend on theta_c | currency per person |
beta_k | Change in incremental net benefit at draw k per unit of theta_c, which may depend on theta_i. With several remaining parameters each has its own slope and mean | currency per person per unit of theta_c |
mu_c | Mean of the remaining parameter theta_c, used in every model run of the single loop | units of theta_c |
K | Number of values of theta_i drawn, each with one model run | a whole number |
ibar_k | Incremental net benefit at draw k of theta_i with theta_c at its mean, equal to the conditional expectation when the model is linear in theta_c | currency per person |
|---|---|---|
EVPPI_1L | Single-loop estimate of the per-person expected value of perfect information about theta_i | currency per person, such as GBP |
Function
Estimation of the expected value of partial perfect information from a probabilistic analysis
Maps a decision model, or the probabilistic sample it has produced, and a chosen group of parameters of interest theta_i to an estimate of the expected value of partial perfect information (EVPPI) per person. Each estimator approximates the expected net benefit of every option given theta_i, takes the best option for each value of theta_i, averages, and subtracts the value of the best option on current information. The definition and its order of expectation and maximisation, with a discrete worked case, are on the Value of Information page (HE-FM-VOI-003), the per-person EVPI that bounds every EVPPI from the same sample is HE-FM-VOI-001, and population scaling is HE-FM-VOI-002. This package adds the estimators set out by the ISPOR Value of Information Task Force, with checks that each estimate is at least zero and no larger than the EVPI. Notation follows the Expected Value of Partial Perfect Information article.
Implementations
Excel
Single-loop EVPPI from intercept and slope columns
With alpha_k and beta_k for each draw of theta_i in ranges named Alpha and Beta and the mean of theta_c in a cell named MuC, the formula returns the estimate. In practice the model is run once per draw with theta_c set at its mean, and that column of incremental net benefits takes the place of Alpha+Beta*MuC in the same formula.
=SUMPRODUCT(((Alpha+Beta*MuC)>0)*(Alpha+Beta*MuC))/COUNT(Alpha)-MAX(AVERAGE(Alpha)+AVERAGE(Beta)*MuC,0)
Assumptions
Net benefit linear in the remaining parameters given theta_i
For each value of theta_i the incremental net benefit is a straight-line function of theta_c, or a sum of products of complementary parameters that are not correlated where they are multiplied together, so its expectation over theta_c equals the model evaluated at their means. A parameter that enters squared or inside an exponential breaks the shortcut.
Remaining parameters independent of the parameters of interest for the shortcut
One mean mu_c applies at every draw of theta_i only when theta_c is independent of theta_i. Under correlation the mean of theta_c given each value of theta_i is required, and Brennan and colleagues report that the shortcut with a single set of means can then be substantially inaccurate.
Worked examples
Article's EVPPI for the treatment effect with the utility at its mean
Illustrative, from the article's example, with the utility B coded as an indicator that is 1 when high and has a mean of 0.5. Given high A the incremental net benefit is GBP 1,000 plus GBP 1,000 times the indicator, and given low A it is GBP -2,400 plus GBP 2,600 times the indicator. At the mean these give GBP 1,500 and GBP -1,100, and the shortcut returns GBP 550, the same as the nested estimator, because the model is linear in the indicator.
K = 2; alpha_k = [1000,-2400]; beta_k = [1000,2600]; mu_c = 0.5; ibar_k = [1500,-1100]; EVPPI_1L = 550
Article's EVPPI for the utility with the treatment effect at its mean
With the treatment effect A coded as an indicator with a mean of 0.5, the incremental net benefit is GBP 200 plus GBP 1,800 times the indicator given high B and GBP -2,400 plus GBP 3,400 times the indicator given low B. At the mean these give GBP 1,100 and GBP -700, and the shortcut returns the article's EVPPI for B of GBP 350.
K = 2; alpha_k = [200,-2400]; beta_k = [1800,3400]; mu_c = 0.5; ibar_k = [1100,-700]; EVPPI_1L = 350
Article's adverse-event cost that never changes the decision
Limiting case. The article's third parameter moves incremental net benefit by GBP 100 up or down and adds to the part that depends on A and B, whose mean is GBP 200. Both plug-in values, GBP 300 and GBP 100, favour the new treatment, so the decision never changes and the EVPPI is zero.
K = 2; alpha_k = [100,-100]; beta_k = [1,1]; mu_c = 200; ibar_k = [300,100]; EVPPI_1L = 0
Common errors
Single-loop shortcut applied to a model curved in the remaining parameter
Suppose, for illustration, that a remaining parameter takes the values 0 or 2 with equal probability, so its mean is 1, and enters squared, with incremental net benefit GBP 1,000 plus GBP 250 times its square given high A and GBP -1,600 plus GBP 250 times its square given low A. The true conditional means are GBP 1,500 and GBP -1,100 and the EVPPI for A is GBP 550. Plugging in the mean gives GBP 1,250 and GBP -1,350, a current decision that wrongly rejects the new treatment and an EVPPI of GBP 625 (computed here for illustration).
Marginal mean used for a remaining parameter correlated with theta_i
When theta_c is correlated with theta_i, the mean that applies at draw k is the mean of theta_c given theta_i at that draw, and a single marginal mean for every draw misstates each conditional expected net benefit. Brennan and colleagues report that even relatively small correlation can make the shortcut substantially inaccurate.
Sources
ISPOR single-loop scheme for EVPPI
Rothery C, Strong M, Koffijberg HE, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of information analytical methods. Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(3) 277-286. Section on EVPPI computation, which states that when the model is linear or multilinear in the complementary parameters, with no correlation between those multiplied together, the double loop can be replaced by a single loop using their mean values, Algorithm 3, and the preference for single-loop methods where applicable to reduce Monte Carlo error.
Analytic inner expectation in linear and multilinear models
Strong M, Oakley JE, Brennan A. Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample, a nonparametric regression approach. Medical Decision Making. 2014;34(3) 311-326. Methods section after Equation 4, which notes that in the linear case the inner expectation has an analytic solution and in the multilinear case it does if inputs are independent, while the outer expectation rarely has one because of the maximisation step.
Accuracy of shortcut algorithms for partial EVPI
Brennan A, Kharroubi S, O'Hagan A, Chilcott J. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Medical Decision Making. 2007;27(4) 448-470. Abstract, which reports that alternative computation methods and shortcut algorithms are discussed with the mathematical conditions for their use, and that if relatively small correlation or nonlinearity is present the shortcut algorithm can be substantially inaccurate.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0