Expected Value of Sample Information

Expected Value of Sample Information for Study Design

Expected value of sample information (EVSI) puts a money or health value on a particular proposed study, with a stated design and sample size, before anyone runs it. The calculation averages, over every result the study could plausibly produce, how much better the treatment decision would be once those results are known. Perfect information is never on offer in practice, so EVSI is the value of information measure that speaks to real research proposals, and it is the quantity that research costs are set against when judging whether a study is worth funding. The HEMA report on analysing and presenting uncertainty for HTA decisions recommends reporting the expected value of perfect information alongside every probabilistic analysis, with EVSI used where a research decision is actually in prospect.

Where EVSI sits among the value of information measures

The value of information measures answer progressively narrower questions about the same decision. The expected value of perfect information (EVPI) values removing all uncertainty, and the expected value of partial perfect information (EVPPI) values removing all uncertainty about a chosen group of parameters. EVSI values a study that reduces, rather than eliminates, that uncertainty, and the first ISPOR Value of Information Task Force report ties it to a given research study with a specific sample size and a particular design.

The measures are nested. When a study informs one group of parameters and nothing else, its EVSI is bounded above by the EVPPI for that group, and EVPPI is in turn bounded by EVPI. In one case study from a published comparison of estimation methods, EVSI from all four methods rose as the proposed trial's sample size increased, with EVPPI marking its upper limit. Because the decision maker remains free to ignore a study's findings and keep the current choice, EVSI cannot be negative, although a particular study result can still lead to a choice that later proves worse.

The EVSI formula and its Bayesian logic

EVSI compares two expected values. One is the expected net benefit of the best strategy chosen on current evidence; the other is the expected net benefit of the best strategy chosen after the study reports, averaged over the results it might give. The second ISPOR task force report writes the difference as follows, here in net benefit terms.

$$\text{EVSI} = E_{X}\left[\max_{d} E_{\theta \mid X}\{NB(d,\theta)\}\right] - \max_{d} E_{\theta}\{NB(d,\theta)\}$$

where $d$ indexes the strategies compared, $\theta$ is the vector of uncertain model parameters, $NB(d,\theta)$ is the net benefit of strategy $d$ at parameter values $\theta$, $X$ is the data the proposed study would collect, $E_{\theta \mid X}$ is the expectation over the updated (posterior) distribution of $\theta$ once $X$ is known, and $E_X$ is the expectation over the datasets the study could produce given current evidence.

The data are observable but not yet observed, which is why the outer expectation is needed. In the Bayesian set-up described by Kunst and colleagues, the distribution used in the probabilistic sensitivity analysis acts as the prior distribution, and a sampling distribution for the data describes how the study would generate results; Bayes theorem combines the two into a posterior distribution for each simulated dataset. Net benefit is usually expressed as net monetary benefit or net health benefit at a stated cost-effectiveness threshold. A study result has value only if it could change the strategy chosen, so a study whose every plausible result leaves the current decision in place has an EVSI of zero.

What a study design has to specify

EVSI is defined for a whole study design. The first ISPOR report states that the informativeness of research depends on its sample size and allocation, length of follow-up and endpoints, and notes that these factors also drive its cost. Relative treatment effects may call for a randomised trial, while observational studies may suit parameters such as event costs or health-related quality of life. The second report recommends choosing the data-generating distribution to reflect how the data would be analysed if the study were conducted, and reporting the prior distribution and likelihood used.

The decision context also changes the result. Strong and colleagues list costs of delaying or reversing adoption, whether the adoption decision will be fully implemented and whether a study extends across jurisdictions as factors that affect EVSI. The same logic applies to research already under way, where EVSI can value extending the follow-up of an ongoing trial; that use is covered on the adaptive value of information page.

Worked example with two candidate studies

All numbers in this example are illustrative. A health system is choosing between standard care A and a new treatment B. Net monetary benefit per patient is GBP 1,000 for A whatever the truth. For B it is GBP 1,600 if B works as hoped (state F) and GBP 600 if it does not (state U), and current evidence gives state F a probability of 0.45. A real study usually returns a continuous estimate; here each study returns a positive or negative result so that every step stays visible.

Step 1 finds the decision on current evidence.

$$E[NB_B] = 0.45 \times 1{,}600 + 0.55 \times 600 = 720 + 330 = 1{,}050$$

where 0.45 and 0.55 are the probabilities of states F and U. B is adopted now, with expected net benefit of GBP 1,050 per patient against GBP 1,000 for A.

Step 2 finds the ceiling. With perfect information B would be chosen in state F and A in state U, giving expected net benefit of $V_{PI} = 0.45 \times 1{,}600 + 0.55 \times 1{,}000 = 1{,}270$, so EVPI is GBP 220 per patient.

Step 3 values study S1, a smaller study whose result points the right way with probability 0.7 in either state. A positive result arises with probability $P(+) = 0.45 \times 0.7 + 0.55 \times 0.3 = 0.48$, and Bayes theorem raises the probability of state F to $P(F \mid +) = 0.315 / 0.48 = 0.656$, so B is kept with expected net benefit GBP 1,256.25. A negative result (probability 0.52) lowers it to $P(F \mid -) = 0.135 / 0.52 = 0.260$, giving B an expected net benefit of GBP 859.62, so the decision switches to A at GBP 1,000.

$$\text{EVSI}_{S1} = 0.48 \times 1{,}256.25 + 0.52 \times 1{,}000 - 1{,}050 = 603 + 520 - 1{,}050 = 73$$

where 0.48 and 0.52 are the probabilities of a positive and a negative result and 1,050 is the expected net benefit on current evidence.

Step 4 repeats this for study S2, a larger study that points the right way with probability 0.9. A positive result (probability 0.46) gives state F a probability of 0.880 and B an expected net benefit of GBP 1,480.43; a negative result (probability 0.54) gives 0.083 and GBP 683.33, so the decision switches to A. Expected net benefit with S2 is $V_{S2} = 0.46 \times 1{,}480.43 + 0.54 \times 1{,}000 = 1{,}221$, and EVSI is GBP 171 per patient.

All the value comes from the negative result, the only result that changes the decision. For S1 it equals $\text{EVSI}_{S1} = 0.52 \times (1{,}000 - 859.62) = 73$, the chance of switching times the net benefit gained by switching. If 10,000 future patients would be treated on the basis of the result, population EVSI is GBP 0.73 million for S1 and GBP 1.71 million for S2; at study costs of GBP 0.4 million and GBP 1.2 million, the expected net benefit of sampling is GBP 0.33 million and GBP 0.51 million, so the larger study is the better buy despite costing three times as much.

Computing EVSI in real models

Outside simple cases EVSI needs simulation. Ades, Lu and Claxton derived Monte Carlo methods that extended EVSI calculations to medical decision models with multiple sources of uncertainty, with approximate methods for the common case where evidence on treatment efficacy comes as relative measures such as odds ratios or hazard ratios. The second ISPOR report notes that an analytic solution exists when incremental net benefit is normally distributed and the study gives a known reduction in its variance; otherwise the basic approach is a nested double loop, generating plausible datasets in an outer loop and sampling from the posterior for each one in an inner loop. Strong and colleagues cite published analyses in which this would have taken 7.5 days and 37.5 days.

Kunst and colleagues give practical guidance on four faster approximation methods, which are regression on the probabilistic analysis sample, importance sampling, Gaussian approximation and moment matching. They cite an earlier three-case-study comparison in which none was systematically superior in accuracy, precision or computation time. The Gaussian approximation and moment matching methods estimate EVSI across sample sizes at about the cost of one estimate, which suits a search for the optimal sample size; importance sampling is not recommended for studies of more than 1,000 participants, and moment matching is not recommended for fewer than 10. The wider workflow and software are described on the VOI analysis page.

Using EVSI in research and funding decisions

EVSI becomes decision-relevant once it is scaled to everyone whose care the results could change and compared with the cost of the study. The first ISPOR report recommends comparing population EVSI for the proposed design with its expected cost, calculating EVSI and the expected net benefit of sampling for a range of designs to find the one with the largest net benefit, and presenting the results across a range of cost-effectiveness thresholds. When no design has a positive expected net benefit of sampling, it treats current evidence as sufficient for the decision.

The HEMA report states that further research, whether only in research or, where routine adoption is compatible, through managed access, is warranted only where the expected value of resolving uncertainty exceeds the cost of the research and of the health forgone by waiting. It adds that a low value of information despite high uncertainty is a reason not to demand more evidence.

Limits and common misreadings

EVSI is only as good as the model and the uncertainty fed into it. The first ISPOR report states that any value of information analysis is conditional on the model and on how parameter uncertainty is specified, and that no assessment can capture unknown unknowns. Generating plausible datasets can also be hard when the data-generating process involves bias, censoring, missing data or measurement error, as the second report notes.

A frequent error is comparing per-patient EVSI directly with the total cost of a study, when the per-patient value has to be scaled to the beneficiary population first. Another is reading a high EVSI as a forecast that the study will show benefit; EVSI values the ability to respond to whatever the study finds, and in the example above it comes entirely from results that would reverse adoption. Standard calculations also assume a risk-neutral decision maker, and the per-patient calculation that Strong and colleagues describe assumes costless reversal of the decision, perfect implementation and no delay in either the study or adoption, so departures from these assumptions need explicit treatment.

Sources

• Fenwick E, Steuten L, Knies S, Ghabri S, Basu A, Murray JF, Koffijberg H, Strong M, Sanders Schmidler GD, Rothery C. Value of information analysis for research decisions: an introduction. Report 1 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(2):139-150. doi:10.1016/j.jval.2020.01.001
• Rothery C, Strong M, Koffijberg H, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of information analytical methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(3):277-286. doi:10.1016/j.jval.2020.01.004
• Steuten L, Thavorn K, Cipriano LE, Guertin JR, McQueen B, Schmidler G, Sullivan S, Wailoo A, Sculpher M. Methods to Analyze and Present Uncertainty in Economic Evaluation to Support HTA Decisions. HEMA Report 2. Health Economics Methods Advisory; 2026. https://hemamethods.org/wp-content/uploads/2026/10/HEMA-Report-2_For-Publication_100726.pdf
• Ades AE, Lu G, Claxton K. Expected value of sample information calculations in medical decision modeling. Medical Decision Making. 2004;24(2):207-227. doi:10.1177/0272989X04263162
• Strong M, Oakley JE, Brennan A, Breeze P. Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast, nonparametric regression-based method. Medical Decision Making. 2015;35(5):570-583. doi:10.1177/0272989X15575286
• Heath A, Kunst NR, Jackson C, Strong M, Alarid-Escudero F, Goldhaber-Fiebert JD, Baio G, Menzies NA, Jalal H. Calculating the expected value of sample information in practice: considerations from 3 case studies. Medical Decision Making. 2020;40(3):314-326. doi:10.1177/0272989X20912402
• Kunst N, Wilson ECF, Glynn D, Alarid-Escudero F, Baio G, Brennan A, Fairley M, Goldhaber-Fiebert JD, Jackson C, Jalal H, Menzies NA, Strong M, Thom H, Heath A. Computing the expected value of sample information efficiently: practical guidance and recommendations for four model-based methods. Value in Health. 2020;23(6):734-742. doi:10.1016/j.jval.2020.02.010
• Vervaart M, Strong M, Claxton KP, Welton NJ, Wisløff T, Aas E. An efficient method for computing expected value of sample information for survival data from an ongoing trial. Medical Decision Making. 2022;42(5):612-625. doi:10.1177/0272989X211068019