VOI Analysis

VOI Analysis from PSA Output to a Decision on Further Research

In health economics, a VOI (value of information) analysis is the applied procedure that turns the probabilistic output of a cost-effectiveness model into a recommendation about research. It works in stages, starting with the expected value of perfect information (EVPI) for the whole decision, then partial perfect information (EVPPI) for groups of parameters, then sample information (EVSI) and the expected net benefit of sampling (ENBS) for specific study designs, with each result scaled to the people whose care the evidence could change and compared with what the research would cost.

The two 2020 reports of the ISPOR Value of Information Task Force set out this workflow, with a stop-or-continue rule at each stage, guidance on choosing the beneficiary population and time horizon, and approximation methods and software for EVPPI and EVSI, and the NICE methods manual (PMG36) sets out how uncertainty and research recommendations are handled in appraisal. The decision theory and formulas are on the value of information page, and EVPI, population EVPI, EVPPI, EVSI and expected net benefit of sampling each have their own page. Value of information analysis is the same method under its full name.

Two questions, asked in order

A VOI analysis separates the adoption decision from the research decision. The first ISPOR report notes that, under Bayesian decision theory, a risk-neutral decision maker should choose the option with the highest expected payoff whatever the uncertainty, while decision uncertainty still matters for judging whether more evidence is worth collecting. The second report sets the same order, with the expected-value decision first and the question of further research second.

Payoffs are usually expressed as net health benefit or net monetary benefit; in money terms, the value of research can be set directly against its expected cost. The first report lists four kinds of decision the analysis can support, namely research prioritisation by funders, the design of efficient studies, reimbursement (including coverage with evidence development) and decisions over a technology's life cycle. Where adoption carries sunk costs, it notes that "only in research" may be preferable to "approval with research".

The workflow from model to research decision

The first ISPOR report sets the analysis out as a sequence in which each stage either stops the process or justifies the next, more demanding calculation. The steps below follow it, with detail from the second report.

1. Build and scope the decision model. State the decision problem, perspective, target population, comparators, outcomes and time horizon.
2. Assign a distribution to every uncertain parameter. A parameter left out because little is known about it is treated as known with certainty. Represent dependent parameters jointly, use expert elicitation where evidence is thin and quantify structural uncertainty where possible.
3. Run the probabilistic sensitivity analysis and keep the draws. Store each sampled parameter set with the net benefit of every strategy, often called the probabilistic analysis (PA) dataset; every later step reads from it.
4. Estimate EVPI and scale it to the population. EVPI comes from the same draws in a single loop. If population EVPI is below the expected cost of research, research is not worthwhile and the analysis stops. If it is above, the condition is necessary but not sufficient.
5. Estimate EVPPI for groups of parameters a study would inform. EVPPI is typically not additive, so parameters that one study would inform are grouped. Population EVPPI for each group is compared with the expected cost of research that would inform it. A high EVPPI marks a parameter whose uncertainty drives decision uncertainty, which is not always the most uncertain parameter.
6. Estimate EVSI and ENBS for candidate designs. EVSI depends on sample size, allocation, follow-up and endpoints. Research cost covers fixed costs, a variable cost per participant and the opportunity cost for participants given the option expected to be inferior. A positive ENBS is a sufficient condition; the design with the largest ENBS is the most efficient; if no design has a positive ENBS, current evidence is treated as sufficient.
7. Report across thresholds. Present results for a range of cost-effectiveness thresholds, emphasising explicit ones such as the PMG36 range of £25,000 to £35,000 per QALY.
8. Repeat when the decision changes. Reassess when new research is published or a new comparator appears.

Scaling to the beneficiary population

Per-person values become decision-relevant only when multiplied by everyone who could benefit from the information. Patients in later decisions count too. The second ISPOR report gives the population value as follows.

$$\text{Population VOI} = \text{VOI}_{\text{per person}} \times \sum_{t=0}^{T} \frac{I_t}{(1+d)^t}$$

where $\text{VOI}_{\text{per person}}$ is the per-person EVPI, EVPPI or EVSI, $I_t$ is incidence (people newly facing the decision) in period $t$, $T$ is the last period of the time horizon and $d$ is the discount rate per period.

The first report bases the population on prevalent and/or incident cohorts and calls the effective time horizon the harder choice, listing factors such as patent expiry, launch of a substitute in the same class, new diagnostic tests and price changes. Published studies tend to report one year and then 10 to 20 years without clear justification, although results can be highly sensitive to the horizon, so it should be justified and alternatives explored in scenarios. Because benefits arrive only once a study reports, the population is usually adjusted for the study's duration, while participants in the better arm benefit meanwhile, which matters most in rare diseases. The second report adds gradual uptake and use of results in several jurisdictions.

Worked example of a population EVPI gate (illustrative)

All numbers in this example are illustrative. A probabilistic analysis gives a per-person EVPI of £180 at £25,000 per QALY, the lower end of the range in PMG36. About 4,000 people a year newly face the treatment choice, the decision is expected to stay relevant for 10 years (periods 0 to 9), and the discount rate is 3.5% a year, the PMG36 reference-case rate for costs and health effects. The proposed trial would cost £2.5 million.

Step 1 sums the discount factors from year 0 to year 9 in the following way.

$$\sum_{t=0}^{9} \frac{1}{1.035^t} = 8.6077$$

where 1.035 is one plus the 3.5% rate and $t$ is the year.

Step 2 multiplies by annual incidence to give the discounted beneficiary population, as follows.

$$4{,}000 \times 8.6077 = 34{,}430.8$$

where 4,000 is annual incidence and 34,430.8 is the discounted number of people who could benefit. Step 3 applies the per-person EVPI, as follows.

$$180 \times 34{,}430.8 = 6{,}197{,}544$$

where 180 is per-person EVPI in pounds and the result is population EVPI in pounds. That is about £6.20 million, or 247.9 QALYs at £25,000 per QALY.

Step 4 repeats the calculation under the alternatives the first report asks analysts to explore.

Each scenario exceeds the £2.5 million cost, so further research is potentially worthwhile and the analysis moves on to EVPPI. This is a necessary condition only, because EVPI assumes all uncertainty is removed, and a real trial will remove part of it. Had the trial cost £4 million, the five-year horizon would have stopped the process while the base case would not, which is why the horizon has to be justified. At £35,000 per QALY the per-person EVPI would be recomputed from the PSA, not rescaled.

Computing EVPPI and EVSI in practice

EVPI needs only the stored draws, but EVPPI and EVSI contain a nested expectation that, in its basic form, needs a double loop of simulation. Strong, Oakley and Brennan describe a patient-level model where 1,000 outer loops, 1,000 inner loops and 10,000 patient runs per evaluation would mean ${10}^{11}$ model evaluations. Three routes reduce this burden.

• Single-loop shortcuts. The second ISPOR report notes that when the model is linear or multilinear in the remaining parameters, with no correlation between multiplied terms, their mean values can replace the inner loop.
• Regression on the PSA sample for EVPPI. Strong, Oakley and Brennan regress each strategy's net benefit on the parameters of interest with a generalised additive model or Gaussian process regression, using only the PSA sample. The model need not be rerun, which helps when it was built in specialist software or the analyst does not own it. They report that a generalised additive model can fail with more than about six parameters that interact, where a Gaussian process is preferred, and residual plots check the fit.
• Approximation methods for EVSI. Kunst and colleagues compare regression-based, importance sampling, Gaussian approximation and moment matching methods, citing an earlier three-case-study comparison in which none was systematically superior in accuracy, precision or computation time.

Moment matching rescales the simulations used for EVPPI using a small number of nested runs, usually 30 to 50, and rests on a Bayesian model. Kunst and colleagues do not recommend it for studies of fewer than 10 patients, and it is less accurate when EVPPI is small relative to EVPI; ideally EVPPI exceeds 40% of EVPI. Moment matching and the Gaussian approximation estimate EVSI across sample sizes at about the cost of one estimate, which suits a search for the optimal sample size; importance sampling is not recommended for studies of more than 1,000 participants.

On software, the second ISPOR report names R libraries such as BCEA and the web tools SAVI and BCEAweb. SAVI computes EVPI and EVPPI from uploaded PSA results, and Kunst and colleagues explain how it can also give EVSI once simulated study summaries are added. The voi package for R covers EVPI, EVPPI, EVSI and ENBS with several computational methods under one interface. Fast tools still need each method's assumptions checked.

How NICE frames uncertainty and further research

The NICE technology appraisal manual (PMG36) does not name value of information analysis, but several requirements ask questions a VOI analysis answers. Section 4.7.1 asks for an overall assessment of uncertainty that highlights uncertainties unlikely to be reduced by further evidence, and section 4.7.2 asks the model to quantify decision uncertainty as the probability that a different decision would be reached if true cost effectiveness were known. Section 6.2.32 asks committees to weigh the likelihood of decision error and its consequences for patients and the NHS.

When evidence is highly uncertain, section 6.2.30 allows recommendations that include managed access (medicines only), data collection or research. Under section 6.4.6, a recommendation with managed access needs plausible potential to be cost effective at the current price, new evidence expected from trials or NHS practice, and data collectable within up to 5 years. Before recommending use only in research, section 6.4.14 asks the committee to consider the potential value to the NHS of additional evidence, what could be gained by reconsidering the decision once results arrive and how long that will take, balancing net benefits to current patients against those to current and future patients. Population EVSI and the delay adjustment give these judgements a quantitative form.

Limits and common misreadings

The first ISPOR report stresses that results are conditional on the model and on how parameter uncertainty is specified, and that no assessment can include unknown unknowns. The second report notes that structural uncertainty is often ignored, which amounts to assuming the model is perfect. Standard VOI also assumes a risk-neutral decision maker and that practice follows the decision; the first report notes that risk neutrality has been challenged and implementation is often imperfect.

A common misreading treats a high error probability as high VOI. VOI weighs the chance of a wrong decision by its consequences, so frequent small errors can have little value.

VOI analysis also differs from payback methods, and the first report notes that payback approaches cannot separate gains due to research from gains due to the resulting change in implementation.

Sources

• Fenwick E, Steuten L, Knies S, Ghabri S, Basu A, Murray JF, Koffijberg H, Strong M, Sanders Schmidler GD, Rothery C. Value of information analysis for research decisions: an introduction. Report 1 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(2):139-150. doi:10.1016/j.jval.2020.01.001
• Rothery C, Strong M, Koffijberg H, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of information analytical methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health. 2020;23(3):277-286. doi:10.1016/j.jval.2020.01.004
• Strong M, Oakley JE, Brennan A. Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: a nonparametric regression approach. Medical Decision Making. 2014;34(3):311-326. doi:10.1177/0272989X13505910
• Kunst N, Wilson ECF, Glynn D, Alarid-Escudero F, Baio G, Brennan A, Fairley M, Goldhaber-Fiebert JD, Jackson C, Jalal H, Menzies NA, Strong M, Thom H, Heath A. Computing the expected value of sample information efficiently: practical guidance and recommendations for four model-based methods. Value in Health. 2020;23(6):734-742. doi:10.1016/j.jval.2020.02.010
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 4.5.1, 4.7.1, 4.7.2, 6.2.30, 6.2.32, 6.3.4, 6.3.7, 6.4.6 and 6.4.14.
• Jackson C, Heath A. voi: Expected Value of Information. R package version 1.0.3. CRAN; 2024. https://cran.r-project.org/package=voi
• Strong M, Oakley JE, Brennan A. SAVI: Sheffield Accelerated Value of Information. Web application, release 2.2.1; 2025. https://savi.shef.ac.uk/SAVI/