Expected Value

Expected Value as the Probability-Weighted Mean of Costs, QALYs and Model Outputs

Almost every figure a health economic model reports, from the expected cost of a strategy to its expected QALYs and expected net benefit, is the expectation of a random quantity. As a statistical quantity, expected value is defined for discrete and continuous outcomes, and its linearity lets expected costs and QALYs be built up branch by branch and simulation by simulation. Linearity fails for curved functions, so a model run at mean inputs does not in general give mean outputs, and functions of several parameters, skewed patient-level costs and variance as a second expectation each need care, as two illustrative examples show. Choosing between strategies by their expected costs and outcomes is a separate decision principle, covered on the expected value analysis page.

The expectation of a random cost or health outcome

A random variable assigns a number, such as a cost in pounds or a QALY total, to each possible outcome of an uncertain process. Its expected value weights every possible value by its probability. For an outcome with a finite set of values, it is defined as follows.

$$\mathbb{E}[X] = \sum_{k=1}^{K} p_k x_k, \qquad \sum_{k=1}^{K} p_k = 1$$

where $X$ is the random outcome, $x_k$ is its $k$-th possible value, $p_k$ is the probability of that value and $K$ is the number of values. For a continuous non-negative outcome, such as a cost or a survival time, the sum becomes an integral.

$$\mathbb{E}[X] = \int_{0}^{\infty} x\, f(x)\, dx$$

where $f(x)$ is the probability density of $X$; for an outcome that can be negative, such as an incremental cost, the integral runs over the whole real line. The expectation exists only when the sum or integral converges, and for skewed costs or survival it depends heavily on the upper tail.

The expected value carries the units of $X$ and describes a distribution, not any one patient. It is a property of a probability model, whereas the mean of observed data is a statistic that estimates it. By linearity, the expected budget for $N$ patients is $N \times \mathbb{E}[C]$, where $C$ is cost per patient. By the law of large numbers, the average over a large cohort of independent patients with the same distribution also approaches the expected value. Uncertainty shared by every patient, such as an uncertain treatment effect, does not average out in this way.

Expectations at chance nodes and across model cycles

Decision models are built from conditional pieces, and the law of total expectation is what allows those pieces to be combined. Linearity then lets the pieces be added and rescaled. If the branches $B_1, \ldots, B_J$ leaving a chance node are mutually exclusive and exhaustive, the expectation combines as follows.

$$\mathbb{E}[X] = \sum_{j=1}^{J} P(B_j)\, \mathbb{E}[X \mid B_j]$$

where $P(B_j)$ is the probability of branch $j$ and $\mathbb{E}[X \mid B_j]$ is the expected outcome among patients who take it. Applied repeatedly from the terminal nodes back to the root, this identity is the rollback calculation set out on the chance node, folding back and decision tree pages.

For any two outcomes $X$ and $Y$ and constants $a$ and $b$, $\mathbb{E}[aX + bY] = a\,\mathbb{E}[X] + b\,\mathbb{E}[Y]$, whether or not $X$ and $Y$ are correlated. This is why the expected total cost of a Markov model is the sum of discounted expected costs per cycle, and why expected net benefit equals the threshold times expected QALYs minus expected costs. Linearity does not extend to ratios, so the ICER is formed from expected costs and QALYs after they have been calculated.

The Monte Carlo mean in probabilistic sensitivity analysis

When the parameters are uncertain, a model output becomes a function of random inputs, and its expected value is an average over the parameter distribution. That average is usually estimated by simulation. Formally, it is written as follows.

$$\mathbb{E}_\theta\left[g(\theta)\right] = \int g(\theta)\, \pi(\theta)\, d\theta \approx \frac{1}{N} \sum_{n=1}^{N} g\left(\theta^{(n)}\right)$$

where $g(\theta)$ is an output such as expected cost given parameter values $\theta$, $\pi(\theta)$ is the joint density representing parameter uncertainty, $\theta^{(n)}$ is the $n$-th sampled set and $N$ is the number of simulations. Doubilet and colleagues described this approach for medical decision trees in 1985, recording the mean and standard deviation of each strategy's expected utility across simulations. Monte Carlo averaging of this kind is the basis of probabilistic sensitivity analysis, and the simulated mean carries a Monte Carlo error with standard error $s_g / \sqrt{N}$, where $s_g$ is the output's standard deviation across draws.

Patient-level models add a second layer. In a microsimulation, $\mathbb{E}[Y] = \mathbb{E}_\theta\big[\mathbb{E}[Y \mid \theta]\big]$, where $Y$ is one simulated patient's outcome. The inner expectation averages over patients, the stochastic or first-order uncertainty in the ISPOR-SMDM task force terms, and the outer one over parameter, or second-order, uncertainty. O'Hagan, Stevenson and Madan noted that each run of such a model may simulate many thousands of patients, so few runs are feasible, and developed analysis-of-variance methods to estimate the mean and variance of the output more efficiently.

Where linearity stops, and Jensen's inequality

The mean of a linear function equals the function of the mean. For a curved function the two differ. Jensen showed in 1906 that the direction of the gap follows the curvature.

$$\mathbb{E}\left[g(X)\right] \ge g\left(\mathbb{E}[X]\right) \ \text{for convex } g, \qquad \mathbb{E}\left[g(X)\right] \le g\left(\mathbb{E}[X]\right) \ \text{for concave } g$$

where $g$ is a function of the random quantity $X$. Health economic models contain many curved functions, among them exponentials, powers and the back-transformation of log costs. A deterministic run at mean inputs is then biased, in a direction set by the curvature where $g$ is convex or concave throughout; the consequences for choosing between strategies are shown on the expected value analysis page.

The expected value of a function of several parameters

Most outputs depend on many parameters, and their dependence matters as well as their spread. The simplest case is a product. For the number of admissions times the cost per admission, the expectation is as follows.

$$\mathbb{E}[NU] = \mathbb{E}[N]\,\mathbb{E}[U] + \operatorname{Cov}(N, U)$$

where $N$ is the number of admissions, $U$ is the cost per admission and $\operatorname{Cov}(N, U)$ is their covariance. With an illustrative mean of 1.5 admissions, a mean of £3,000 per admission and a covariance of 300, the expected cost is £4,800 (1.5 × 3,000 + 300 = 4,800), not the £4,500 from multiplying the means. For a general smooth function, a second-order expansion shows where the gap comes from.

$$\mathbb{E}\left[g(\boldsymbol{\theta})\right] \approx g(\boldsymbol{\mu}) + \frac{1}{2} \sum_{i=1}^{m} \sum_{j=1}^{m} \frac{\partial^2 g}{\partial \theta_i \, \partial \theta_j} \operatorname{Cov}(\theta_i, \theta_j)$$

where $\boldsymbol{\theta}$ is the vector of $m$ parameters, $\boldsymbol{\mu}$ is the vector of their means, the second derivatives are taken at $\boldsymbol{\mu}$, and $\operatorname{Cov}(\theta_i, \theta_i)$ is the variance of $\theta_i$. The correction vanishes for a linear model, and for a multilinear model with independent parameters, such as a tree in which each parameter appears at most once on every pathway. Otherwise it depends on parameter variance, model curvature and correlation, and correlation can widen or narrow the gap. The expansion indicates sign and rough size; the simulated mean is the estimate.

Worked example of an uncertain hazard and a five-year fracture probability

The figures are illustrative. A falls-prevention model needs the probability of a hip fracture within five years. The annual hazard $h$ has a mean of 0.10 and a standard deviation of about 0.071, represented by a gamma distribution with shape $k = 2$ and scale $s = 0.05$, and each fracture costs £12,000. Under a constant hazard, $p = 1 - e^{-5h}$, a hazard-to-probability conversion that is concave in $h$.

1. Run at the mean hazard. The probability is $p = 1 - e^{-0.5} \approx 0.3935$, giving an expected fracture cost of about £4,722 per person.

2. Take the expectation over the hazard. For a gamma-distributed hazard the survival term has a closed form.

$$\mathbb{E}\left[e^{-ht}\right] = (1 + s\,t)^{-k}$$

where $t$ is the horizon in years and $s$ and $k$ are the scale and shape. With $t = 5$ this is 1.25 raised to the power −2, or 0.64, so the expected probability is 0.36 and the expected cost £4,320 per person.

3. Compare. The deterministic run overstates the probability by about 0.033, or about 393 rather than 360 fractures per 1,000 people, and the expected cost by about £402 per person. The probability is concave in $h$, so its expectation lies below its value at the mean; fracture-free survival $e^{-5h}$ is convex, so its expectation, 0.64, lies above the deterministic 0.6065.

4. Check with the expansion. The second derivative $-t^2 e^{-ht}$ is about −15.16 at the mean and the hazard variance is $k s^2 = 0.005$, so the correction is about −0.038 and the approximate expectation 0.356, close to the exact 0.36. A simulation of 10,000 draws would have a standard error of about 0.0019, as the probability has a standard deviation of about 0.187 across draws.

The bias would shrink with a more precise hazard and grow with a longer horizon, up to a horizon of about 20 to 25 years in this example, beyond which the absolute gap narrows again.

Expected costs from skewed patient-level data

Patient-level costs are usually skewed to the right, because a minority of patients have complications, reoperations or long stays. Thompson and Barber argued in 2000 that health care policy needs the total cost of treating all patients, which is the arithmetic mean cost multiplied by the number of patients, so the arithmetic mean is the informative summary. In their menorrhagia trial example, for endometrial resection, the median cost was £523, the geometric mean £683 and the arithmetic mean £790.

The statistical method decides which summary is compared. Thompson and Barber noted that a t test on untransformed costs compares arithmetic means, one on log costs compares geometric means, and the Mann-Whitney U test compares whole distributions; they judged t tests reliable when skewness is not too extreme or samples are moderately large, with bootstrapping as a check. Mihaylova and colleagues add that means compared on a transformed scale need back-transformation, commonly by Duan's smearing method when the error is homoskedastic, and review generalised linear models, which are estimated on the original cost scale and need no back-transformation. Thompson and Nixon found that cost-effectiveness conclusions can depend on how the upper tail beyond the observed data is modelled.

Worked example of the mean and median of a log-normal cost

The figures are illustrative. Annual costs after an admission follow a log-normal distribution with a median of £2,000 and a standard deviation of log cost of 1.0. The expected cost is calculated as follows.

$$\mathbb{E}[C] = e^{\mu + \sigma^2 / 2}$$

where $C$ is annual cost per patient, $\mu = \ln 2000 \approx 7.601$ is the mean of log cost and $\sigma = 1.0$ is its standard deviation. The median and geometric mean both equal $e^{\mu}$, £2,000.

1. Calculate the mean. The expected cost is £2,000 times $e^{0.5} \approx 1.6487$, about £3,297, or about 65% above the median. Exponentiating the mean log cost returns the median, and the factor $e^{\sigma^2/2}$ is Jensen's inequality for the convex exponential.

2. Locate the mean in the distribution. About 69% of patients cost less than the expected value, and the costliest 10% account for about 39% of total cost.

3. Set the budget. For 5,000 patients the expected annual budget is about £16.49 million, whereas the median or back-transformed log mean gives £10.0 million, a shortfall of about £6.49 million, or 39.3% of the true total.

4. Test the tail. With a standard deviation of log cost of 1.2 and the same median, the expected cost rises to about £4,109, so with marked skewness the mean depends on the modelled tail, a sensitivity of the kind Thompson and Nixon reported for the choice of cost distribution.

Variance as an expectation

Variance is itself an expected value, the expected squared distance of an outcome from its mean. It measures spread and recurs throughout uncertainty analysis. Formally, it is written as follows.

$$\operatorname{Var}(X) = \mathbb{E}\left[(X - \mathbb{E}[X])^2\right] = \mathbb{E}\left[X^2\right] - \left(\mathbb{E}[X]\right)^2$$

where $\operatorname{Var}(X)$ is the variance of $X$. In the fracture example the hazard variance of 0.005 largely sets the size of the Jensen gap, and the rearranged identity, $\mathbb{E}[X^2] = (\mathbb{E}[X])^2 + \operatorname{Var}(X)$, is the step used in the relapse example on the expected value analysis page. The variance of patient costs governs how precisely a trial estimates mean cost, and the variance of simulated outputs gives the Monte Carlo standard error. A decision taken on expected values does not use variance directly, a point about risk attitudes taken up on the expected utility page.

Expected value beside nearby terms

Several terms share the word "expected" or the idea of an average but answer different questions. A sample mean estimates an expected value from data, and a median describes a typical patient rather than a budget. Expected utility averages utilities of outcomes so that attitudes to risk enter, expected value analysis is the rule of choosing by expectations, and expected net benefit applies that rule at a stated threshold. The expected value of perfect information is the expected gain from resolving uncertainty before choosing, so there the expectation is taken of a gain in net benefit, not of a cost or a QALY total.

Misreadings of expected values

Two statistical misreadings are illustrated above. One is equating an expectation with the central value of a skewed distribution, when more than two thirds of patients in the log-normal example fall below the mean. The other is equating the function of mean inputs with the mean of the function, which overstates the fracture probability by about 9% of its expected value. A further error is to report a median, geometric mean or exponentiated mean log cost as the expected cost, understating the budget. Another is to average patient-level or subgroup ratios instead of forming a ratio of expectations, and a last is to treat a mean cost as secure when it rests on an extrapolated upper tail.

Sources

• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford: Oxford University Press; 2006.
• Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value in Health. 2012;15(6):835-842. doi:10.1016/j.jval.2012.04.014
• Casella G, Berger RL. Statistical Inference. 2nd ed. Pacific Grove, CA: Duxbury; 2002.
• Doubilet P, Begg CB, Weinstein MC, Braun P, McNeil BJ. Probabilistic sensitivity analysis using Monte Carlo simulation: a practical approach. Medical Decision Making. 1985;5(2):157-177. doi:10.1177/0272989X8500500205
• Jensen JLWV. Sur les fonctions convexes et les inégalités entre les valeurs moyennes. Acta Mathematica. 1906;30:175-193. doi:10.1007/BF02418571
• Mihaylova B, Briggs A, O'Hagan A, Thompson SG. Review of statistical methods for analysing healthcare resources and costs. Health Economics. 2011;20(8):897-916. doi:10.1002/hec.1653
• O'Hagan A, Stevenson M, Madan J. Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA. Health Economics. 2007;16(10):1009-1023. doi:10.1002/hec.1199
• Thompson SG, Barber JA. How should cost data in pragmatic randomised trials be analysed? BMJ. 2000;320(7243):1197-1200. doi:10.1136/bmj.320.7243.1197
• Thompson SG, Nixon RM. How sensitive are cost-effectiveness analyses to choice of parametric distributions? Medical Decision Making. 2005;25(4):416-423. doi:10.1177/0272989X05276862