Importance Sampling

Importance Sampling in Health Economic Simulation

Importance sampling estimates an average under one probability distribution, such as the expected cost per patient in a decision model, by simulating from a different distribution chosen by the analyst and multiplying each result by the ratio of the two densities. The deliberate distortion puts more draws where the answer is decided, and the weights undo it, so the estimate still refers to the original model. In health economics the technique reduces Monte Carlo error when expected costs or outcomes hinge on rare events that plain Monte Carlo simulation seldom reaches. The same reweighting lets existing probabilistic sensitivity analysis draws be reused under a new distribution, which is how some estimators of the expected value of sample information avoid rerunning the model. Owen calls importance sampling the hardest variance reduction method to use well, because a good proposal can cut the variance many times over while a poor one can inflate it, even to infinity.

The reweighting identity

Every Monte Carlo estimate in a model is an average of some output over the distribution of the random inputs. Owen calls the model's own distribution the nominal distribution, the alternative the importance distribution, and the ratio of their densities the likelihood ratio. Provided the importance density is positive wherever the output times the nominal density is non-zero, the expectation can be rewritten as an expectation under the alternative.

$$\mu = E_p[f(X)] = \int f(x)\,p(x)\,dx = \int \frac{f(x)\,p(x)}{q(x)}\,q(x)\,dx = E_q[f(X)\,w(X)], \qquad w(x) = \frac{p(x)}{q(x)}$$

where $X$ is the vector of random model inputs (event draws for a simulated patient, or one set of parameter values in a probabilistic analysis), $f(X)$ is the output of interest such as cost, QALYs or net benefit, $p$ is the density the model specifies for $X$, $q$ is the proposal density actually sampled, $w(X)$ is the importance weight and $\mu$ is the target expectation.

The estimator averages weighted outputs over independent draws from the proposal. Owen's Theorem 9.1 shows that it is unbiased and gives its variance.

$$\hat{\mu}_q = \frac{1}{n}\sum_{i=1}^{n} f(X_i)\,w(X_i), \qquad \operatorname{Var}(\hat{\mu}_q) = \frac{\sigma_q^2}{n}, \qquad \sigma_q^2 = E_q[f(X)^2\,w(X)^2]-\mu^2$$

where $X_1, \ldots, X_n$ are independent draws from $q$, $n$ is the number of draws, $\hat{\mu}_q$ is the importance sampling estimate and $\sigma_q^2$ is the variance of one weighted output under $q$.

Sampling from the proposal leaves the model's logic untouched. A simulated patient given a complication more often than the model says carries a weight below one through the rest of the history, and a patient spared it carries a weight slightly above one, so each later cost and QALY enters the average at its correct share.

Choosing where to send the draws

The variance formula shows what a good proposal looks like. When the output is never negative, a density proportional to output times nominal density gives zero variance, but Owen notes that it cannot be used, since building it requires the answer. The general result that the best density is proportional to the absolute output times the nominal density is credited to Kahn and Marshall. A practical proposal imitates that shape, favouring plausible inputs that move the output a great deal.

Tails matter more than the centre. The weight is large wherever the proposal is thin relative to the nominal density, so Owen calls light-tailed proposals dangerous and advises that, as a rule, the proposal should have tails at least as heavy as the nominal distribution. For a normal input a Student t proposal is a common choice, while a normal proposal for a Student t input can easily give infinite variance.

The number of uncertain inputs matters too. With independent inputs the weight is a product of one ratio per input, and Owen shows that its second moment grows exponentially with the number of inputs unless the variability of the per-input ratios shrinks sharply. A probabilistic model with dozens of distributions therefore tolerates only small changes to most of them, which favours tilting only the few parameters that drive the output. Owen also describes defensive mixtures, credited to Hesterberg, which draw a fixed share of samples from the nominal distribution itself. With a share of 0.3, no weight can exceed 1/0.3, or about 3.3.

Worked example with a rare costly complication

All numbers in this example are illustrative. A patient-level simulation includes a surgical complication with probability 0.002 per patient and a treatment cost of GBP 50,000. The quantity wanted is the expected complication cost per patient, 50,000 × 0.002 = GBP 100, estimated from n = 10,000 simulated patients.

1. Plain Monte Carlo. Each patient's complication cost is GBP 50,000 with probability 0.002 and zero otherwise, so its variance is 50,000 × 50,000 × 0.002 × 0.998 = 4,990,000 GBP². The standard error of the mean is √(4,990,000 / 10,000) = √499 = GBP 22.34, about 22% of the answer, and only 10,000 × 0.002 = 20 complications are expected in the run.

2. A proposal that oversamples the complication. The simulation is rerun with the complication probability set to 0.05, so about 500 complications are expected. A patient with the complication gets weight 0.002 / 0.05 = 0.04 and a patient without it gets 0.998 / 0.95 = 1.0505. The weighted cost is 50,000 × 0.04 = GBP 2,000 for a complication and zero otherwise.

3. Check the estimate is unbiased. The expected weighted cost is 0.05 × 2,000 = GBP 100, as required. The two weight terms contribute 0.05 × 0.04 = 0.002 and 0.95 × 1.0505 = 0.998, so the mean weight is 1, as it should be for any valid proposal.

4. Variance and standard error. The second moment of the weighted cost is 0.05 × 2,000 × 2,000 = 200,000, so the variance is 200,000 − 10,000 = 190,000 GBP². The standard error is √(190,000 / 10,000) = √19 = GBP 4.36. The variance ratio is 4,990,000 / 190,000 = 26.3, so plain Monte Carlo would need about 263,000 patients for the same precision.

5. A proposal pointing the wrong way. Setting the complication probability to 0.0005 gives a weight of 0.002 / 0.0005 = 4 and a weighted cost of GBP 200,000 per complication. The mean is still 0.0005 × 200,000 = GBP 100, but the second moment is 0.0005 × 200,000 × 200,000 = 20,000,000, the variance is 20,000,000 − 10,000 = 19,990,000 GBP², and the standard error rises to √1,999 = GBP 44.71. That variance is 19,990,000 / 4,990,000 = 4.0 times the plain Monte Carlo variance.

6. The zero-variance proposal. Setting the complication probability to 1 gives every patient weight 0.002 and a weighted cost of exactly 50,000 × 0.002 = GBP 100. It works only because 50,000 × 0.002 can already be computed without simulation, so in a real model the ideal proposal guides design but cannot be built.

7. Interpretation. The proposal in step 2 buys the precision of 263,000 ordinary runs for the cost of 10,000, and every proposal stays centred on GBP 100. The gain belongs to the complication cost alone, and the weights must be carried into every other output taken from the same run, including QALYs and the proportion of runs in which a strategy is cost effective.

Self-normalised weights for Bayesian targets

The plain estimator needs the full densities. A posterior distribution produced by Bayes' theorem is usually known only up to a constant, so the weights are divided by their own sum.

$$\tilde{\mu}_q = \frac{\sum_{i=1}^{n} f(X_i)\,w_u(X_i)}{\sum_{i=1}^{n} w_u(X_i)}$$

where $w_u(x)$ is an unnormalised weight, any fixed positive multiple of $p(x)/q(x)$, $\tilde{\mu}_q$ is the self-normalised estimate, and the other symbols are as before.

Owen shows that this ratio converges to the target with probability one when the proposal is positive wherever the target density is, a stronger condition than the plain estimator needs. The ratio is biased in finite samples, although it is exact for a constant output, so the weighted probabilities of an event and its complement add to one.

When the proposal is the prior distribution, the unnormalised weight of each draw is its likelihood, and the weighted draws approximate the posterior distribution. Heitjan and Li used importance sampling to approximate the posterior of a trial-based cost-effectiveness model, with gamma-distributed costs in each combination of treatment arm and binary effectiveness outcome, in a heart disease trial. Menzies and colleagues describe this reweighting, as sampling importance resampling, for model calibration, and report reasonable results when calibration targets are not overly strong relative to the prior.

Reusing probabilistic sensitivity analysis draws

Reweighting can answer a question about a different distribution without new model runs. Owen calls this what-if simulation and says it suits small perturbations, with the original sampling distribution having tails at least as heavy as the new ones. For normally distributed inputs shifted in mean, his population effective sample size is $n \exp(-\lVert\theta-\theta_0\rVert^2)$, with the shift measured in standard deviations, so a one standard deviation shift in one input keeps about 37% of the draws' information and a two standard deviation shift keeps under 2%.

Value of information analysis uses the same idea. Menzies estimated the posterior that a hypothetical study would produce by reweighting existing prior draws, so that EVSI could be computed from a conventional probabilistic analysis with no further model evaluations. His first algorithm gave results similar to the conventional two-level estimator when study evidence was weak but underestimated EVSI when the evidence was strong. The ConVOI guidance by Kunst and colleagues describes weights computed from the likelihood of the study data given the parameters and applied to the EVPPI simulations. It does not recommend the method for future studies of more than 1,000 participants, because the likelihood tends to 0 as the sample size increases. A larger study gives a narrower posterior, fewer prior draws fall inside it, and the weights collapse onto a handful of draws.

Reading the weights before reading the result

Unequal weights carry less information than the same number of equal ones. The Kish effective sample size, used by Owen and in the Menzies calibration tutorial, summarises this.

$$n_e = \frac{\left(\sum_{i=1}^{n} w_i\right)^2}{\sum_{i=1}^{n} w_i^2}$$

where $w_i$ is the weight of draw $i$, $n$ is the number of draws and $n_e$ is the number of equally weighted draws that would give the same precision for a generic mean.

Owen's population version is $n$ divided by the expected squared weight under the proposal. In the worked example it is 10,000 / 1.0485 = 9,537 for the 0.05 proposal and 10,000 / 1.0045 = 9,955 for the 0.0005 proposal, although the second is the worse estimator of the complication cost. Owen stresses that efficiency still depends on the output being averaged, and that a large effective sample size cannot prove success, because an important region may have been missed by every draw. The variance estimate uses the same weights, so badly skewed weights can hide their own problem.

Vehtari and colleagues note that estimates can be highly variable when the weights have a heavy right tail, which happens when the proposal misses parts of the target. Their Pareto smoothed importance sampling fits a generalised Pareto distribution to the largest weights to stabilise them, and reports a Pareto shape diagnostic of finite-sample convergence with effective sample size and Monte Carlo error estimates. A model report should state the proposal and the effective sample size.

Where importance sampling sits among simulation methods

Importance sampling corrects independent draws from a fixed proposal with weights. Latin hypercube sampling instead spreads equally weighted draws across each input's range, which improves coverage of typical values but gives a rare tail no more draws than its probability implies. Common random numbers sharpen the difference between strategies by pairing their random streams. Markov chain Monte Carlo reaches a posterior through dependent draws from a chain, while sequential Monte Carlo builds on importance sampling by updating weighted samples as data arrive.

Limits and common misreadings

Importance sampling reduces only simulation noise. Parameter and structural uncertainty stay as the model specifies them, and a precise estimate from a misspecified model is still wrong.

A proposal tuned to one output does not carry its gain to others. A proposal sharpened for the complication cost may help or harm the estimate of QALYs, so Owen advises that samples serving many outputs use a proposal positive wherever the nominal density is positive, with tails at least as heavy.

One error to avoid is dropping the weights in some summaries. Every mean, probability, scatter plot and cost-effectiveness acceptability curve built from importance-sampled draws must use the weights, since unweighted draws describe the proposal and not the model.

Sources

• Heitjan DF, Li H. Bayesian estimation of cost-effectiveness: an importance-sampling approach. Health Economics. 2004;13(2):191-198. https://doi.org/10.1002/hec.825
• Kahn H, Marshall AW. Methods of reducing sample size in Monte Carlo computations. Journal of the Operations Research Society of America. 1953;1(5):263-278. https://doi.org/10.1287/opre.1.5.263
• Kunst N, Wilson ECF, Glynn D, Alarid-Escudero F, Baio G, Brennan A, et al. Computing the expected value of sample information efficiently: practical guidance and recommendations for four model-based methods. Value in Health. 2020;23(6):734-742. https://doi.org/10.1016/j.jval.2020.02.010
• Menzies NA. An efficient estimator for the expected value of sample information. Medical Decision Making. 2016;36(3):308-320. https://doi.org/10.1177/0272989X15583495
• Menzies NA, Soeteman DI, Pandya A, Kim JJ. Bayesian methods for calibrating health policy models: a tutorial. PharmacoEconomics. 2017;35(6):613-624. https://doi.org/10.1007/s40273-017-0494-4
• Owen AB. Monte Carlo theory, methods and examples. 2013. Chapter 9, Importance sampling. https://artowen.su.domains/mc/
• Vehtari A, Simpson D, Gelman A, Yao Y, Gabry J. Pareto smoothed importance sampling. Journal of Machine Learning Research. 2024;25(72):1-58. https://arxiv.org/abs/1507.02646