Convergence

Convergence in Health Economic Models and Bayesian Evidence Synthesis

In a health economic analysis, a result has converged when running more iterations, more probabilistic draws or more simulated patients would no longer change it by an amount that matters for the decision. The question arises in Bayesian evidence synthesis, probabilistic sensitivity analysis and patient-level simulation, and in estimation and calibration routines. It matters because the NICE technology appraisal manual (PMG36) asks for enough model simulations to minimise Monte Carlo error, and the NICE Decision Support Unit's guidance on network meta-analysis asks for particular care in checking MCMC convergence. Convergence says nothing about whether the model is right, since a chain can converge to the posterior of a misspecified model.

Two different questions behind the same word

The word covers two problems that are easily confused. In Markov chain Monte Carlo (MCMC), each draw depends on the one before and the chain only gradually forgets its starting values. Gelman and Rubin describe the draws as coming from a sequence of distributions that converge to the target distribution, so the question is whether the chains have reached it.

In a probabilistic sensitivity analysis, parameter sets are drawn independently, so there is no starting point to forget. The question there is precision, whether the average of a finite number of runs is close enough to the long-run value.

An MCMC run that has converged can still carry too much Monte Carlo error if too few draws are kept.

Checking MCMC convergence with multiple chains

The standard check compares chains. Gelman and Rubin recommend several independent sequences with starting points drawn from an overdispersed distribution, and NICE DSU Technical Support Document 2 (TSD 2) suggests at least three chains from widely different but sensible initial values, checking that each chain then samples from the same posterior.

The potential scale reduction factor

The usual diagnostic is the potential scale reduction factor, written $\hat{R}$. In the current textbook form, each chain is split in half after warm-up, so the statistic tests stationarity within chains as well as mixing between them.

$$\hat{R} = \sqrt{\frac{\widehat{\mathrm{var}}^{+}}{W}}, \qquad \widehat{\mathrm{var}}^{+} = \frac{n-1}{n}W + \frac{1}{n}B$$

where $m$ is the number of split chains, $n$ is the number of retained draws in each, $W$ is the average of the within-chain variances, $B$ is $n/(m-1)$ times the sum of squared deviations of the chain means from their overall mean, and $\widehat{\mathrm{var}}^{+}$ is a pooled estimate of the posterior variance. Before convergence, $W$ understates the posterior variance because no chain has yet covered the whole distribution, so $\hat{R}$ sits above 1 and declines towards 1 as the chains run longer.

The threshold for "near 1" has tightened. Gelman and colleagues, in Bayesian Data Analysis, describe 1.1 as a threshold they have generally been satisfied with. Vehtari and colleagues later showed that the traditional statistic can miss chains with heavy tails or unequal variances, and proposed a rank-normalised version. They recommend using the sample only when $\hat{R}$ is below 1.01, with at least four chains and a rank-normalised effective sample size above 400. They also report a separate tail effective sample size, because chains often resolve the centre of a distribution better than its tails.

Warm-up, run length and Monte Carlo error

Early draws reflect the starting values, so they are discarded. Bayesian Data Analysis calls this phase warm-up rather than burn-in and by default discards the first half of each chain. TSD 2 asks for the burn-in and the retained sample to be conservatively large, with both reported, and cites a guideline that the Monte Carlo error should be no more than 5% of the posterior standard deviation of each parameter of interest. Autocorrelated draws carry less information than independent ones, which the effective sample size measures.

Illustrative calculation of the potential scale reduction factor

Hypothetically, a Bayesian network meta-analysis estimates a log odds ratio with two chains, each split in half after warm-up, giving $m = 4$ split chains of $n = 500$ draws. Every split chain has a within-chain variance of 0.010, so $W = 0.010$.

Run 1, one split chain out of line. The split-chain means are $-0.52$, $-0.48$, $-0.50$ and $-0.30$, with an overall mean of $-0.45$. The squared deviations are 0.0049, 0.0009, 0.0025 and 0.0225, which sum to 0.0308, so $B = (500/3) \times 0.0308 = 5.1333$.

$$\widehat{\mathrm{var}}^{+} = \frac{499}{500} \times 0.010 + \frac{5.1333}{500} = 0.0202$$

where 0.010 is $W$ and 5.1333 is $B$.

$$\hat{R} = \sqrt{\frac{0.0202}{0.010}} = 1.42$$

where 0.0202 is the pooled variance and 0.010 is $W$. The value is far above 1.1 and 1.01, so the draws should not be pooled. The analyst would inspect the outlying chain, try other starting values or a reparameterisation, and run longer.

Run 2, chains mixed. After a longer run the split-chain means are $-0.47$, $-0.45$, $-0.44$ and $-0.44$, again with an overall mean of $-0.45$. The squared deviations sum to 0.0006, so $B = (500/3) \times 0.0006 = 0.1$.

$$\widehat{\mathrm{var}}^{+} = \frac{499}{500} \times 0.010 + \frac{0.1}{500} = 0.01018$$

where the inputs are as before, giving $\hat{R} = \sqrt{1.018} = 1.009$. This passes the 1.01 rule, but the draws are usable only if effective sample size checks also pass for every parameter that feeds the economic model.

Monte Carlo error and the number of probabilistic simulations

The NICE manual (PMG36, section 4.7.11) states that a probabilistic analysis should use enough model simulations to minimise the effect of Monte Carlo error, and that reviewing the variance around net benefits or ICERs as the number of simulations increases is a way of judging whether more runs are needed. Hatswell and colleagues note that guidance asking for a sufficient number of simulations, or for convergence, seldom defines what convergence means, and propose judging convergence from the variance of cost-effectiveness outcomes as a function of the number of simulations.

Because the parameter sets are independent, the precision of an average follows the usual standard error formula.

$$\mathrm{MCSE}(\bar{x}) = \frac{s}{\sqrt{N}}, \qquad \mathrm{MCSE}(\hat{p}) = \sqrt{\frac{\hat{p}(1-\hat{p})}{N}}$$

where $\bar{x}$ is the mean of an output such as incremental net monetary benefit over $N$ runs, $s$ is the standard deviation of that output across runs, and $\hat{p}$ is an estimated proportion such as the probability that a strategy is cost-effective. Quadrupling the number of runs halves the standard error. The standard error of a proportion is largest when $\hat{p}$ is 0.5, so the cost-effectiveness acceptability curve is least precise near the threshold where the curves cross.

The expected value of perfect information is harder still, because its per-run gain is zero whenever the strategy chosen on current information is also best, so the estimate rests on the runs where the choice would change. ICERs become unstable when incremental QALYs are near zero, which is one reason to monitor net benefit instead.

Sizing a probabilistic analysis for a decision

In this illustrative example, a cohort model compares a new treatment with standard care at a threshold of GBP 20,000 per QALY. After 1,000 probabilistic runs, mean incremental net monetary benefit is GBP 1,500, its standard deviation across runs is GBP 12,000, and the new treatment is cost-effective in 55% of runs.

1. Standard error of mean net benefit. $\mathrm{MCSE} = 12{,}000 / \sqrt{1{,}000} = 379.47$, so about GBP 379.
2. Interval for the mean. The 95% half-width is $h = 1.96 \times 379.47 = 743.8$, giving roughly GBP 756 to GBP 2,244. The whole interval is positive, so more runs are unlikely to change the sign of mean net benefit.
3. Probability of cost-effectiveness. $\sqrt{0.55 \times 0.45 / 1{,}000} = 0.0157$, so the 55% figure has a standard error of about 1.6 percentage points.
4. A closer decision. If mean net benefit had been GBP 300 with the same spread, the interval at 1,000 runs would run from about minus GBP 444 to GBP 1,044 and the sign would be uncertain. For the half-width $h = 1.96 \times 12{,}000 / \sqrt{N}$ to fall below 300, $\sqrt{N}$ must exceed 78.4, so $N$ must exceed 6,146.56, meaning 6,147 runs or more.
5. A precision target. To bring the standard error of mean net benefit down to GBP 100 needs $N = (12{,}000 / 100)^2 = 14{,}400$ runs.

The number of runs therefore depends on how close the decision is and which output is reported, not on a round number chosen in advance.

Patient-level simulation adds a second layer

Individual patient simulation has sampling error from the variation between simulated patients, which NICE DSU Technical Support Document 15 calls first-order uncertainty and which falls as more patients are simulated. A probabilistic analysis then usually needs two nested loops, an inner loop over patients and an outer loop over parameter sets.

TSD 15 states that its authors know of no widely used algorithm in health economics for choosing the number of patients, expects modellers to justify the number chosen, and suggests plotting incremental costs, QALYs and the cost per QALY with error bars against the number simulated, then checking that results are comparable under different random number seeds. It reports a result from O'Hagan and colleagues that, for models without patient interaction where the only aim is the mean, one patient per parameter set with as many parameter sets as possible is the most efficient use of computing time, although acceptability curves still need a reasonable number of patients.

Convergence of estimation and calibration routines

Fitting a parametric survival model by maximum likelihood estimation often uses a routine such as the Newton-Raphson method, and a message that the optimiser has converged means only that its stopping rule was met. A flat likelihood or a parameter on its boundary can still return estimates, so a fit is worth repeating from different starting values.

Calibration raises the same issue. Vanni and colleagues set out calibration in seven steps and treat the criteria for accepting a parameter set as a good fit, which they call convergence criteria, separately from the stopping rule that ends the search. A 2025 scoping review of 117 studies of cancer simulation models by Zhang and colleagues found that acceptance criteria were reported in 53 studies and a stopping rule in 46, so many published calibrations do not say why the search stopped.

What convergence does and does not establish

Convergence describes the computation and says nothing about whether the model is right. A chain can converge to the posterior of a misspecified model. Apparent convergence can also mislead, and TSD 2 gives an example in which one set of starting values converges, but to a different posterior. More iterations reduce Monte Carlo error, but they do not narrow the posterior itself, which reflects the evidence and the prior. In a cohort model, the movement of the trace towards an absorbing or stationary state is a property of the Markov chain, unrelated to whether a probabilistic analysis has run long enough.

Common errors when reporting convergence

Most errors treat one summary as proof. A report is easier to check when it names the diagnostic, its threshold, the number of chains or runs and the output tested.

• One chain only. One chain cannot show that another start would settle elsewhere.
• The old threshold alone. An $\hat{R}$ below 1.1 can appear before convergence, hence the newer 1.01 rule with effective sample size checks.
• A round number of runs with no check. The variance of net benefit should be shown to be small relative to the decision.
• Checking the mean only. Stable mean costs and QALYs do not imply a stable acceptability probability or EVPI.
• Reading an optimiser message as validation. A converged fit or calibration may still be implausible or externally invalid.

Sources

• Davis S, Stevenson M, Tappenden P, Wailoo A. NICE DSU Technical Support Document 15: cost-effectiveness modelling using patient-level simulation. Sheffield: Decision Support Unit, ScHARR, University of Sheffield; 2014. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD15_Patient-level_simulation.pdf
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. Sheffield: Decision Support Unit, ScHARR, University of Sheffield; 2011, last updated September 2016. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD2-General-meta-analysis-corrected-2Sep2016v2.pdf
• Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. 3rd ed. Boca Raton: CRC Press; 2013. Chapter 11. http://www.stat.columbia.edu/~gelman/book/BDA3.pdf
• Gelman A, Rubin DB. Inference from iterative simulation using multiple sequences. Statistical Science. 1992;7(4):457-472. https://doi.org/10.1214/ss/1177011136
• Hatswell AJ, Bullement A, Briggs A, Paulden M, Stevenson MD. Probabilistic sensitivity analysis in cost-effectiveness models: determining model convergence in cohort models. PharmacoEconomics. 2018;36(12):1421-1426. https://doi.org/10.1007/s40273-018-0697-3
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. Section 4.7.11. https://www.nice.org.uk/process/pmg36/chapter/economic-evaluation-2
• Vanni T, Karnon J, Madan J, White RG, Edmunds WJ, Foss AM, Legood R. Calibrating models in economic evaluation: a seven-step approach. PharmacoEconomics. 2011;29(1):35-49. https://doi.org/10.2165/11584600-000000000-00000
• Vehtari A, Gelman A, Simpson D, Carpenter B, Bürkner PC. Rank-normalization, folding, and localization: an improved $\hat{R}$ for assessing convergence of MCMC. Bayesian Analysis. 2021;16(2):667-718. https://doi.org/10.1214/20-BA1221
• Zhang Y, Lipa N, Alagoz O. A scoping review on calibration methods for cancer simulation models. Medical Decision Making. 2025;45(8):965-975. https://doi.org/10.1177/0272989X251353211