Concept Architecture
Concept
Theoretically, Markov Chain Monte Carlo (MCMC) is a class of stochastic simulation algorithms used to generate samples from complex probability distributions that cannot be sampled directly. It combines Markov chain theory with Monte Carlo simulation, constructing a Markov chain whose stationary distribution is the target probability distribution. In health economics, MCMC forms the principal computational framework for Bayesian parameter estimation, probabilistic evidence synthesis, and uncertainty analysis, particularly where analytical solutions are unavailable.
Mathematically, MCMC constructs a Markov chain that satisfies conditions such as irreducibility, aperiodicity, and detailed balance so that its limiting distribution converges to the target posterior distribution. Samples generated after an initial burn-in period approximate independent draws from the posterior distribution and are used to estimate expectations, credible intervals, and other quantities of interest through Monte Carlo integration.
In practice, MCMC is implemented using algorithms such as Metropolis-Hastings, Gibbs sampling, or Hamiltonian Monte Carlo within Bayesian software including JAGS, WinBUGS, OpenBUGS, Stan, and NIMBLE. In health economic evaluation, posterior samples generated by MCMC are used to parameterise probabilistic sensitivity analyses, estimate treatment effects in network meta-analysis, quantify parameter uncertainty, and calculate decision uncertainty measures such as cost-effectiveness acceptability curves.
Purpose
Used to estimate posterior probability distributions, quantify parameter uncertainty, perform Bayesian evidence synthesis, and generate probabilistic inputs for health economic models.
Mathematical Formulae
Primary Formula
The target posterior distribution is
�(? | y) = L(y | ?)�(?) / ?L(y | ?)�(?) d?
where:
- �(? | y) = posterior distribution
- L(y | ?) = likelihood
- �(?) = prior distribution
Monte Carlo estimates are obtained from posterior samples:
E[f(?)] � (1/N) �???? f(??)
Supporting Formulae
Stationary distribution:
P(???? ? ??????) ? �(?)
Detailed balance condition:
�(i)P(i,j) = �(j)P(j,i)
Related Mathematical Methods
- Bayesian inference
- Monte Carlo integration
- Markov chains
- Metropolis-Hastings algorithm
- Gibbs sampling
- Hamiltonian Monte Carlo
- Bayesian network meta-analysis
- Probabilistic sensitivity analysis
Example
A Bayesian network meta-analysis estimates the relative effectiveness of four antihypertensive medicines.
Prior distributions are assigned to treatment effects and between-study heterogeneity.
An MCMC algorithm generates 50,000 posterior samples following a burn-in of 10,000 iterations.
Posterior samples estimate:
- Mean log-odds ratio = ?0.38
- 95% credible interval = ?0.56 to ?0.20
These posterior samples are propagated through a probabilistic Markov model to estimate incremental costs, QALYs, and the probability that each treatment is cost-effective at a willingness-to-pay threshold of �20,000 per QALY.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| RAND | =RAND() | Generate uniform random numbers for simulation algorithms |
| NORM.INV | =NORM.INV(RAND(),0,1) | Generate normally distributed proposal values |
| AVERAGE | =AVERAGE(B2:B50001) | Estimate posterior mean from sampled values |
| PERCENTILE.INC | =PERCENTILE.INC(B2:B50001,0.025) | Estimate lower credible interval |
| PERCENTILE.INC | =PERCENTILE.INC(B2:B50001,0.975) | Estimate upper credible interval |
VBA (Optional)
Automate iterative MCMC sampling, convergence monitoring, and export of posterior samples for probabilistic sensitivity analysis.
Sources
- Briggs AH, Claxton K, Sculpher MJ. Decision Modelling for Health Economic Evaluation. Oxford University Press; 2006.
- Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford University Press; 2015.
- Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. 3rd ed. CRC Press; 2013.
- Brooks S, Gelman A, Jones GL, Meng XL, eds. Handbook of Markov Chain Monte Carlo. CRC Press; 2011.
- ISPOR-SMDM Modeling Good Research Practices Task Force. Model Parameter Estimation and Uncertainty Analysis.
Related Concepts (4)
Library
Publications
1
Parameter Estimation and Uncertainty: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6 — Briggs, Weinstein, Fenwick, Karnon, Sculpher & Paltiel, Task Force Report 6 ed., 2012 (Value in Health / Medical Decision Making)
Best-practice guidance on parameter estimation and the characterisation of uncertainty in decision models, covering probabilistic sensitivity analysis, distributional choices, and correlation between parameters.
Journal ArticleView source →
Frequently Asked Questions (6)
What is Markov chain Monte Carlo?
A class of algorithms generating samples from a complex probability distribution by constructing a Markov chain that converges to that distribution.
Source: Gelfand & Smith 1990
How does Markov chain Monte Carlo sample a difficult distribution?
When a distribution cannot be sampled directly, Markov chain Monte Carlo builds a chain of successive draws in which each depends only on the one before, designed so that, after enough steps, the values it visits are distributed according to the target. Running the chain long enough yields a large sample from a distribution that could not be sampled any other way, and averages taken from that sample estimate the quantities of interest. Convergence to the target is what makes the trick work. Gilks and colleagues (1996) set out the approach.
Source: Gilks et al. 1996
How does Markov chain Monte Carlo work?
Markov chain Monte Carlo works by defining a Markov chain that moves between states in the parameter space such that its long-run, stationary distribution equals the target distribution of interest. The chain is run for many iterations; early samples, before convergence, are discarded as burn-in, and the remaining samples are treated as draws from the target. Because each state depends only on the previous one, the chain explores the distribution, spending time in regions in proportion to their probability, yielding representative samples.
Source: Gelfand & Smith 1990
Why is Markov chain Monte Carlo used in Bayesian analysis?
Markov chain Monte Carlo is used in Bayesian analysis because the posterior distribution is often complex, high-dimensional, and impossible to sample or integrate directly, yet inference requires summarising it. These methods generate samples from the posterior, from which quantities such as means, intervals, and probabilities are estimated. Their development, notably by Gelfand and Smith in statistics, made Bayesian analysis of complex models practical, so Markov chain Monte Carlo underlies much modern Bayesian computation, including in health economic and evidence synthesis models.
Source: Gelfand & Smith 1990
What are common Markov chain Monte Carlo algorithms?
Common Markov chain Monte Carlo algorithms include the Metropolis-Hastings algorithm, which proposes candidate moves and accepts or rejects them by a rule ensuring convergence, and Gibbs sampling, which updates each variable in turn from its conditional distribution. Gibbs sampling is a special case suited to problems where the full conditionals are known. Other variants improve efficiency for particular problems. These algorithms all construct a Markov chain converging to the target distribution, differing in how they generate the chain's moves.
Source: Hastings 1970
What are the challenges of Markov chain Monte Carlo?
Markov chain Monte Carlo faces challenges of convergence and mixing: the chain must be run long enough to reach and represent the target distribution, and it can converge slowly or explore the space poorly, especially with correlated parameters, so results may be unreliable if stopped too early. Assessing convergence is not straightforward and requires diagnostics. Choosing good proposals or parameterisations affects efficiency. These challenges mean careful application and checking are needed for Markov chain Monte Carlo results to be trustworthy.
Source: Gelfand & Smith 1990
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 9 Oct 2025
Content version: 1.0.0
Canonical Identity
- Term code
- HE-EM-MP-024
Stable URI · Machine-readable · Resolvable · CC BY 4.0