Concept Architecture
Concept
Theoretically, Markov Microsimulation is an individual-level state-transition modelling approach in which each simulated individual progresses independently through a Markov process according to transition probabilities that depend on the individual's current health state and, where appropriate, individual characteristics. Unlike cohort-based Markov models, microsimulation explicitly represents patient heterogeneity and allows outcomes to be estimated at the individual level before aggregation across the simulated population. In health economics, Markov microsimulation is used when patient-level variation materially influences clinical or economic outcomes.
Mathematically, Markov microsimulation represents each individual by a stochastic sequence of state transitions generated from a transition probability matrix. Random sampling determines each individual's movement between health states at every model cycle. Population-level estimates are obtained by averaging outcomes across a large number of independently simulated individuals, approximating the expected values of the underlying Markov process.
In practice, transition probabilities are estimated from clinical trials, observational studies or survival analyses, while individual patient characteristics may influence transition risks through subgroup-specific probabilities or regression models. Large numbers of simulated individuals are required to reduce first-order Monte Carlo error. Model outputs include expected costs, life years and quality-adjusted life years, together with measures of uncertainty obtained through probabilistic sensitivity analysis.
Purpose
Used to evaluate healthcare interventions by simulating disease progression for individual patients, capturing patient heterogeneity and estimating long-term clinical and economic outcomes at the population level.
Mathematical Formulae
Primary Formula
Individual state transition:
P(X?,??? = j | X?,? = k) = p??
where:
- X?,? = health state of individual i at cycle t
- p?? = probability of transition from state k to state j
Supporting Formulae
State update using transition probabilities:
???,??? = ???,???
Population mean outcome:
? = (1/N) ????? Y?
where:
- Y? = simulated outcome for individual i
- N = number of simulated individuals
Related Mathematical Methods
- Markov chains
- Monte Carlo simulation
- Microsimulation
- Matrix algebra
- Survival analysis
- Random number generation
- Probabilistic sensitivity analysis
Example
A microsimulation evaluates two treatments for type 2 diabetes by simulating 100,000 individual patients. Each annual cycle, every patient faces probabilities of myocardial infarction, stroke, renal failure or death based on their current health state. Random sampling determines individual transitions. After simulation, average discounted lifetime costs are estimated at �42,350 per patient and average quality-adjusted life expectancy at 11.84 QALYs. These population averages are compared across treatment strategies to estimate cost-effectiveness.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| RAND | =RAND() | Generate random numbers for individual state transitions |
| IF | =IF(RAND()<TransitionProb,NextState,CurrentState) | Simulate patient-specific transitions |
| INDEX | =INDEX(TransitionMatrix,CurrentState,NextState) | Retrieve transition probabilities |
| SUMPRODUCT | =SUMPRODUCT(Costs,Weights) | Aggregate simulated costs or health outcomes |
| AVERAGE | =AVERAGE(SimulatedOutcomes) | Estimate mean population outcomes |
VBA (Optional)
Automate large-scale patient-level simulations, repeated Monte Carlo iterations and aggregation of lifetime health and economic outcomes.
Sources
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford University Press; 2006.
- Siebert U, Alagoz O, Bayoumi AM, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Medical Decision Making. 2012;32(5):690?700.
- Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322?338.
- Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices?overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force. Medical Decision Making. 2012;32(5):667?677.
- NICE. Health Technology Evaluation Manual.
- Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford University Press.
Related Concepts (5)
Library
Publications
1
State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 — Siebert, Alagoz, Bayoumi, Jahn, Owens, Cohen & Kuntz, Task Force Report 3 ed., 2012 (Value in Health / Medical Decision Making)
Best-practice guidance for cohort and individual-based state-transition (Markov) models, covering development, analysis, validation and reporting.
Journal ArticleView source →
Frequently Asked Questions (6)
What is Markov microsimulation?
A Markov model simulating individual patients one at a time through their health state history, capturing heterogeneity a cohort model cannot represent.
Source: Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322-338. doi:10.1177/0272989X9301300409.
Why does Markov microsimulation give each patient a memory?
Microsimulation sends patients through the model one at a time, so each carries its own record of what has happened to it, which the model can consult when deciding what happens next. This memory lets transition probabilities depend on an individual's history, such as how long they have been ill or how many events they have already had, which a cohort model cannot represent because it holds only aggregate proportions. The individual memory is what makes the extra realism possible. Siebert and colleagues (2012) describe this feature.
Source: Siebert et al. 2012
How does Markov microsimulation work?
Markov microsimulation works by simulating patients individually: each patient starts in a state and, at each cycle, moves to a next state determined by sampling against the transition probabilities, so their path unfolds stochastically. Costs and effects accrue along each patient's history. The patient can carry attributes and accumulated history that influence their transitions. Many patients are simulated, and their outcomes are averaged to estimate expected results, so the model builds population outcomes from individual histories rather than tracking proportions.
Source: Sonnenberg & Beck 1993
What can Markov microsimulation capture that a cohort model cannot?
Markov microsimulation can capture individual heterogeneity, since each simulated patient can have distinct characteristics affecting their transitions, and patient history, since a patient can carry a memory of past events that influences future risk, which a memoryless cohort model cannot represent without adding many states. It can also handle time or history dependence naturally. These features let it model situations where individual variation and history matter, which is its main advantage over the aggregate cohort approach.
Source: Sonnenberg & Beck 1993
What are the costs of Markov microsimulation?
Markov microsimulation is more demanding than a cohort model: it requires simulating many individual patients, often many thousands, to average out the random variation in individual paths, making it computationally intensive, and its results carry Monte Carlo, or first-order, sampling error that a deterministic cohort model does not. It is also more complex to build, check, and communicate, and it requires data on individual characteristics. These costs mean microsimulation is used when the individual-level detail it captures materially affects the results.
Source: Sonnenberg & Beck 1993
Why does Markov microsimulation require many simulated patients?
Markov microsimulation requires many simulated patients because each individual's path is determined by random sampling, so any single patient's outcome is subject to chance, and the average over few patients would be noisy. Simulating many patients averages out this first-order variability, giving a stable estimate of the expected outcome. The number needed depends on the variability and the precision required; too few leave the results affected by sampling noise, so enough patients are simulated to make the estimates sufficiently stable.
Source: Sonnenberg & Beck 1993
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 7 Oct 2025
Content version: 1.0.0
Canonical Identity
- Term code
- HE-EM-MM-013
Stable URI · Machine-readable · Resolvable · CC BY 4.0