Concept Architecture
Concept
Theoretically, Markov Assumption is the fundamental assumption underlying Markov processes and Markov state-transition models whereby the future evolution of a process depends only on its current state and not on the sequence of events that preceded it. It is founded on stochastic process theory and enables complex dynamic systems to be represented using transition probabilities between discrete states. The assumption exists because it provides a mathematically tractable framework for modelling disease progression, treatment pathways and long-term health outcomes.
Mathematically, the Markov Assumption is represented by the conditional independence property stating that the probability distribution of the next state depends solely on the present state. This property allows transition probabilities to be represented by a transition probability matrix and applied recursively across successive cycles.
In practice, the Markov Assumption is assessed conceptually rather than estimated directly. Health economists determine whether disease history materially influences future transitions beyond the current health state. Where the assumption is violated, alternative approaches such as tunnel states, semi-Markov models or individual-level simulation may be required. The assumption underpins most cohort Markov models used in health technology assessment.
Purpose
Used to simplify the mathematical representation of disease progression by assuming that future transitions depend only on the current health state, facilitating state-transition modelling and long-term economic evaluation.
Mathematical Formulae
Primary Formula
P(X??? = j � X? = i, X???, ?, X?) = P(X??? = j � X? = i)
Supporting Formulae
Transition probability:
p?? = P(X??? = j � X? = i)
Transition matrix:
P = [p??]
n-step transition matrix:
P?�? = P�
Related Mathematical Methods
- Markov Chain
- Markov Cohort Model
- State Transition Model
- Semi-Markov Model
- Matrix Algebra
- Stochastic Processes
Example
A chronic disease model contains the states Healthy, Diseased and Dead. Under the Markov Assumption, a patient currently in the Diseased state has the same probability of progressing to Dead during the next annual cycle regardless of whether they entered the Diseased state one year or five years previously. If disease duration influences progression, the assumption is violated and a semi-Markov or tunnel-state model may be more appropriate.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| MMULT | =MMULT(B2:D4,G2:G4) | Apply transition matrices to state vectors |
| MMULT | =MMULT(B2:D4,B2:D4) | Calculate multi-cycle transition probabilities |
| SUMPRODUCT | =SUMPRODUCT(B2:D2,G2:G2) | Calculate expected state occupancy after one cycle |
| MINVERSE | =MINVERSE(B2:D4) | Matrix operations for advanced Markov analyses |
VBA (Optional)
Automate repeated Markov cycle calculations and scenario analyses while applying fixed transition probabilities across model cycles.
Sources
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation.
- Sonnenberg FA, Beck JR. Markov Models in Medical Decision Making. Medical Decision Making. 1993.
- Norris JR. Markov Chains.
- Puterman ML. Markov Decision Processes: Discrete Stochastic Dynamic Programming.
- ISPOR-SMDM Modeling Good Research Practices Task Force Reports.
Related Concepts (3)
Library
Publications
1
An Introduction to Markov Modelling for Economic Evaluation — Briggs & Sculpher, Vol. 13, No. 4 ed., 1998 (PharmacoEconomics)
The foundational tutorial paper introducing Markov (state-transition) models for health economic evaluation, covering health states, cycle length, transition probabilities and the calculation of expected costs and outcomes. Widely cited as the standard entry point to Markov modelling.
Journal ArticleView source →
Frequently Asked Questions (6)
What is the Markov assumption?
The defining property of a Markov model that the probability of a future transition depends only on the current state, not on prior history.
Source: Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322-338. doi:10.1177/0272989X9301300409.
How does the Markov assumption simplify a model?
By holding that the chance of each transition depends only on the patient's current state, the assumption frees the model from having to remember how a patient reached that state. It can then describe the whole disease with one set of transition probabilities per state, rather than a separate set for every possible history leading to it. This keeps the number of parameters manageable and the structure tractable. The simplification is the reason Markov models are so widely used, and also the source of their main limitation. Sonnenberg and Beck (1993) explain it.
Source: Sonnenberg & Beck 1993
Why is the Markov assumption made?
The Markov assumption is made because it greatly simplifies the model: by having transitions depend only on the current state, the model can be represented compactly by a set of states and transition probabilities, without tracking each patient's history. This makes cohort models efficient and transparent. The assumption is a reasonable approximation when the current state captures what matters for future risk, and it is the property that gives Markov models their simplicity and wide use in health economic evaluation.
Source: Sonnenberg & Beck 1993
When does the Markov assumption fail?
The Markov assumption fails when future transition probabilities depend on history, not just the current state, for instance when the risk of an event depends on how long a patient has been in a state, on past events, or on the number of previous occurrences. In such cases, two patients in the same state but with different histories face different risks, which a memoryless model cannot represent. Where history matters, the assumption is violated, so the model must be adapted or a different approach used.
Source: Sonnenberg & Beck 1993
How is the Markov assumption relaxed when needed?
When the Markov assumption does not hold, it can be relaxed by adding states that encode the relevant history, so that the state captures what would otherwise be memory, for example separate states for time since an event or number of occurrences. Tunnel states are used to represent time-dependent risk within the framework. Alternatively, an individual-level simulation, which can carry each patient's history, may be used. These approaches let history influence transitions while retaining the state-based structure or moving beyond it.
Source: Sonnenberg & Beck 1993
How does the Markov assumption relate to memorylessness?
The Markov assumption and memorylessness are the same property under different names: both state that future transitions depend only on the current state, with no influence from the path taken to reach it. Memorylessness describes the lack of dependence on history, and the Markov assumption is the modelling assumption that this holds. A Markov process is memoryless by definition, so the two terms express the defining feature of Markov models, that the present state alone determines the probabilities of future transitions.
Source: Sonnenberg & Beck 1993
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 7 Oct 2025
Content version: 1.0.0
Canonical Identity
- Persistent URI
- https://healtheconomics.wiki/concept/markov-assumption
- Term code
- HE-EM-MM-010
Stable URI · Machine-readable · Resolvable · CC BY 4.0