Concept Architecture
Concept
Theoretically, Hidden Markov Model is a stochastic state-space model in which the underlying system evolves according to a Markov process, but the true states are not directly observable. Instead, the hidden states are inferred from observable outcomes generated probabilistically from each state. In health economics, hidden Markov models are used to represent diseases in which the true clinical status cannot be observed directly, such as latent disease progression inferred from biomarkers, diagnostic tests or symptom profiles.
Mathematically, a hidden Markov model consists of an unobserved Markov chain governing transitions between latent states together with an observation model that specifies the probability of each observable outcome conditional on the underlying state. Model parameters include the initial state distribution, the transition probability matrix and the emission probability matrix. These parameters are estimated from longitudinal data using likelihood-based methods.
In practice, hidden Markov models are fitted using repeated observations from clinical studies, registries or electronic health records. Parameters are commonly estimated using the Baum?Welch algorithm, while the most probable sequence of hidden states is inferred using the Viterbi algorithm. Estimated state probabilities may subsequently inform health economic models evaluating disease progression, treatment effects and long-term costs and outcomes.
Purpose
Used to infer unobservable disease states from observable clinical data, estimate disease progression and support health economic models in which underlying health status cannot be measured directly.
Mathematical Formulae
Primary Formula
Joint probability of hidden states and observations:
P(X, Y) = �?? ????? a?????? ????? b??(y?)
where:
- X = (x?, ?, x?) = hidden state sequence
- Y = (y?, ?, y?) = observed data
- �?? = initial state probability
- a?? = P(X? = j | X??? = i) = transition probability
- b?(y) = P(Y? = y | X? = i) = emission probability
Supporting Formulae
Transition probability:
a?? = P(X? = j | X??? = i)
Emission probability:
b?(y) = P(Y? = y | X? = i)
Forward recursion:
�?(j) = [?? �???(i)a??]b?(y?)
Related Mathematical Methods
- Markov chains
- State-space modelling
- Baum?Welch algorithm
- Viterbi algorithm
- Forward-backward algorithm
- Maximum likelihood estimation
- Expectation-Maximisation algorithm
Example
Patients with Alzheimer's disease are followed annually. The underlying disease severity is assumed to consist of three unobserved states: mild, moderate and severe. Annual cognitive assessment scores are observed but are imperfect indicators of disease severity. A hidden Markov model estimates transition probabilities between latent disease states and the probability of each assessment score conditional on the true disease state. These estimated disease trajectories are subsequently incorporated into a cost-effectiveness model to project long-term healthcare costs and quality-adjusted life years.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| MMULT | =MMULT(CurrentState,TransitionMatrix) | Calculate predicted latent state probabilities |
| SUMPRODUCT | =SUMPRODUCT(StateProbabilities,EmissionProbabilities) | Calculate observation likelihood |
| LN | =LN(Likelihood) | Compute log-likelihood during parameter estimation |
| EXP | =EXP(LogLikelihood) | Transform log-likelihood values when required |
VBA (Optional)
Automate iterative estimation of transition and emission probabilities using the Expectation-Maximisation procedure for longitudinal patient data.
Sources
- Rabiner LR. A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE. 1989;77(2):257?286.
- Zucchini W, MacDonald IL, Langrock R. Hidden Markov Models for Time Series: An Introduction Using R. 2nd ed. Chapman & Hall/CRC.
- Jackson CH, Sharples LD. Hidden Markov models for the progression of disease. Statistics in Medicine. 2002.
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford University Press; 2006.
- NICE. Health Technology Evaluation Manual.
- Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford University Press.
Related Concepts (3)
Library
Publications
1
State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 — Siebert, Alagoz, Bayoumi, Jahn, Owens, Cohen & Kuntz, Task Force Report 3 ed., 2012 (Value in Health / Medical Decision Making)
Best-practice guidance for cohort and individual-based state-transition (Markov) models, covering development, analysis, validation and reporting.
Journal ArticleView source →
Frequently Asked Questions (7)
What is a hidden markov model?
A statistical model in which the true underlying state cannot be directly observed, only indicated indirectly and probabilistically by observable data.
Source: Rabiner 1989
How does a hidden Markov model differ from an ordinary Markov model?
In an ordinary Markov model the state a patient occupies is taken to be known at each step. A hidden Markov model instead assumes the true state cannot be observed directly and is only inferred from imperfect signals it produces, such as test results that indicate but do not confirm the underlying condition. The model links the unseen states to the observable data probabilistically, allowing the likely state to be estimated. This suits conditions that can only be detected indirectly. Rabiner (1989) set out the framework.
Source: Rabiner 1989
What is a hidden Markov model?
A hidden Markov model is a statistical model in which the true underlying state cannot be directly observed, but is indicated indirectly and probabilistically by observable data. The system moves between hidden states according to a Markov process, and each state generates observable outputs with certain probabilities, so the observations give clues to the unseen state. Described in a well-known tutorial by Rabiner, hidden Markov models are used where the state of interest is not directly measurable but produces observable signals.
Source: Rabiner 1989
How does a hidden Markov model work?
A hidden Markov model has underlying states that follow a Markov process, with transition probabilities between them, and emission probabilities specifying how likely each observable output is in each state. The states are hidden, but the observations are seen. From a sequence of observations, methods infer the probabilities of the hidden states, the most likely state sequence, or the model's parameters. The model thus links an unobservable state process to observable data through the emission probabilities, allowing the hidden states to be estimated.
Source: Rabiner 1989
What distinguishes hidden states from observations?
In a hidden Markov model, the hidden states are the true underlying conditions of the system, which cannot be observed directly, while the observations are the measurable outputs that each state generates probabilistically. The same observation can arise from different states with different probabilities, so the observations do not reveal the state with certainty, only give probabilistic evidence about it. This distinction, between an unobservable state process and the observable signals it produces, is the defining feature of a hidden Markov model.
Source: Rabiner 1989
What problems do hidden Markov models solve?
Hidden Markov models address three main problems: evaluating the probability of an observed sequence given the model; decoding, finding the most likely sequence of hidden states given the observations; and learning, estimating the model's transition and emission probabilities from data. Standard algorithms solve each. These allow the hidden states to be inferred, the model fitted, and observations assessed, so hidden Markov models can be used to interpret sequential data in which the underlying state is not directly seen.
Source: Rabiner 1989
How are hidden Markov models applied in health?
In health, hidden Markov models are applied where a patient's true underlying state, such as a disease stage, is not directly observable but produces measurable signals, such as test results or symptoms, that indicate it probabilistically. They can model disease progression where the true state is uncertain but observed through imperfect measurements, infer the likely underlying state from observations, and account for measurement error. This suits situations where the state of interest must be inferred from indirect, noisy data rather than observed directly.
Source: Rabiner 1989
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 7 Oct 2025
Content version: 1.0.0
Canonical Identity
- Persistent URI
- https://healtheconomics.wiki/concept/hidden-markov-model
- Term code
- HE-EM-MM-009
Stable URI · Machine-readable · Resolvable · CC BY 4.0