Topic

Markov and state-transition models

State-transition models divide a disease into health states and move a cohort or individual patients between them in fixed cycles, using transition probabilities. Costs and quality-adjusted life years accrue according to time spent in each state. Concepts include the Markov assumption, cycle length, half-cycle correction and tunnel states.

Concepts in this topic

• Absorbing StateAn absorbing state is a state in a Markov model that patients cannot leave once they enter it, usually death, so the whole cohort eventually ends there.
• Cohort ModelA cohort model follows the expected movement of a defined group through mutually exclusive health states over time.
• Cohort SimulationCohort simulation runs a Markov or other state-transition model by moving the expected shares of a hypothetical cohort between health states each cycle.
• Cohort TraceA cohort trace is the cycle-by-cycle record of a state-transition model’s expected cohort distribution across its health states under a specified strategy.
• CycleA cycle is a defined time interval during which a discrete-time state-transition model updates health-state occupancy and accumulates relevant outcomes and costs.
• Cycle LengthCycle length is the duration of one cycle in a Markov or other discrete-time state-transition model, such as a month or a year, set before the model runs.
• Half-Cycle CorrectionHalf-cycle correction is a within-cycle adjustment in discrete-time state-transition models that approximates time-accruing costs or health effects between recorded state-occupancy endpoints.
• Hidden Markov ModelA statistical model in which the true underlying state cannot be directly observed, only indicated indirectly and probabilistically by observable data.
• Markov AssumptionThe Markov assumption is that the next transition in a state-transition model depends only on the current state and model time, not on earlier history.
• Markov ChainA Markov chain is a random process moving between states in which the next state depends only on the current state. It underlies Markov cohort models.
• Markov Cohort ModelA Markov model tracking a hypothetical cohort collectively as proportions across health states over cycles, rather than simulating each patient separately.
• Markov ModelA Markov model is a memoryless state-transition model used in economic evaluation: people move between health states each cycle, accruing costs and QALYs.
• MemorylessnessThe defining property of a Markov process that future transition probabilities depend only on the present state, with no influence from the path taken.
• State OccupancyThe amount of time, typically in cycles, that a patient or cohort proportion spends within a given health state in a Markov model.
• State RewardA cost, utility, health outcome, or other consequence accrued while a modelled person occupies a specified state for a defined amount of time.
• State Space ModelA modelling framework representing a system's evolution over time through a defined set of possible states and transition probabilities between them.
• State Transition ModelA general term for a decision-analytic model, such as a Markov model, representing disease progression as movement between defined health states.
• Time-Homogeneous ModelA Markov model in which transition probabilities remain constant throughout the entire modelled time horizon, regardless of how many cycles have passed.
• Time-Inhomogeneous ModelA Markov model in which transition probabilities are allowed to vary over the time horizon, such as mortality risk rising with age.
• Transient StateA health state within a Markov model that a patient can leave and potentially return to, unlike an absorbing state such as death.
• Transition MatrixA transition matrix is the table of probabilities of moving between, or staying in, health states in one cycle of a Markov model, with rows summing to one.
• Transition RewardA cost or outcome value applied at the moment a patient moves between two health states, unlike a state reward accrued over time.
• Treatment Sequence ModelA decision-analytic model representing a patient's ordered progression through a series of treatments, moving to the next option after failure or discontinuation.
• Tunnel StateA tunnel state is one of a fixed sequence of one-cycle temporary states in a Markov model, letting risks, costs or utilities vary with time in a state.