Absorbing State

Absorbing State: how death closes a Markov cohort

A Markov model that follows a cohort until it is fully accounted for needs at least one state that patients cannot leave, and in medical models that state is almost always death. The absorbing state is what allows such a model to finish: as cycles pass, more and more of the cohort moves into it, and the costs and health outcomes the model accumulates slow towards their final totals. How the absorbing state is specified shapes the time horizon, the handling of mortality and several technical corrections.

This article explains what makes a state absorbing and how it appears in the transition matrix, then works through a small three-state example. It covers why death is usually the only absorbing state, how background and disease mortality flow into it, what it means for the time horizon and the half-cycle correction, and the errors that most often arise.

What makes a state absorbing

Sonnenberg and Beck, in their widely cited practical guide to Markov models, state the requirement directly: for a Markov process to end, it must have at least one state that the patient cannot leave. These are called absorbing states because, after enough cycles, the entire cohort will have been absorbed by them. A transient state, by contrast, is one that patients can move out of.

Sonnenberg and Beck argue that in medical models the absorbing state must represent death, because death is the only state a patient cannot leave. Sonnenberg and Beck note that one death state is usually enough, since the utility of being dead is zero, but that more than one can be used if the model needs to track different causes of death.

How it appears in the transition matrix

A transition matrix lists the probability of moving from each state (rows) to each state (columns) in one cycle, and each row sums to one. For an absorbing state, the probability of staying is one and the probability of moving anywhere else is zero.

$$P = \begin{pmatrix} p_{WW} & p_{WS} & p_{WD} \ 0 & p_{SS} & p_{SD} \ 0 & 0 & 1 \end{pmatrix}$$

where the rows and columns are the states Well (W), Sick (S) and Dead (D), and $p_{WS}$, for example, is the transition probability from Well to Sick in one cycle. The last row, with 1 in the Dead column, is what makes Dead absorbing. The zero at the start of the second row means that, in this model, sick patients cannot recover to Well.

Worked example: a three-state cohort

The figures below are illustrative. A cohort of 1,000 people starts in Well. Each cycle, 85 per cent of the Well group stay Well, 10 per cent become Sick and 5 per cent die; 80 per cent of the Sick group stay Sick and 20 per cent die.

In cycle 2, Well is 850 × 0.85 = 722.5. Sick receives 850 × 0.10 = 85 new cases and keeps 100 × 0.80 = 80, giving 165. Dead keeps all 50 of its members and gains 850 × 0.05 = 42.5 from Well and 100 × 0.20 = 20 from Sick, giving 112.5. The three states still sum to 1,000, and the Dead column can only grow, because nothing ever leaves it. Run for enough cycles, this cohort simulation moves almost the whole cohort into Dead.

Mortality and the absorbing state

In practice the route into death usually has two parts. Sonnenberg and Beck describe the first as the probability of dying from unrelated causes, which rises continuously as patients age and is usually taken from a life table. The second is the extra mortality caused by the disease itself.

The ISPOR-SMDM guidance on state-transition modelling asks analysts to state the assumed relationship between disease-specific and background mortality. It also warns that, when extrapolating beyond a trial's duration, reductions in all-cause mortality should not be applied directly, because background mortality from other causes increases with age. The guidance suggests instead applying a relative reduction to disease-specific mortality, or subtracting life-table mortality from total mortality to estimate the reduction in disease-specific mortality. Because an assumption of additive rates can give very different results from a multiplicative one, the guidance also asks for the impact of this assumption to be assessed.

Time horizon, outcomes and the half-cycle correction

Because the cohort is absorbed gradually, the pace of absorption determines how long a lifetime model must run. The ISPOR-SMDM guidance notes that common approaches include modelling to an age of 120 years, or tracking the cohort until more than 99.9 per cent are dead, and that when an intervention affects mortality the time horizon should be lifetime. Life expectancy in the model is the time the cohort spends outside the absorbing state, so a treatment that extends survival shows up as slower absorption into Dead, and a lifetime horizon is needed to capture the life-years and QALYs gained from delayed deaths. Sonnenberg and Beck point out that the proportion in the Dead state is always slightly less than 100 per cent, because in every cycle some patients have a chance of remaining alive, so a stopping rule is needed.

No ongoing utility or cost accrues in the absorbing state. Sonnenberg and Beck argue that the incremental utility of the death state must be zero, because patients spend an unlimited time there, and a non-zero value would make the total utility of the model infinite. Costs in the death state are likewise normally set to zero in practice. The absorbing state also matters for the half-cycle correction: the ISPOR-SMDM guidance recommends applying it to costs and effectiveness in the first cycle, and in the final cycle as well when a lifetime horizon is not used. Sonnenberg and Beck make the related point that a simulation stopped before the cohort has been absorbed needs a correction in its final cycle.

Absorbing states, tunnel states and memory

A Markov model has no memory: its transition probabilities depend only on the current state (and, in a time-dependent model, on the cycle or age), not on how patients reached it or how long they have been there. The absorbing state fits this property naturally, since once a patient is dead nothing further can happen.

Where a model needs to reflect time since an event, such as a higher risk in the first months after surgery, it uses a tunnel state. Sonnenberg and Beck describe tunnel states as states that can be visited only in a fixed sequence, used to apply a temporary adjustment to utilities or transition probabilities that lasts more than one cycle. Tunnel states are transient; they lead patients through a sequence and on to other states, including the absorbing one.

Common modelling errors

A frequent error is a model with no reachable absorbing state, or one where the rows of the transition matrix do not sum to one, so that the cohort leaks or grows between cycles. Checking that every row sums to one and that the whole cohort is accounted for in every cycle catches the second problem; the first is caught by confirming that every living state has a route to Dead and that the Dead column rises across the Markov trace.

Another is to treat a good health outcome, such as cure, as absorbing when patients could in fact relapse or still die from other causes. In medical models death is usually the only state that truly cannot be left, and treating any other state as absorbing needs a clear justification. A third is to stop the model too early, before most of the cohort has been absorbed, which cuts off future costs and QALYs and can bias the comparison between treatments.

Sources

• Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322-338.
• Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820.
• Alarid-Escudero F, Krijkamp E, Enns EA, Yang A, Hunink MGM, Pechlivanoglou P, Jalal H. An introductory tutorial on cohort state-transition models in R using a cost-effectiveness analysis example. Medical Decision Making. 2023;43(1):3-20.
• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford University Press. 2006.