Time-Homogeneous Model

Time-Homogeneous Model in Markov Cohort Analysis

In a time-homogeneous model, one transition matrix is applied in every cycle, so the chance of moving between health states depends only on the state a person occupies and never on how far the model has run. In economic evaluation it is the simplest form of Markov model and the usual starting point in modelling tutorials, because the whole path of the cohort follows from powers of a single matrix. The price of that simplicity is a hidden assumption that every exit from every state happens at a constant rate, which is rarely true of mortality over a lifetime horizon. A wrong choice changes life expectancy, QALYs gained and the ICER, most of all when the model extrapolates far beyond the data. Models whose probabilities change over the horizon are covered on the time-inhomogeneous model page.

Constant transition probabilities and the Markov property

Two separate conditions are often run together. The Markov property limits what a transition may depend on to the current state, and time homogeneity adds that those probabilities never vary between cycles.

$$\Pr(X_{t+1} = j \mid X_t = i) = p_{ij} \quad \text{for every cycle } t = 0, 1, 2, \dots$$

where $X_t$ is the health state occupied at cycle $t$, $i$ and $j$ index the states, and $p_{ij}$ is the transition probability for a move between them, written without a time subscript because it does not change. A model can satisfy the Markov property and still be time-inhomogeneous, as when background mortality is read from a life table cycle by cycle. The first condition has its own page, the Markov assumption.

Sonnenberg and Beck, in their practical guide to Markov models in medical decision making, classified Markov processes by whether the transition probabilities are constant over time. They called the special type with constant probabilities a Markov chain and noted that, with an absorbing state, its behaviour can be found as an exact solution by matrix algebra. Probability texts use the term more widely, as the Markov chain page explains. Alarid-Escudero and colleagues avoid the ambiguity by describing the simplest cohort models as time-independent (time-homogeneous), with transition probabilities and other model parameters fixed over the simulated time horizon.

The matrix stays constant while the number of events changes. Because the cohort alive shrinks every cycle, a constant probability of death produces fewer deaths in each later cycle, and the cohort trace keeps changing. Homogeneity is also a statement about transitions only, so costs or utilities may still vary by cycle.

One matrix raised to a power

With a single matrix, the state distribution after any number of cycles has a closed form. Grinstead and Snell show that entry $ij$ of the matrix power $P^n$ is the probability that a chain starting in state $i$ is in state $j$ after $n$ steps, and that a starting distribution $u$ becomes $uP^n$.

$$m_n = m_0 P^{n}$$

where $m_0$ is the row vector of the shares of the cohort in each state at the start, $P$ is the constant transition matrix and $m_n$ is the distribution after $n$ cycles. In a time-inhomogeneous model the power is replaced by a product of different matrices, one for each cycle.

The same constancy allows the fundamental matrix solution, which Sonnenberg and Beck describe as fast and "exact", giving the expected time spent in each state from each starting state. They list among its disadvantages the restriction to constant transition probabilities and wrote that discounting cannot be used with it because of its time variance. The Markov chain page works through the calculation, including a way to fold a constant discount rate into the matrix. A model can instead be run as a cohort simulation, which matches the matrix result for a homogeneous model when membership is counted at the start of each cycle.

The constant hazard inside every row

Each row of a homogeneous matrix fixes the distribution of time spent in that state. The probability of remaining in state $i$ for $k$ more cycles is $p_{ii}^{k}$, a geometric decline, so the expected number of cycles in the state, counting the cycle of entry, is as follows.

$$E[T_i] = \sum_{k=0}^{\infty} p_{ii}^{k} = \frac{1}{1 - p_{ii}}$$

where $T_i$ is the number of cycles spent in state $i$ on one visit, counted at the start of each cycle, and $p_{ii}$ is the constant probability of staying in $i$ for one more cycle. The manual for the msm package for R gives the continuous-time counterpart, in which a single period of occupancy in state $i$ of a time-homogeneous model has an exponential distribution with rate $-q_{ii}$ and mean $-1/q_{ii}$, where $q_{ii}$ is the diagonal entry of the constant transition intensity matrix.

A time-homogeneous model therefore makes the same assumption as an exponential survival model, a constant hazard for every exit. NICE Decision Support Unit Technical Support Document 14 describes the exponential as the simplest parametric model, with a hazard that is constant over time, and states that exponential models are only suitable if the observed hazard is approximately constant and non-zero.

Where constant probabilities fail in health economic models

The commonest failure is mortality from other causes. Sonnenberg and Beck observed that the probability of dying from unrelated causes increases continuously as the patient gets older, and that the greater accuracy of age-specific mortality rates had led to more reliance on time-variant probabilities. Alarid-Escudero and colleagues, in their tutorial on time-dependent cohort models, call a time-independent model limited in many applications, because background mortality changes as the cohort ages and the risk of cancer recurrence may change with time since diagnosis.

Two sources state the consequence. The ISPOR-SMDM report on state-transition modelling warns that, when extrapolating beyond a trial's duration, reductions in all-cause mortality should not be applied directly since background mortality increases with age. The msm manual notes that where the hazard of death increases with age, a homogeneous model underestimates the number of deaths when forecasting far into the future.

Time dependence comes in two forms, and they affect homogeneity differently. Alarid-Escudero and colleagues separate simulation-time dependence, on time since the start of the model such as age, from state-residence time dependence, on time already spent in a state. The first makes the model time-inhomogeneous. The second is handled with tunnel states, which enlarge the state space while each matrix entry stays constant, so a model whose only time dependence sits in tunnels remains time-homogeneous. State expansion can also represent age, but the tutorial warns that substantial expansion can be cumbersome and computationally expensive, and the ISPOR-SMDM report calls the extreme case state explosion. Time dependency in general has its own page.

Worked example with rising background mortality over a lifetime

All numbers in this example are illustrative. A cohort aged 60 with a chronic disease is modelled with Alive and Dead states in annual cycles. The hazard of death has two parts, a constant excess disease hazard of 0.04 a year and a background hazard of 0.008 a year at age 60 that rises by a factor of $e^{0.09}$, about 9.4%, each year of age. A new treatment halves the excess hazard to 0.02. Disease management costs GBP 1,000 a year in both arms, the treatment adds GBP 3,000 a year while alive, and utility is 0.80. Results are undiscounted until step 6, and life expectancy is the expected number of cycles alive counted at the start of each cycle less half a cycle.

1. Homogeneous model with age-60 mortality held fixed. Under standard care the total hazard is 0.008 + 0.04 = 0.048, so the annual probability of death is one minus $e^{-0.048}$, or 0.04687. Expected cycles alive are 1/0.04687 = 21.34, and life expectancy is 21.34 − 0.5 = 20.84 years. With treatment the hazard is 0.028, the probability is 0.02761, and life expectancy is 1/0.02761 − 0.5 = 35.72 years. The gain is 35.717 − 20.837 = 14.880 years.

2. Time-inhomogeneous model with age-specific background mortality. The probability of death in each cycle uses the hazard at the age reached at the start of the cycle.

$$q_t = 1 - \exp\left(-\left(0.008\, e^{0.09 t} + h_{\text{ex}}\right)\right)$$

where $q_t$ is the probability of death in cycle $t$, $t = 0$ is age 60 and $h_{\text{ex}}$ is the excess hazard, 0.04 under standard care and 0.02 with treatment. Under standard care $q_0 = 0.04687$, as in step 1. At age 80 the background hazard is 0.008 × 6.0496 = 0.0484, since $e^{1.8} = 6.0496$, giving $q_{20} = 0.0846$, and at age 90 it is 0.1190, giving $q_{30} = 0.1470$. A cohort run to absorption gives life expectancy of 14.48 years under standard care and 18.28 years with treatment, a gain of 3.80 years.

3. Compare the survival curves. At 10 years the two models are close, with 61.9% of the standard care cohort alive in the homogeneous model and 59.2% in the age-specific model. At age 100 the homogeneous model still has $e^{-0.048 \times 40} = 14.7\%$ of the standard care cohort alive and $e^{-0.028 \times 40} = 32.6\%$ of the treated cohort, against 1.0% and 2.2% in the age-specific model.

4. Fit the constant probability to 10-year survival. A modeller who chose the constant probability to reproduce the age-specific model's 10-year survival of 0.5922 would take one minus the tenth root of 0.5922, which is 0.05105, and life expectancy would be 1/0.05105 − 0.5 = 19.09 years. The treated arm, matched to its 10-year survival of 0.7233, gets 0.03188 and 30.87 years. The gain of 11.78 years is still about three times the age-specific figure, despite an exact match at 10 years.

5. Cost per QALY. In the step 1 model, costs are 1,000 × 20.837 = GBP 20,837 and 4,000 × 35.717 = GBP 142,868, and the QALY gain is 0.80 × 14.880 = 11.904. In the age-specific model, costs are 1,000 × 14.481 = GBP 14,481 and 4,000 × 18.278 = GBP 73,112, and the QALY gain is 0.80 × 3.797 = 3.0376.

$$\text{ICER}_{\text{hom}} = \frac{142868 - 20837}{11.904} = \frac{122031}{11.904} = 10251, \qquad \text{ICER}_{\text{age}} = \frac{73112 - 14481}{3.0376} = \frac{58631}{3.0376} = 19302$$

where each ICER is the incremental cost in GBP divided by the incremental QALYs of treatment against standard care, from the homogeneous and the age-specific model respectively.

6. Discount. At an illustrative 3.5% a year, the homogeneous model gives 1/(1 − 0.95313/1.035) = 12.64 discounted cycles alive under standard care and 16.53 with treatment, a gain of 3.89, against 2.12 from a discounted run of the age-specific model, and the discounted ICERs are GBP 17,194 and GBP 24,234 per QALY. Discounting narrows the gap, because the extra years in the homogeneous model arrive late, but does not close it.

7. Interpretation. Holding age-60 mortality fixed keeps a third of the treated cohort alive at 100, credits the treatment with 3.9 times the undiscounted life-years gained and roughly halves the undiscounted ICER. The models agree closely over the first few years, so short trial follow-up cannot tell them apart, and the difference comes from the extrapolated tail. Both models apply the treatment effect to disease-specific mortality only, in line with the ISPOR-SMDM advice.

When a constant matrix is a defensible choice

A time-homogeneous model is defensible when every transition, including death from other causes, can be argued to have an approximately constant hazard over the horizon. That is most plausible over short horizons, in which the cohort ages little. A homogeneous version of a model is also a useful check on code, since a cohort run should reproduce the expected stay $E[T_i]$ for each state.

Evidence can test the assumption before it is adopted. TSD 14 recommends log-cumulative hazard plots for testing whether a parametric model such as the exponential suits the observed hazard, and warns that extrapolation methods can differ widely when substantial extrapolation is needed, so the choice should be justified. The msm manual describes a Pearson-type goodness-of-fit test and transition intensities that are piecewise constant in time, reporting an analysis in which piecewise-constant intensities gave a greatly improved fit. Where the assumption is doubtful, an age-specific version of the model is a natural scenario analysis.

Misreadings of time homogeneity

Most errors come from confusing what is held constant. Others come from trusting a fit over a short period.

1. Treating homogeneity as the Markov property. An age-dependent model is still Markov, and a model with tunnel states can still be time-homogeneous.
2. Expecting constant event counts. A constant probability applied to a shrinking cohort gives fewer events in each later cycle.
3. Holding starting-age mortality fixed over a lifetime. As in the worked example, this leaves implausible shares of the cohort alive at very old ages and inflates life-years gained.
4. Trusting a good short-term fit. A constant probability matched to trial follow-up can reproduce observed survival and still overstate the extrapolated tail.

Sources

• Alarid-Escudero F, Krijkamp E, Enns EA, Yang A, Hunink MGM, Pechlivanoglou P, Jalal H. A tutorial on time-dependent cohort state-transition models in R using a cost-effectiveness analysis example. Medical Decision Making. 2023;43(1):21-41. https://doi.org/10.1177/0272989X221121747
• Grinstead CM, Snell JL. Introduction to Probability. 2nd rev. ed. Providence, RI: American Mathematical Society; 1997. Chapter 11.
• Jackson C. Multi-state modelling with R: the msm package. Version 1.8.2, 7 November 2024. MRC Biostatistics Unit, Cambridge. https://cran.r-project.org/web/packages/msm/vignettes/msm-manual.pdf
• Latimer N. NICE DSU Technical Support Document 14: Survival analysis for economic evaluations alongside clinical trials, extrapolation with patient-level data. Sheffield: Decision Support Unit, ScHARR, University of Sheffield; June 2011, last updated March 2013. https://sheffield.ac.uk/nice-dsu/tsds/survival-analysis
• Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. https://doi.org/10.1016/j.jval.2012.06.014
• Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322-338. https://doi.org/10.1177/0272989X9301300409