Time-Inhomogeneous Model

Time-Inhomogeneous Model with Cycle-Specific Transition Matrices

In a time-inhomogeneous model the cohort moves forward under a transition matrix that can differ from cycle to cycle, so the chance of dying, progressing or recovering can depend on how far the model has run. A lifetime cost-effectiveness model needs this whenever risks change materially over the horizon, as background mortality does with age and as recurrence or progression risks often do with time since diagnosis or treatment. Treating those risks as constant can distort projected survival, costs and QALYs, more so over a long time horizon. The contrasting case, one matrix for every cycle, is the time-homogeneous model.

How cycle-specific matrices replace a single matrix

Sonnenberg and Beck, in their practical guide to Markov models, call the special case with constant transition probabilities a Markov chain. In the general case the probabilities may change, and their running example is death from unrelated causes, which rises as the patient gets older.

The tutorial on time-dependent cohort state-transition models by Alarid-Escudero and colleagues writes the matrix as a function of time, with every row of every cycle's matrix summing to one. The cohort is then advanced one cycle at a time.

$$m_{t+1} = m_t P_t, \qquad t = 0, 1, \dots, n_T - 1$$

where $m_t$ is the row vector giving the share of the cohort in each health state at cycle $t$, $P_t$ is the transition matrix that applies during cycle $t$, with entries $p_{ij}(t)$ for movement from state $i$ to state $j$, and $n_T$ is the number of cycles in the model. Applying the update repeatedly gives the state distribution after $n$ cycles.

$$m_n = m_0 P_0 P_1 \cdots P_{n-1}$$

where $m_0$ is the initial state vector and the matrices are multiplied in the order the cycles occur. When every $P_t$ equals one matrix $P$ the product collapses to $m_0 P^n$, the homogeneous case. The order matters, so averaging or reshuffling the matrices changes the cohort trace. Sonnenberg and Beck add that the exact matrix algebra solution, which yields the expected time in each state directly, is restricted to constant probabilities, so a time-inhomogeneous model is evaluated by running the cohort cycle by cycle.

Where the changing probabilities come from

The standard example of time dependence is background mortality taken from a life table. Sonnenberg and Beck describe two ways to supply it. A continuous function such as the Gompertz can generate each cycle's rate, or rates that follow no simple function can be stored in a table indexed by cycle number. Alarid-Escudero and colleagues convert age-specific life-table hazards to cycle probabilities, assuming a constant hazard within each year of age.

$$p_{HD}(t) = 1 - e^{-\mu(a_0 + t)}$$

where $p_{HD}(t)$ is the probability of moving from the healthy state to death in cycle $t$, $\mu(a)$ is the annual mortality hazard at age $a$, and $a_0$ is the age of the cohort at the start, here with a one-year cycle.

Disease risks often come from a parametric survival model fitted to trial or observational data. The same tutorial, in its tunnel-state example for time spent in a state, turns a Weibull curve into cycle probabilities by assuming a constant rate within each cycle, so the cycle rate is the increase in the cumulative hazard across that cycle.

$$H(t) = (\lambda t)^{\gamma}, \qquad \mu(t) = H(t) - H(t-1), \qquad p(t) = 1 - e^{-\mu(t)}$$

where $H(t)$ is the cumulative hazard at the end of cycle $t$, with $t$ in years for a one-year cycle, $\lambda$ and $\gamma$ are the Weibull scale and shape parameters, $\mu(t)$ is the rate that applies in cycle $t$, and $p(t)$ is the resulting transition probability. The last step is a rate-to-probability conversion.

The chosen curve fixes the shape of the time-varying hazard. NICE Decision Support Unit Technical Support Document 14 (TSD 14) explains that a Weibull hazard can rise or fall monotonically but cannot change direction, with the exponential as the special case of a constant hazard. Piecewise constant models impose no shape but, TSD 14 notes, are less useful for the extrapolated part of the curve, where hazards are not observed.

Model time and time spent in a state

Alarid-Escudero and colleagues separate two kinds of time dependency. Simulation-time dependence follows the time since the model started, such as age-specific mortality, since the whole cohort ages together, or time since diagnosis when the cohort starts at diagnosis. State-residence time dependence follows time spent in a state, which differs between members of the cohort who entered at different cycles.

A time-inhomogeneous model captures only the first kind. The second needs tunnel states, which enlarge the matrix so that each cycle spent in a state has its own row. The tutorial describes cycle-specific probabilities as a shortcut that works when the cohort experiences a time-varying process simultaneously as a function of time from the start, and advises modellers to consider simulation-time dependence first because tunnels add states. Recurrence risk that depends on time since surgery fits a cycle-indexed matrix when the whole cohort has surgery at the start, but risk that depends on time since a recurrence occurring partway through the model does not.

Choices that change the result

The way disease and background mortality are combined is one of the larger choices. The ISPOR-SMDM good practice report on state-transition modelling asks analysts to state the assumed relationship between them, because additive and multiplicative rates can give very different results. Alarid-Escudero and colleagues apply constant hazard ratios to the age-specific background rate, so mortality in their sick states also rises with age.

The duration of the treatment effect is a second choice. A hazard ratio held for a lifetime keeps the treated and untreated matrices apart in every cycle. The NICE manual (PMG36, section 4.6.20) asks for routine scenarios on extrapolation, including a treatment effect that stops or diminishes gradually over time, one sustained for people who continue treatment, and one sustained after discontinuation when lasting benefit is clinically plausible.

Cycle probabilities derived from a hazard assume a constant rate inside each cycle, which suits a short cycle length better than a long one. The curve family decides how risk behaves after the data end, and TSD 14 stresses that the plausibility of the extrapolated portion is likely to have a very large influence on estimated mean survival and can be justified using external data, biological plausibility or clinical expert opinion.

Worked example of adjuvant therapy with falling recurrence risk

All numbers in this example are illustrative. A cohort of 1,000 people aged 60 enters a three-state model at surgery for an early cancer, with states recurrence-free (F), recurrence (R) and dead (D) and a one-year cycle. Recurrence follows a Weibull curve with $\lambda$ = 0.01 per year and $\gamma$ = 0.5, so the risk is highest soon after surgery. Background mortality from an illustrative life table is 0.008, 0.009 and 0.010 per year at ages 60, 61 and 62. People with recurrence die with probability 0.15 per cycle.

1. Cycle rates from the cumulative hazard. $H(1) = (0.01 \times 1)^{0.5} = 0.1$, $H(2) = 0.1414$ and $H(3) = 0.1732$. The recurrence rates are therefore 0.1000, 0.0414 and 0.0318 in cycles 1 to 3, giving recurrence probabilities 0.0952, 0.0406 and 0.0313.

2. Combine with background death. Death is applied first, as in the Alarid-Escudero tutorial, and recurrence is conditional on surviving the cycle. Background death probabilities are 0.0080, 0.0090 and 0.0100. In cycle 1, F to R is 0.992 × 0.0952 = 0.0944, and staying in F is 1 − 0.0944 − 0.0080 = 0.8976. Cycles 2 and 3 give F to R of 0.0402 and 0.0310, with stay probabilities 0.9508 and 0.9590.

3. Run the cohort. In cycle 1, 1,000 × 0.0944 = 94.4 people recur, 1,000 × 0.008 = 8 die and 897.6 stay recurrence-free. In cycle 2, 897.6 × 0.0402 = 36.1 recur, 897.6 × 0.009 = 8.1 die from F and 94.4 × 0.15 = 14.2 die after recurrence, leaving 853.4, 116.3 and 30.2. In cycle 3, 853.4 × 0.0310 = 26.5 recur, 853.4 × 0.010 = 8.5 die from F and 116.3 × 0.15 = 17.4 die after recurrence. After three cycles the trace is 818.4 recurrence-free, 125.3 with recurrence and 56.2 dead.

4. A homogeneous version fitted to the same three years. A constant recurrence rate of 0.1732 / 3 = 0.0577 per year has the same three-year cumulative hazard, with recurrence probability 0.0561. With constant background mortality of 0.009, F to R is 0.991 × 0.0561 = 0.0556 every cycle, and staying in F is 0.9354. Recurrences fall at 55.6, 52.0 and 48.6 per cycle, and after three cycles the trace is 818.4 recurrence-free, 133.0 with recurrence and 48.5 dead.

5. Interpretation. Both models match the observed three-year recurrence-free share, yet the time-inhomogeneous model places recurrences earlier, so 7.7 more people per 1,000 have died by year 3. Beyond the data the two diverge further. Leaving death aside, the Weibull curve gives a ten-year recurrence-free share of $e^{-0.3162}$ = 0.729, against $e^{-0.5774}$ = 0.561 under the constant rate. A treatment hazard ratio of 0.7 held for ten years would avoid 0.801 − 0.729 = 0.072 recurrences per person under the Weibull curve and 0.668 − 0.561 = 0.107 under the constant rate. The homogeneous model credits the therapy with about half as many again, because it keeps baseline risk high in years the trial never observed.

Reading results in economic evaluation and HTA

The cycle-specific matrices hold most of a lifetime model's assumptions, so they are the first thing to check. The ISPOR-SMDM report suggests presenting intermediate outputs such as fixed-horizon risks and modelled survival curves, which can be compared with empirical survival data.

A good fit within the trial says little about the extrapolated cycles, as the worked example shows. PMG36 section 4.6.20 asks that assumptions used to extrapolate the treatment effect have external and internal validity and be reported transparently, so alternative hazard shapes and waning scenarios belong in the analysis.

Building the model in a spreadsheet or code

Sonnenberg and Beck describe storing rates or probabilities in a table indexed by cycle number and retrieving them as the model runs, which in a spreadsheet is a look-up keyed on cycle or attained age. Alarid-Escudero and colleagues hold the matrices in a three-dimensional array whose third dimension is time since the start, and note that updating the time-varying entries at each cycle instead would save memory. They also check that every probability lies between zero and one and that every row of every cycle's matrix sums to one. In a time-inhomogeneous model one faulty age band can create or lose part of the cohort in that cycle alone.

Boundaries with neighbouring models

A time-inhomogeneous model is still a Markov model in the strict sense, and the Markov assumption still applies. The ISPOR-SMDM report states that assumption as transition probabilities depending neither on past states nor on time spent in the current state, and a cycle index shared by the whole cohort is neither. A semi-Markov process, in which risk depends on time in the current state, is the case that needs tunnel states or another structure.

When history matters for many states at once, tunnel states multiply quickly. Alarid-Escudero and colleagues warn of a state explosion and suggest an individual-level model such as Markov microsimulation when the required number of states becomes unmanageable. Time-varying rewards are a separate matter. Sonnenberg and Beck give discounting as an example of utilities that vary with time, yet a model with discounting and constant transition probabilities remains time-homogeneous.

Common errors with time-varying transitions

Most errors come from giving the cycle index a job it cannot do or from trusting the fitted curve too far. Each one changes the matrices for many cycles at once.

1. Using one average mortality rate for a lifetime horizon. Over a long horizon, Sonnenberg and Beck note, mortality rises significantly in later cycles.
2. Indexing by cycle what depends on time in state. Risk after a mid-model event needs tunnel states or an individual-level model, since cycle number does not track it.
3. Holding disease-state mortality at its starting-age value. Where disease-state mortality adds an excess rate to background mortality or multiplies it by a hazard ratio, the combined rate must use the age-specific background rate in every cycle.
4. Reading the fitted curve as evidence beyond follow-up. The extrapolated matrices rest on the chosen hazard shape and need scenario analysis.

Sources

• Alarid-Escudero F, Krijkamp E, Enns EA, Yang A, Hunink MGM, Pechlivanoglou P, Jalal H. A tutorial on time-dependent cohort state-transition models in R using a cost-effectiveness analysis example. Medical Decision Making. 2023;43(1):21-41. https://doi.org/10.1177/0272989X221121747
• Latimer N. NICE DSU Technical Support Document 14: Survival analysis for economic evaluations alongside clinical trials, extrapolation with patient-level data. Decision Support Unit; June 2011, last updated March 2013. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD14-Survival-analysis.updated-March-2013.v2.pdf
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Section 4.6.20. https://www.nice.org.uk/process/pmg36/chapter/economic-evaluation
• Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. https://doi.org/10.1016/j.jval.2012.06.014
• Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322-338. https://doi.org/10.1177/0272989X9301300409