Continuous Model

Continuous Models, Rates and Cycle Approximations in Decision Models

Most cohort models in health economic evaluation advance in fixed cycles, yet the disease processes they represent unfold in continuous time, so each such model is an approximation of a continuous model. How close the approximation comes depends on the cycle length, on how rates are turned into cycle probabilities and on how costs and QALYs are counted within each cycle, and the error can carry through to incremental results. Continuous time is represented by hazard rates, intensity matrices and differential equations, and the fixed-cycle Markov model has a continuous-time counterpart, with transmission and partitioned survival models placed in relation to both, and a worked illness-death example follows. "Continuous" here refers to time, not to a continuous outcome variable in the statistical sense.

Continuous time compared with fixed cycles

A discrete model in the time sense moves the cohort forward in fixed steps and records state occupancy only at the step boundaries. A continuous-time model follows the cohort at every instant, so a person can progress or die at any point in a year. The distinction concerns time, and a model can have discrete health states and still run in continuous time. The Discrete Model page also uses "discrete" for distinct states and events, whereas here the term is about time.

The ISPOR-SMDM task force on conceptualising a model lists whether time should be represented as continuous or discrete among the problem characteristics that guide the choice of method, and notes that only a single transition may occur within a cycle of a state-transition model, so short cycles are needed when events are likely. Van Rosmalen and colleagues describe discrete-time evaluation as assuming that health states change only at the end of a cycle, while clinical events typically occur in continuous time. Choices about cycle length and within-cycle correction are therefore attempts to make a discrete-time model reproduce the continuous process it stands for.

Rates describe change at each instant

In continuous time the basic quantity is a rate, the instantaneous speed at which people leave a state, known in survival analysis as the hazard rate. A rate has units of events per person per unit of time and, unlike a probability, can exceed one. With a single exit and a constant hazard, the share still in the state follows a simple differential equation, as follows.

$$\frac{dS(t)}{dt} = -\lambda S(t), \qquad S(t) = e^{-\lambda t}$$

where $S(t)$ is the proportion of the starting cohort still in the state at time $t$ and $\lambda$ is the constant rate of leaving per unit of time. The probability of leaving within a cycle of length $\Delta$ is then given below.

$$p = 1 - e^{-\lambda \Delta}$$

where $p$ is the cycle transition probability and $\Delta$ is the cycle length in the same time unit as the rate. The rate-to-probability conversion page sets out the derivation. The ISPOR-SMDM task force on state-transition modelling states that probabilities should be converted between time units through rates, and that probabilities should never be called rates.

The transition intensity matrix for several states

With several states and several exits, the rates form a transition intensity matrix. The cohort distribution then evolves according to a system of linear differential equations. Their solution over any interval is a matrix exponential, as follows.

$$\frac{d\boldsymbol{\pi}(t)}{dt} = \boldsymbol{\pi}(t)\, Q, \qquad P(\Delta) = e^{\Delta Q}$$

where $\boldsymbol{\pi}(t)$ is the row vector of state occupancy at time $t$, $Q$ is the intensity matrix whose off-diagonal entry $q_{ij}$ is the rate from state $i$ to state $j$ and whose diagonal entry is minus the sum of the other entries in its row, and $P(\Delta)$ is the matrix of transition probabilities over an interval $\Delta$.

Entries of $P(\Delta)$ generally depend on several rates at once. Chhatwal, Jayasuriya and Elbasha show that the common practice of converting each probability separately is applicable only to two-state models and can produce significant error when there are competing risks. They also link the search for a shorter-cycle matrix to the embeddability problem, since not every discrete-time matrix corresponds to a continuous-time Markov chain.

Worked example of an illness-death model with three rates

The figures are illustrative. A cohort starts Well and can move to Sick or Dead, and Sick people can die. The constant annual rates are 0.30 from Well to Sick, 0.05 from Well to Dead and 0.40 from Sick to Dead.

1. Exact one-year probabilities. The model is progressive, so the matrix exponential has a closed form. Staying Well depends on the total exit rate of 0.35, as follows.

$$p_{WW} = e^{-0.35} = 0.7047$$

where $p_{WW}$ is the one-year probability of remaining Well. Being Sick at one year allows for people who fall Sick and then die within the year, as follows.

$$p_{WS} = \frac{0.30}{0.40 - 0.35}\left(e^{-0.35} - e^{-0.40}\right) = 6 \times (0.704688 - 0.670320) = 0.206208$$

where $p_{WS}$ is the one-year probability of moving from Well to Sick and being Sick at the year end. The remainder is death by either route, as follows.

$$p_{WD} = 1 - 0.7047 - 0.2062 = 0.0891$$

where $p_{WD}$ is the one-year probability of moving from Well to Dead.

Of the exact death probability of 0.0891, only about 0.0422, which is $(0.05/0.35) \times (1 - 0.7047)$, is direct death from Well. The other 0.0469 is people who fall Sick and die in the same year, a two-step path that an annual cycle allowing one transition cannot hold.

2. Annual probabilities built row by row from the rates. Two shortcuts are common. Separate conversion treats each rate as if it were the only risk, giving a Well to Sick probability of $p_{WS}^{sep} = 1 - e^{-0.30} = 0.2592$ and the following.

$$p_{WD}^{sep} = 1 - e^{-0.05} = 0.0488, \qquad p_{WW}^{sep} = 1 - 0.2592 - 0.0488 = 0.6920$$

where $p_{WD}^{sep}$ and $p_{WW}^{sep}$ are the separately converted probabilities of death and of staying Well. A competing-risk split shares the total exit probability in proportion to the rates, as follows.

$$p_{WS}^{cr} = \frac{0.30}{0.35} \times (1 - 0.7047) = 0.2531, \qquad p_{WD}^{cr} = \frac{0.05}{0.35} \times (1 - 0.7047) = 0.0422$$

where $p_{WS}^{cr}$ and $p_{WD}^{cr}$ are the competing-risk probabilities of moving to Sick and to Dead. This split gets the probability of staying Well right, but its death probability covers direct death only, so the 0.0469 who die through Sick within the year are missing.

3. Exact life expectancy. Expected time in each state follows directly from the rates, as follows.

$$L_W = \frac{1}{0.35} = 2.857, \qquad L_S = \frac{0.30}{0.35} \times \frac{1}{0.40} = 2.143$$

where $L_W$ and $L_S$ are the expected years spent Well and Sick, the first factor in $L_S$ is the share of people leaving Well who go to Sick, and the second is the mean time in Sick. Total life expectancy is 2.857 + 2.143 = 5.000 years.

4. The same model in cycles. A cohort simulation to a lifetime horizon gives the totals of alive time, in years, shown in the table.

5. Interpretation. Two errors are at work. With the exact matrix, counting at either end of the cycle is out by about half a year, and the half-cycle correction brings the total to 5.004 years. The second error lies in the probabilities, and the correction cannot touch it. Any annual matrix built row by row from the rates leaves out the two-step path through Sick, so it keeps too many people alive. Separate conversion partly offsets this by overstating direct death, which is why it overstates life expectancy by 0.300 years, about 6%, while the competing-risk split, which removes the offset, does worse at 5.486 years. Correcting for competing exits is therefore not enough; the remedy is the matrix exponential, which gives the exact annual matrix, or shorter cycles, which shrink the error to 0.023 years here. The last row shows the risk Chhatwal and colleagues describe, in which converting the exact annual matrix to monthly probabilities entry by entry with the traditional formula gives 4.874 years, further from the true value than the annual model it started from.

Accumulating discounted costs and QALYs over continuous time

A continuous-time model values a strategy by integrating state utilities or costs over time with a continuous discount factor. Nothing is counted at cycle boundaries, so no within-cycle correction is needed, although the integral is usually computed numerically. For QALYs the calculation is as follows.

$$E = \int_0^T \sum_{s} u_s \, \pi_s(t) \, e^{-\rho t} \, dt, \qquad \rho = \ln(1 + d)$$

where $E$ is expected discounted QALYs per person, $T$ is the time horizon, $u_s$ is the utility of state $s$, $\pi_s(t)$ is the proportion in state $s$ at time $t$, $\rho$ is the continuous discount rate and $d$ is the annual discount rate. State costs in place of utilities give expected discounted costs.

In a continuous model, numerical integration evaluates this integral, and a numerical solver produces $\pi_s(t)$ whenever the differential equations have no closed form, so step size and method affect the result. A discrete-time model instead replaces the integral with a weighted sum of occupancy at cycle boundaries, which makes within-cycle correction a numerical integration rule. Elbasha and Chhatwal used the continuous-time solution as the gold standard when comparing seven such rules, and Simpson's 1/3 rule converged fastest; the half-cycle correction page covers the methods in practice.

Continuous-time models used in health economic evaluation

Several model families work in continuous time but solve the time problem differently. Some calculate expected occupancy mathematically, some sample individual event times, and some integrate differential equations numerically.

Continuous-time Markov cohort models

These keep the states of a cohort model but replace the transition matrix with an intensity matrix. Van Rosmalen and colleagues give a mathematical solution for expected time in each state that handles age-dependent rates and discounting, with tunnel states for rates that depend on time in a state; in their hepatitis B example it was more accurate than the discrete-time version without much computation time. Alarid-Escudero and colleagues note that a cohort model can likewise be implemented with differential equations.

Continuous-time individual simulation

Discrete event simulation samples each simulated person's time to the next event, so the clock jumps from event to event; the task force on the method calls this a key asset. Soares and Canto e Castro point out that continuous-time simulation is inherently stochastic, so its precision depends on the number of individuals simulated, whereas a discretised model works with the cohort at the aggregate level.

Differential-equation transmission models

In a dynamic transmission model, the risk of infection depends on how many people are infectious at that moment, so flows between compartments feed back on each other. The ISPOR-SMDM task force on dynamic transmission modelling notes that most such models have been built with system dynamics, in which flows between compartments are represented by differential equations. It asks for the equations to be published, and warns that the Euler method often used in spreadsheets is less accurate than other numerical techniques. The SIR model and compartmental model pages give the equations.

Partitioned survival models as areas under curves

A partitioned survival model works from survival curves rather than from transition probabilities, and those curves can be read as continuous functions of time, although the model is mostly evaluated at cycle points. Technical Support Document 19 from the Decision Support Unit explains that state membership comes from non-mutually exclusive survival curves and that the area under the overall survival curve estimates mean life expectancy. Whenever those areas are computed at discrete time points, the integration and discounting questions above apply.

When continuous time changes the decision

A continuous-time formulation earns its extra work when a discrete approximation would distort incremental costs or QALYs. The guidance points to several warning signs.

• Long cycles relative to event rates. The state-transition task force asks for cycles short enough that an event occurs at most once per cycle, and shorter cycles when life expectancy is short.
• Competing exits or rapid sequences of events, where probabilities built row by row from rates misstate the process.
• Continuous risk functions and time-to-event data, which the conceptualisation task force says models representing time continuously can implement more accurately.
• Interaction between people, such as transmission.

Elbasha and Chhatwal found that a less accurate correction method can lead to a wrong decision. In stylised examples, Soares and Canto e Castro found that appropriate discretisation of rates with a continuity correction gave unbiased results even at longer cycle lengths, and argued that where both are possible a discretised cohort model is preferable to a continuous-time simulation. The question for an appraisal is whether the discrete approximation is close enough for the decision.

Continuous models and neighbouring concepts

Continuous time is a property that cuts across model types defined by other features, such as the unit of analysis or the source of transition information. A state transition model can be built in fixed cycles or in continuous time with the same states. Discrete event simulation runs in continuous time; its "discrete" refers to events, not to time. Most dynamic transmission models are written as differential equations, and partitioned survival models take their outputs as areas under continuous survival curves. A discrete-time cohort Markov model is the usual approximation of a continuous-time Markov process.

Where continuous models are misread

A common error is treating a rate and a probability as interchangeable, for example entering an annual rate of 0.30 as a transition probability. Another is building cycle probabilities row by row from rates when people can pass through more than one state within a cycle. The worked example shows that adjusting only for competing exits does not remove this error; the matrix exponential or shorter cycles are needed.

A continuous model still involves approximation, since differential equations are usually solved numerically and the solver and step size affect the answer. Applying an annual discount rate as if it were a continuous one also changes discounted totals. Finally, a continuous-time Markov model keeps the Markov property, so the risk of leaving a state does not depend on time already spent in it, even when rates change with age or calendar time. Constant rates are what make waiting times exponential. Where risk depends on time in state, tunnel states, a semi-Markov structure or individual simulation are needed.

Sources

• Alarid-Escudero F, Krijkamp E, Enns EA, Yang A, Hunink MGM, Pechlivanoglou P, Jalal H. An introductory tutorial on cohort state-transition models in R using a cost-effectiveness analysis example. Medical Decision Making. 2023;43(1):3-20. https://pmc.ncbi.nlm.nih.gov/articles/PMC9742144/
• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford: Oxford University Press; 2006.
• Chhatwal J, Jayasuriya S, Elbasha EH. Changing cycle lengths in state-transition models: challenges and solutions. Medical Decision Making. 2016;36(8):952-964. https://pmc.ncbi.nlm.nih.gov/articles/PMC5045797/
• Elbasha EH, Chhatwal J. Theoretical foundations and practical applications of within-cycle correction methods. Medical Decision Making. 2016;36(1):115-131. https://pubmed.ncbi.nlm.nih.gov/26092831/
• Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Möller J. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Value in Health. 2012;15(6):821-827. https://doi.org/10.1016/j.jval.2012.04.013
• Pitman R, Fisman D, Zaric GS, Postma M, Kretzschmar M, Edmunds J, Brisson M. Dynamic transmission modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-5. Value in Health. 2012;15(6):828-834. https://pmc.ncbi.nlm.nih.gov/articles/PMC7110742/
• Roberts M, Russell LB, Paltiel AD, Chambers M, McEwan P, Krahn M. Conceptualizing a model: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2. Value in Health. 2012;15(6):804-811. https://pmc.ncbi.nlm.nih.gov/articles/PMC4207095/
• Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. https://doi.org/10.1016/j.jval.2012.06.014
• Soares MO, Canto e Castro L. Continuous time simulation and discretized models for cost-effectiveness analysis. PharmacoEconomics. 2012;30(12):1101-1117. https://pubmed.ncbi.nlm.nih.gov/23116289/
• van Rosmalen J, Toy M, O'Mahony JF. A mathematical approach for evaluating Markov models in continuous time without discrete-event simulation. Medical Decision Making. 2013;33(6):767-779. https://pubmed.ncbi.nlm.nih.gov/23715464/
• Woods B, Sideris E, Palmer S, Latimer N, Soares M. NICE DSU Technical Support Document 19: Partitioned survival analysis for decision modelling in health care: a critical review. Sheffield: Decision Support Unit; 2017. https://orda.shef.ac.uk/articles/report/TSD_19_Partitioned_survival_analysis_as_a_decision_modelling_tool/33848653