Transition Probability

Transition Probability: Markov Model Inputs, Rate Conversion and Competing Risks

Transition probabilities are the inputs that move a cohort between health states in a Markov model or other state-transition model, so an error in one of them is repeated in every cycle and carries through to projected survival, costs and QALYs. This page explains how each probability is tied to a starting state, a destination and a cycle length, and why every row of a transition matrix must sum to one. It then works through converting a probability to a different cycle length via rates, splitting competing exits from one state, deriving probabilities from hazard ratios and survival curves, and checking the results against the whole model.

Specify the start, destination, and interval

An entry $p_{ij}(t)$ describes movement from state $i$ at the start of a cycle to state $j$ at its end, conditional on being in $i$ at the start and on any stated covariates or treatment. It is not a general lifetime chance of ever visiting $j$. The state definitions, cycle length, age or time dependence, and competing destinations must accompany a numerical value.

For a row-stochastic transition matrix, all destinations from a starting state are mutually exclusive at the cycle boundary and exhaust the modelled possibilities. Each entry is between zero and one, and the row sums to one. A “remain in the same state” entry is a transition probability too, even when no clinical event occurs during the cycle.

$$0\leq p_{ij}(t)\leq1,\qquad\sum_j p_{ij}(t)=1.$$

Here $p_{ij}(t)$ is the probability of moving from state $i$ to state $j$ in cycle $t$, and the sum runs over every destination $j$, including $i$ itself.

A transition probability is a model parameter describing a specified conditional event. A transition matrix organises all such parameters for every start and end state under the same convention. The matrix structure does not by itself establish that a particular probability is clinically credible.

Change the time unit without dividing a probability

Probabilities are tied to their observation interval. Dividing an annual event probability by 12 generally does not give the exact monthly probability because people who have already had the event are no longer at risk in the same way. The ISPOR-SMDM good practice report on state-transition modelling lists transition probabilities and cycle length among the formal elements of a model and recommends that probabilities be converted from one time unit to another through rates. A simple conversion is possible under a constant event hazard and a single relevant exit over the interval.

For a constant hazard $h$ and interval length $\Delta$ in matching time units:

$$p(\Delta)=1-\exp(-h\Delta),\qquad h=-\frac{\ln\bigl(1-p(\Delta)\bigr)}{\Delta},$$

where $p(\Delta)$ is the probability that the event occurs within an interval of length $\Delta$ and $h$ is the constant hazard (rate) of that event.

In an illustrative case, a one-year probability of 0.20 under that assumption implies an annual hazard of $h=-\ln(1-0.20)\approx0.2231$ per year, and the corresponding monthly probability is:

$$p_{\mathrm{month}}=1-(1-0.20)^{1/12}\approx0.01842.$$

Here $p_{\mathrm{month}}$ is the one-month probability implied by the same constant hazard.

This is about 1.842% per month. Simple division gives 0.20 divided by 12, about 0.01667, and applying that figure for 12 months implies a one-year probability of about 0.1826 rather than 0.20. The conversion assumes the same hazard across the period and no additional competing event that alters the risk process. When a state has more than one exit, converting each exit separately with this single-event formula gives probabilities that no longer reproduce the source data, as the next section shows.

Account for competing exits from one state

A person who leaves a starting state for one destination may no longer be available for another destination during the same cycle, which is the setting of competing risks. The destination probabilities therefore need a coherent joint derivation. Under constant cause-specific hazards $h_1$ and $h_2$ for two absorbing exits over interval $\Delta$, the total exit hazard is $H=h_1+h_2$.

$$p_{i1}=\frac{h_1}{H}\bigl(1-\exp(-H\Delta)\bigr),\quad p_{i2}=\frac{h_2}{H}\bigl(1-\exp(-H\Delta)\bigr),\quad p_{ii}=\exp(-H\Delta),\qquad H>0.$$

Here $p_{i1}$ and $p_{i2}$ are the probabilities of moving from state $i$ to destinations 1 and 2 within the cycle, $p_{ii}$ is the probability of remaining in state $i$, and each exit receives its share $h_k/H$ of the total probability of leaving.

For illustration, let $h_1=0.12$ and $h_2=0.08$ per year and use a three-month cycle ($\Delta=0.25$ years). The total exit probability is $p_{\mathrm{exit}}=1-\exp(-0.05)\approx0.04877$. The coherent destination probabilities are about 0.02926 and 0.01951, while the probability of remaining is about 0.95123; the unrounded values sum to one.

The same illustrative hazards show the pitfall that arises when evidence reports several exits over a longer period. Over one year they give exit probabilities of about 0.10876 and 0.07251 and a probability of remaining of about 0.81873. Converting each annual exit to three months separately with the single-event formula gives about 0.02838 and 0.01864, both below the correct values, and leaves about 0.95298 remaining. Over four cycles that row keeps about 0.8248 of the cohort in the starting state rather than 0.8187, so the model no longer reproduces the one-year evidence. Under the same constant-hazard and absorbing-exit assumptions, the consistent route recovers the total hazard from the probability of remaining, $H=-\ln(0.81873)\approx0.2000$ per year, and allocates the exits in proportion to the observed exit probabilities (0.60 and 0.40).

The formula is a teaching case with constant competing hazards and absorbing exits. Reversible transitions, multiple within-cycle moves, time-varying hazards, and dependence on history may require a more general multistate method, such as deriving the full transition matrix from a continuous-time rate model or taking a root of an observed transition matrix by eigendecomposition. The eigendecomposition route needs evidence on all transitions from a state over the same follow-up period and can return negative or complex values, so the simple proportional allocation should not be treated as universal.

Derive probabilities from evidence that matches the model

Observed transition counts can provide a row estimate when complete state membership is observed at the same intervals as the model cycles. Dividing the count reaching destination $j$ by the count starting in state $i$ is then a starting point, with uncertainty and possible covariate differences still to assess. Censoring, deaths between visits, irregular follow-up, and unobserved intermediate transitions make naive count division unreliable.

Evidence may instead report survival curves, hazards, hazard ratios, or risks over a different period. Each requires an explicit mapping to the transition question, with assumptions about baseline risk, time dependence, and competing events. A hazard ratio is not itself a transition probability, and multiplying a probability by it gives the wrong treatment effect. Under proportional hazards over the cycle, the hazard ratio scales the baseline hazard, which gives:

$$p_T=1-(1-p_C)^{\mathrm{HR}},$$

where $p_C$ is the baseline transition probability for the cycle, $\mathrm{HR}$ is the hazard ratio for the treatment, and $p_T$ is the treated-arm probability for the same cycle. In an illustrative case with $p_C=0.20$ and a hazard ratio of 0.70, $p_T=1-(1-0.20)^{0.70}\approx0.1446$, compared with 0.14 from multiplying the probability directly.

When a parametric survival function $S(t)$ has been fitted to trial or registry data, the probability of the event during the cycle ending at time $t$, among those event-free at the start of that cycle, is:

$$p(t)=1-\frac{S(t)}{S(t-\Delta)},$$

where $\Delta$ is the cycle length and $t$ is time measured from the origin of the survival curve. Unless the fitted curve is exponential, this probability changes from cycle to cycle, which is how a time-dependent transition probability enters the model.

For health economic modelling, transition estimates should reflect the target population and treatment strategy. When risk depends on age or time since the start of the model, the probabilities can vary by model cycle while the model remains Markov. When risk depends on time spent in a state, the process is semi-Markov, and a cohort model usually represents it with tunnel states that record time since entry or by adding the needed history to the state definitions. In probabilistic sensitivity analysis, the probabilities leaving one state should be sampled jointly, for example from a Dirichlet distribution for multinomial count data or by sampling the underlying hazards, so that every simulated row stays valid.

Validate the probability in the whole model

A plausible single number may still produce implausible survival or state occupancy after repeated cycles. Each transition should therefore be checked against its source data and the projected cohort trace, and model outcomes inspected over the full horizon. Arithmetic checks are necessary but cannot substitute for clinical and external validation.

1. Name the conditional event. Record the starting state, destination, population, treatment, and cycle length.
2. Match data intervals. Explain any conversion from observed rates, risks, or survival data and its assumptions.
3. Check all competing destinations. Keep probabilities within range and ensure each starting-state row sums to one.
4. Test trajectories. Compare modelled events, survival, and state membership with appropriate observed or external evidence.
5. Propagate uncertainty coherently. Vary related probabilities or underlying hazards together and examine effects on costs and health outcomes.

Sources

• Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820.
• Fleurence RL, Hollenbeak CS. Rates and probabilities in economic modelling: transformation, translation and appropriate application. PharmacoEconomics. 2007;25(1):3-6.
• Chhatwal J, Jayasuriya S, Elbasha EH. Changing cycle lengths in state-transition models: challenges and solutions. Medical Decision Making. 2016;36(8):952-964.
• Jones E, Epstein D, García-Mochón L. A procedure for deriving formulas to convert transition rates to probabilities for multistate Markov models. Medical Decision Making. 2017;37(7):779-789.
• Gidwani R, Russell LB. Estimating transition probabilities from published evidence: a tutorial for decision modelers. PharmacoEconomics. 2020;38(11):1153-1164. Correction published 2020, doi:10.1007/s40273-020-00958-8.
• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford: Oxford University Press; 2006.