Signature
f_d = p_ii^(d - 1) * (1 - p_ii); G_d = p_ii^(d - 1); F_d = 1 - p_ii^d; d_star = log(0.5) / log(p_ii)
| Inputs | Definition | Unit |
|---|---|---|
p_ii | Probability of staying in state i for another cycle, the same in every cycle of the stay, above zero and below 1 | probability per cycle |
d | Length of the stay in whole cycles, counting the cycle of entry, 1 or more | cycles |
f_d | Probability that one visit to the state lasts exactly d cycles, counting the cycle of entry | probability |
|---|---|---|
G_d | Probability that the stay lasts at least d cycles, so that the person is still in the state in cycle d of the stay | probability |
F_d | Probability that the stay has ended within d cycles of entry | probability |
d_star | Point in cycles at which the probability of having left reaches one half; the median stay is the smallest whole number of cycles at or above it | cycles |
Function
Memoryless transition property of a Markov state and the sojourn-time distribution it implies
Maps the state a person occupies at cycle t, the cycle itself and the path H_t before it to the probability of each next state, under the restriction that the path adds nothing once the current state and cycle are known. With a constant probability p_ii of staying in state i the restriction makes the number of cycles spent in i on one visit geometric, and in continuous time a constant exit rate makes the time spent in i exponential, so in both cases the chance of leaving does not depend on the time already spent in the state. The package gives the geometric distribution of a stay with its median, the residual-stay identity behind memorylessness with a check of a survival curve for time since entry, the exponential residual stay and median, and the gap between the discrete and continuous mean stays at the same rate. Existing records are cited and not restated. The mean episode length 1 divided by 1 minus p_jj is HE-FM-MKCH-002, the cohort update is HE-FM-MM-001 with the trace totals of HE-FN-CSIM-001 and HE-FM-CSIM-001, rate and probability conversion is HE-FN-TP-001 with HE-FM-TP-001 to HE-FM-TP-004 and HE-CF-TP-001 and HE-CF-TP-002, exponential survival and the mean 1 divided by lambda are HE-FM-BTH-003 and HE-FM-CONT-002, the fundamental matrix is HE-FM-ABS-001 with its discounted form HE-FM-MKCH-001 and HE-CF-MKCH-001, the six-month root of an annual matrix is HE-FM-TMAT-001, and relaxing the assumption with tunnel states is HE-FN-TUN-001, HE-FM-TUN-001 to HE-FM-TUN-003 and HE-CF-TUN-001. Notation follows the Markov Assumption article.
Try this function
Implementations
Excel
Geometric Markov sojourn probabilities and median stay from named cells
With the stay probability in StayProb and the stay length in StayCycles, the four formulas return f_d, G_d, F_d and the median stay as a whole number of cycles, held in a cell named MedianStay. ROUNDUP to no decimals gives the smallest whole number at or above d_star, and subtracting 1E-9 first stops rounding noise in LN from pushing an exact whole number up by one.
=StayProb^(StayCycles-1)*(1-StayProb); =StayProb^(StayCycles-1); =1-StayProb^StayCycles; =ROUNDUP(LN(0.5)/LN(StayProb)-1E-9,0)
Assumptions
Stay probability unchanged across the cycles of one Markov state visit
The probability p_ii is the same in every cycle that the person spends in the state, as in a time-homogeneous model, which Sonnenberg and Beck call a Markov chain. In a time-inhomogeneous model the stay probability can change with model time, and the probability of staying at least d cycles becomes the product of the stay probabilities of the cycles concerned, so the geometric form holds only while they are equal.
Cycle of entry counted in the length of a Markov state visit
The count d includes the cycle of entry, so the shortest stay is one cycle, as in the article. A trace counted at the end of each cycle from model start credits a person who starts in the state with p_ii/(1-p_ii) cycles, one fewer, which gives the article's 0.88/0.12 = 7.333 event-free years; the counting weights of a trace are HE-FM-CSIM-001.
One uninterrupted visit described by the geometric Markov sojourn
The distribution describes a single visit. A state that can be left and entered again has several episodes, each with this distribution, and HE-FM-MKCH-002 gives the expected number of episodes and the total time across them.
Worked examples
Six-cycle stay in the article's memoryless post-event state
In the article's memoryless model the post-event state keeps 0.90613 of its members each year, a death probability of 0.09387. A stay of at least six cycles, the entry year and five survived years, has probability 0.90613 to the power 5, about 0.6109, the five-year post-event survival of 0.61088 to which the state was fitted. A stay of exactly six cycles has probability about 0.0573 and about 0.4465 of stays have ended within six cycles, while d_star is about 7.03, so the median stay is 8 cycles (computed here for illustration).
p_ii = 0.90613; d = 6; f_d = 0.0573; G_d = 0.6109; F_d = 0.4465; d_star = 7.0319
One-cycle stay in the article's memoryless post-event state
A stay of exactly one cycle, the cycle of entry only, has probability 1 minus p_ii, 0.09387. Applied to the 10% of the cohort with an event in year one it gives the article's 0.0094 of the cohort dying in year two, against 0.025 with the tunnel state, whose first post-event death probability is 0.25. At d = 1 the limiting values are G_d = 1 and F_d equal to f_d.
p_ii = 0.90613; d = 1; f_d = 0.0939; G_d = 1; F_d = 0.0939; d_star = 7.0319
Median and six-cycle stay in the article's event-free state under standard care
The event-free state under standard care keeps 0.88 a year. Its d_star is about 5.42, so the median stay is 6 cycles, against a mean of 1 divided by 0.12, about 8.333 cycles, from HE-FM-MKCH-002. About 53.6% of stays have ended within six cycles (computed here for illustration).
p_ii = 0.88; d = 6; f_d = 0.0633; G_d = 0.5277; F_d = 0.5356; d_star = 5.4223
Common errors
One-cycle shift between counting conventions for a Markov state stay
The mean stay counting the cycle of entry is 1 divided by 1 minus p_ii, 8.333 cycles for the event-free state at 0.88, while a trace counted at the end of each cycle from model start credits p_ii/(1-p_ii), 7.333. Comparing a mean taken under one convention with a model run under the other misstates the stay by a full cycle, about 12% here.
Mean stay of a memoryless Markov state read as its typical stay
A geometric stay is skewed to the right. With p_ii = 0.90613 the mean is about 10.65 cycles but the median is 8, and 62.7% of stays have ended within 10 cycles (computed here for illustration), so quoting the mean as the typical stay overstates it.
Sources
Grinstead and Snell geometric distribution used for a Markov state sojourn
Grinstead CM, Snell JL. Grinstead and Snell's Introduction to Probability. The CHANCE Project version of 4 July 2006, based on the 2nd edition published by the American Mathematical Society. Chapter 5, section 5.1, Geometric Distribution, pages 184 to 186, which defines T as the number of trials up to and including the first success with P(T = j) = q^(j-1) p for j = 1, 2, 3 and onwards, gives P(T > k) = q^k in Example 5.1, and simulates T as the least integer at or above log(1 minus rnd) divided by log q, the rule applied here to the median with rnd at one half. Taking the exit from the state as the success gives p = 1 minus p_ii.
Sonnenberg and Beck on the Markov property and constant-probability Markov chains
Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4) 322-338. Page 326, The Markov Property, which states that the behaviour of the process after any cycle depends only on its description in that cycle, so that the process has no memory for earlier cycles and all patients in a state have the same prognosis whatever their previous histories, and page 325, Types of Markov Processes, which calls the process with transition probabilities constant over time a Markov chain. The geometric length of a stay follows from these statements and is not itself stated in the paper.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0