Signature
R_ab = p_ii^(a + b) / p_ii^a; h_d = p_ii^(d - 1) * (1 - p_ii) / p_ii^(d - 1)
| Inputs | Definition | Unit |
|---|---|---|
p_ii | Probability of staying in state i for another cycle, the same in every cycle of the stay, above zero and below 1 | probability per cycle |
a | Number of cycles of the stay already completed, counted from entry to the state, 0 or more | cycles |
b | Number of further cycles over which the person must stay, 1 or more | cycles |
d | Cycle of the stay, counting the cycle of entry as 1 | cycles |
R_ab | Probability that a stay already longer than a cycles lasts more than a + b cycles | probability |
|---|---|---|
h_d | Probability of leaving the state in cycle d of the stay among those still in the state in that cycle | probability per cycle |
Function
Memoryless transition property of a Markov state and the sojourn-time distribution it implies
Maps the state a person occupies at cycle t, the cycle itself and the path H_t before it to the probability of each next state, under the restriction that the path adds nothing once the current state and cycle are known. With a constant probability p_ii of staying in state i the restriction makes the number of cycles spent in i on one visit geometric, and in continuous time a constant exit rate makes the time spent in i exponential, so in both cases the chance of leaving does not depend on the time already spent in the state. The package gives the geometric distribution of a stay with its median, the residual-stay identity behind memorylessness with a check of a survival curve for time since entry, the exponential residual stay and median, and the gap between the discrete and continuous mean stays at the same rate. Existing records are cited and not restated. The mean episode length 1 divided by 1 minus p_jj is HE-FM-MKCH-002, the cohort update is HE-FM-MM-001 with the trace totals of HE-FN-CSIM-001 and HE-FM-CSIM-001, rate and probability conversion is HE-FN-TP-001 with HE-FM-TP-001 to HE-FM-TP-004 and HE-CF-TP-001 and HE-CF-TP-002, exponential survival and the mean 1 divided by lambda are HE-FM-BTH-003 and HE-FM-CONT-002, the fundamental matrix is HE-FM-ABS-001 with its discounted form HE-FM-MKCH-001 and HE-CF-MKCH-001, the six-month root of an annual matrix is HE-FM-TMAT-001, and relaxing the assumption with tunnel states is HE-FN-TUN-001, HE-FM-TUN-001 to HE-FM-TUN-003 and HE-CF-TUN-001. Notation follows the Markov Assumption article.
Computational function
Computational function: per-cycle exit probabilities by time since entry from a survival curve checked against one constant exit probability
Takes survival at the end of each of the first K cycles since entry to a state, from follow-up of people who entered it or from a model with tunnel states, and returns the exit probability in each cycle among those still in the state, the constant exit probability that a memoryless state fitted to survival at K cycles would use, the largest gap between the two and the mean stay of the fitted memoryless state. Each per-cycle value is h_d of HE-FM-MKA-002 taken from the data, the ratio of adjacent survival values as in HE-FM-CUMH-002, and the constant comes from the rescaling formula HE-FM-TP-003 with the K-cycle survival as the observed value. A memoryless state needs every per-cycle value equal to the constant; a large gap shows that time since entry carries information that one state discards, and the per-cycle values then become the duration-specific inputs of HE-CF-TUN-001. The check is descriptive. With patient-level data Williams and colleagues used a state-arrival extended Cox model with time in the previous state as a covariate to decide whether to accept the assumption. The inputs differ from the formula's symbols, which describe a stay already known to be geometric.
Inputs and outputs:
S_1toS_5are survival at the end of cycles 1 to 5 since entry, each above zero and no higher than the one before, with survival at entry equal to 1; the calculator line takes five cycles and the code any number K.h_1toh_5are the per-cycle exit probabilities.q_cis the constant exit probability fitted to survival at the last cycle.devis the largest absolute difference between a per-cycle value andq_c.D_cis the mean stay of the fitted memoryless state in cycles, counting the cycle of entry, as in HE-FM-MKCH-002. In the codetolis the largest gap accepted as constant andmemorylessis TRUE whendevis no larger thantol.Assumption: Survival runs on the clock that starts at entry to the state, in one group followed from entry, at whole cycles of the model's cycle length. Censored follow-up is handled before this step, for example by reading the values off a Kaplan-Meier curve. The fitted constant matches survival at the last cycle only.
Worked example (Article's tunnel model for the year after an event): Survival of 0.75 after one year and 0.95 a year after that gives per-cycle exit probabilities of 0.25 and then 0.05, while the constant fitted to five-year survival of 0.61088 is 0.093869, the article's 0.09387. The largest gap is about 0.156, and the fitted state's mean stay of 10.653 cycles is the article's 10.653 post-event years against 16 with the tunnel.
S_1 = 0.75; S_2 = 0.7125; S_3 = 0.676875; S_4 = 0.643031; S_5 = 0.61088; h_1 = 0.25; h_2 = 0.05; h_3 = 0.05; h_4 = 0.05; h_5 = 0.05; q_c = 0.093869; dev = 0.156131; D_c = 10.6532Worked example (Article's memoryless model): The trace of the single post-event state, 0.90613 to the powers 1 to 5 rounded to six decimals, gives per-cycle values within 0.000001 of the constant (computed here for illustration).
S_1 = 0.90613; S_2 = 0.821072; S_3 = 0.743998; S_4 = 0.674159; S_5 = 0.610875; h_1 = 0.09387; h_2 = 0.09387; h_3 = 0.09387; h_4 = 0.09387; h_5 = 0.09387; q_c = 0.09387; dev = 0.000001; D_c = 10.653Worked example (Equal survival steps of 0.9): Survival falling by the same factor 0.9 each cycle is geometric, so every per-cycle value is 0.1, the gap is zero and the mean stay is 10 cycles, a limiting case that checks an implementation.
S_1 = 0.9; S_2 = 0.81; S_3 = 0.729; S_4 = 0.6561; S_5 = 0.59049; h_1 = 0.1; h_2 = 0.1; h_3 = 0.1; h_4 = 0.1; h_5 = 0.1; q_c = 0.1; dev = 0; D_c = 10Excel: With survival at entry, 1, in B1 and survival at the end of cycles 1 to K in B2 downwards (named SurvByCycle), put
=1-B2/B1in C2 and fill down to give the per-cycle values (named ExitByCycle). The constant is=1-INDEX(SurvByCycle,ROWS(SurvByCycle))^(1/ROWS(SurvByCycle))in a cell named ConstExit, the largest gap is=SUMPRODUCT(MAX(ABS(ExitByCycle-ConstExit))), which SUMPRODUCT evaluates as an array in classic Excel without array entry, and the mean stay is=1/ConstExit.R:
exit_check <- function(S, tol = 0.001) { K <- length(S); h <- 1-S/c(1, S[-K]); q_c <- 1-S[K]^(1/K); dev <- max(abs(h-q_c)); list(h = h, q_c = q_c, dev = dev, D_c = 1/q_c, memoryless = dev <= tol) }Base R only;exit_check(c(0.75, 0.7125, 0.676875, 0.643031, 0.61088))returns adevof 0.156131 andmemorylessFALSE.Python:
def exit_check(S, tol=0.001): K = len(S); prev = [1.0]+list(S[:-1]); h = [1-s/p for s, p in zip(S, prev)]; q_c = 1-S[-1]**(1/K); dev = max(abs(x-q_c) for x in h); return h, q_c, dev, 1/q_c, dev <= tolPlain Python with no imports, returning the outputs in the order listed.Test (Geometric survival returns per-cycle values equal to the constant): For survival of 0.9 to the power k the second per-cycle value equals the constant fitted at five cycles. Expected result TRUE. Entering the tunnel model's 0.7125 and 0.75 in place of 0.81 and 0.9 gives 0.05 against 0.1 and FALSE. Excel check
=ABS((1-0.81/0.9)-(1-0.59049^(1/5)))<1E-9Test (Fitted constant reproduces survival at the last cycle): Five cycles at 1 minus 0.093869 return the tunnel model's survival of 0.61088 to five decimal places. Expected result TRUE. Fitting the constant as one fifth of the five-year death probability, about 0.07782, gives 0.6669 and FALSE. Excel check
=ABS((1-0.093869)^5-0.61088)<0.00001Common error (Constant fitted at one horizon read as a fit at every horizon): The constant 0.093869 matches survival at five years by construction but puts one-year survival at about 0.906 against 0.75, so the first-cycle gap of about 0.156 and the article's third fewer post-event years are hidden when only the five-year value is compared.
Source: Grinstead CM, Snell JL. Grinstead and Snell's Introduction to Probability. The CHANCE Project version of 4 July 2006, based on the 2nd edition published by the American Mathematical Society. Chapter 5, section 5.1, Example 5.1, page 186, for the memoryless property P(T > r + s | T > r) = q^s of the geometric distribution. Williams C, Lewsey JD, Mackay DF, Briggs AH. Estimation of survival probabilities for use in cost-effectiveness analyses: a comparison of a multi-state modeling survival analysis approach with partitioned survival and Markov decision-analytic modeling. Medical Decision Making. 2017;37(4) 427-439. Section Multi-state Modeling, for a state-arrival extended model with time in the previous state used to help decide whether to accept the Markov assumption.
h_1 = 1 - S_1; h_2 = 1 - S_2 / S_1; h_3 = 1 - S_3 / S_2; h_4 = 1 - S_4 / S_3; h_5 = 1 - S_5 / S_4; q_c = 1 - S_5^(1 / 5); dev = max(abs(h_1 - q_c), abs(h_2 - q_c), abs(h_3 - q_c), abs(h_4 - q_c), abs(h_5 - q_c)); D_c = 1 / q_c
Try this function
Implementations
Excel
Residual stay and per-cycle exit probability of a geometric Markov sojourn in Excel
With the stay probability in StayProb, the cycles already spent in ElapsedCycles, the further cycles in ExtraCycles and the cycle of the stay in StayCycles, the formulas return R_ab and h_d, the second held in a cell named ExitAtDuration. Filling the second down a column of durations returns the same value in every row.
=StayProb^(ElapsedCycles+ExtraCycles)/StayProb^ElapsedCycles; =StayProb^(StayCycles-1)*(1-StayProb)/StayProb^(StayCycles-1)
Assumptions
Residual stay measured on the clock that starts at entry to the Markov state
The counts a, b and d run from entry to the state, not from model start. The Markov assumption allows the probabilities to follow model time, as p_ij(t) in the article, but not time since entry; Alarid-Escudero and colleagues call the second state-residence time dependence, which a cycle-specific matrix cannot carry when people enter the state at different times.
One homogeneous group followed from entry for a Markov memorylessness check
Comparing exit probabilities across durations as a test needs survival measured from entry in a group that the state is meant to represent. The ISPOR-SMDM report requires states to be homogeneous with respect to observed and unobserved characteristics, so a pattern by duration in pooled data can also come from differences between people within the state, which the state definitions then need to carry.
Worked examples
Five further cycles after one cycle in the article's memoryless post-event state
A person who has spent one cycle in the memoryless post-event state stays at least five more with probability about 0.6109, and anyone still in the state in its third cycle leaves in that cycle with probability 0.0939, the same as in the first.
p_ii = 0.90613; a = 1; b = 5; d = 3; R_ab = 0.6109; h_d = 0.0939
Five further cycles after ten cycles in the article's memoryless post-event state
Someone who entered ten cycles earlier has the same 0.6109 chance of five more cycles and the same exit probability of 0.0939 in the twelfth cycle, so two people in the state face one future whether they arrived one cycle ago or ten, as the article states.
p_ii = 0.90613; a = 10; b = 5; d = 12; R_ab = 0.6109; h_d = 0.0939
Common errors
Survival matched at one duration taken as proof of memorylessness
The article's single post-event state with a death probability of 0.09387 reproduces five-year survival of 0.61088 exactly, yet gives one-year post-event survival of about 0.906 against 0.75 and 10.653 post-event years per event against 16, a third less. In the article this overstates the life-years gained by about 0.64, or 15.7%, so agreement at one horizon does not show that time since entry can be ignored.
Model-time probabilities used for time since entry to a Markov state
Applying the article's post-event death probabilities by model cycle, 0.25 in the first cycle and 0.05 after, gives someone who has the event in model year three a first post-event probability of 0.05 instead of 0.25. A model-time curve treats everyone as having entered at the start, which is right only if all enter together.
Sources
Grinstead and Snell memoryless property of the geometric distribution
Grinstead CM, Snell JL. Grinstead and Snell's Introduction to Probability. The CHANCE Project version of 4 July 2006, based on the 2nd edition published by the American Mathematical Society. Chapter 5, section 5.1, Example 5.1, page 186, which computes P(T > r + s | T > r) = q^(r+s)/q^r = q^s for a geometric waiting time, independent of the time r already served, and states that this property is called the memoryless property and is also obeyed by the exponential distribution.
Williams and colleagues on testing the Markov assumption with time in the previous state
Williams C, Lewsey JD, Mackay DF, Briggs AH. Estimation of survival probabilities for use in cost-effectiveness analyses: a comparison of a multi-state modeling survival analysis approach with partitioned survival and Markov decision-analytic modeling. Medical Decision Making. 2017;37(4) 427-439. Section Multi-state Modeling, which fits a Cox state-arrival extended model for progression to death with a covariate for time in the previous state purely to help decide whether to accept the Markov assumption, finds evidence that the Markov property did not hold and moves to a semi-Markov approach; the Discussion suggests a semi-Markov model when power to detect a violation is insufficient and doubts remain.
Alarid-Escudero and colleagues on simulation-time and state-residence time dependence
Alarid-Escudero F, Krijkamp E, Enns EA, Yang A, Hunink MGM, Pechlivanoglou P, Jalal H. A tutorial on time-dependent cohort state-transition models in R using a cost-effectiveness analysis example. Medical Decision Making. 2023;43(1) 21-41. Introduction, which separates dependence on time since the start of the simulation from dependence on time spent in a state, and states that because the cohort ages together a cycle-specific matrix handles the first, while parameters that depend on time since disease onset cannot be implemented on simulation time and need tunnel states.
ISPOR-SMDM report on homogeneous states and history carried in state definitions
Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6) 812-820. Sections on defining states and heterogeneity with recommendation III-6, which require each state to be homogeneous with respect to both observed and unobserved characteristics that affect transition probabilities, and state that where prior states or time in the current state matter the relevant states should carry that history in their definition.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0