Signature
e_ij = n_ii * p_ij; L_j = 1 / (1 - p_jj); T_ij = e_ij * L_j; h_ij = p_ij / (1 - p_ii)
| Inputs | Definition | Unit |
|---|---|---|
n_ii | Diagonal entry of the fundamental matrix N for state i: expected cycles spent in i by a person starting there, counting the starting cycle and every cycle after a return from j | cycles |
p_ij | Probability of moving from state i to state j in one cycle | probability per cycle |
p_jj | Probability of remaining in state j for another cycle | probability per cycle |
p_ii | Probability of remaining in state i for another cycle | probability per cycle |
e_ij | Expected number of separate entries into state j by a person starting in state i | episodes per person |
|---|---|---|
L_j | Expected number of cycles in one episode of state j, counting the cycle of entry | cycles per episode |
T_ij | Expected cycles spent in state j over all episodes by a person starting in state i, equal to entry i, j of N | cycles |
h_ij | Probability that a person starting in state i enters state j at least once before absorption | probability from 0 to 1 |
Function
Discounted and episode-level occupancy of a time-homogeneous absorbing Markov chain
Maps the transient block Q of a time-homogeneous absorbing Markov chain to the expected number of cycles spent in each transient state, either with each cycle discounted at a constant rate or split into separate episodes of a state that can be left and re-entered. The undiscounted fundamental matrix N, the expected cycles before absorption t = Nc and the absorption probabilities B = NR are HE-FM-ABS-001 and HE-FM-ABS-002, the cohort update s_(t+1) = s_t P is HE-FM-MM-001 and a cohort trace with counting weights and discounting is HE-FM-CSIM-001. Notation follows the Markov Chain article.
Try this function
Implementations
Excel
Episode decomposition of a re-enterable Markov chain state in Excel
With n_ii, p_ij, p_jj and p_ii in cells named N_ii, P_ij, P_jj and P_ii, the four formulas return e_ij, L_j, T_ij and h_ij in turn; the third refers to the first two results held in cells named E_ij and L_j.
=N_ii*P_ij; =1/(1-P_jj); =E_ij*L_j; =P_ij/(1-P_ii)
Assumptions
Re-enterable Markov chain state entered only from the starting state
State j can be entered only from state i, as in a chain whose only transient states are Well and Ill. Where j can also be entered from a third transient state, the entries from that state are added to e_ij.
Exits from the starting state go to the re-enterable state or to absorption
h_ij takes the share of exits from i that go to j, so every exit from i leads to j or to an absorbing state. With another transient state reachable from i, the probability of ever entering j is n_ij divided by n_jj from the fundamental matrix instead.
Time-homogeneous chain started outside the re-enterable state
The probabilities are the same in every cycle and the person starts in i, not in j. A person starting in j has one extra episode already under way: from Ill in the article's chain, 1 plus 8 times 0.10 gives 1.8 episodes and 6 years Ill, the Ill to Ill entry of N (computed here for illustration).
Worked examples
Ill episodes from a Well start in a chain with recovery
From Well, a person spends 12 expected years in Well and 10% fall ill each year, giving 12 × 0.10 = 1.2 expected episodes of illness, each lasting 1 divided by 0.30, or 3.33 years on average, so 4 expected years Ill, the Well to Ill entry of N. The probability of ever falling ill is 0.10/0.15 = 0.6667, two in three, as in the article.
n_ii = 12; p_ij = 0.10; p_jj = 0.70; p_ii = 0.85; e_ij = 1.2; L_j = 3.3333; T_ij = 4; h_ij = 0.6667
Ill episodes from a Well start when no one recovers
If the 0.20 recovery probability is added to staying Ill, giving an Ill row of 0, 0.90, 0.10, no one returns to Well, so n_ii falls to 6.6667 and a Well start has 0.6667 expected episodes, each lasting 10 years, or 6.6667 years Ill, as in the article. Recovery adds episodes (1.2 against 0.67) but shortens each one, so total time Ill falls from 6.67 to 4 years (episode counts computed here for illustration).
n_ii = 6.6667; p_ij = 0.10; p_jj = 0.90; p_ii = 0.85; e_ij = 0.6667; L_j = 10; T_ij = 6.6667; h_ij = 0.6667
Common errors
Ignoring returns to Well after recovery in a Markov chain episode count
Taking the time at risk in Well as 1 divided by 0.15, or 6.67 years, leaves out the years spent in Well after recovering, giving 0.67 episodes and 2.22 years Ill in place of 1.2 episodes and 4 years.
Reading Markov chain mean episode length as mean time ill per person
The mean of 3.33 years applies to each episode. A third of people starting in Well never fall ill, and those who do may have several episodes, so the mean time Ill among people who ever fall ill is 4/0.6667 = 6 years, and the mean over everyone starting in Well is 4 years.
Sources
Grinstead and Snell on expected visits to a transient state
Grinstead CM, Snell JL. Grinstead and Snell's Introduction to Probability. The CHANCE Project version of 4 July 2006, based on the 2nd edition published by the American Mathematical Society (full text read). Chapter 11 Markov Chains, section 11.2 Absorbing Markov Chains: Definition 11.3 and Theorem 11.4, the entry n_ij of the fundamental matrix is the expected number of times the chain is in transient state s_j given that it starts in s_i, with the initial state counted if i = j.
Grinstead and Snell on the mean of the geometric distribution
Grinstead CM, Snell JL. Grinstead and Snell's Introduction to Probability. The CHANCE Project version of 4 July 2006, based on the 2nd edition published by the American Mathematical Society (full text read). Chapter 6 Expected Value and Variance, section 6.1, Example 6.4: the time T to the first success in a Bernoulli trials process has the geometric distribution P(T = j) = q^(j-1) p, and E(T) = 1/p. With success taken as leaving state j, p = 1 minus p_jj gives the mean episode length.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0