Steady State

Steady State

A steady state of a time-homogeneous state-transition model is a probability distribution across states that remains unchanged after one transition step. It is also called a stationary distribution. The term concerns the distribution of a modelled population, not the claim that any particular individual stays in one health state or that real health service demand is permanently constant.

Stationary balance equation

For a discrete-time Markov chain with row-stochastic transition matrix $P$, let row vector $\boldsymbol{\pi}$ contain nonnegative state proportions summing to one. A steady-state distribution satisfies

$$\boldsymbol{\pi}P=\boldsymbol{\pi},\qquad\sum_s\pi_s=1.$$

The first equation says that, for each state, the expected inflow from all states during a cycle balances the proportion currently there. The second normalises the solution as a probability distribution. If the initial distribution is exactly $\boldsymbol{\pi}$, its distribution remains $\boldsymbol{\pi}$ on subsequent steps under the same $P$. A row-stochastic matrix has entries $P_{ss'}=\Pr(X_{t+1}=s'\mid X_t=s)$ and each row sums to one; using a column-stochastic convention instead requires transposing the equations consistently.

Existence, uniqueness and convergence are separate questions. A finite chain has at least one stationary distribution. If it is irreducible, the stationary distribution is unique; if it is also aperiodic, repeated transitions from any initial distribution converge to it. A periodic chain can have a stationary distribution while its cycle-by-cycle distribution oscillates instead of converging. A reducible model can have multiple stationary distributions and long-run outcomes dependent on the initial class.

Exact two-state example

Consider two recurring health-service states, Stable and High Need, with the following one-cycle transition probabilities. The numbers are illustrative and assume the same probabilities in every cycle; they do not describe a particular disease.

Write the steady-state Stable proportion as $p$ and High Need proportion as $1-p$. Balancing the Stable state gives $p=0.80p+0.30(1-p)$, so $0.50p=0.30$ and $p=0.60$. Therefore $\boldsymbol{\pi}=(0.60,0.40)$. Direct multiplication confirms the result: Stable after one cycle is $0.60(0.80)+0.40(0.30)=0.60$; High Need is $0.60(0.20)+0.40(0.70)=0.40$.

Starting instead with everyone Stable, $\boldsymbol{\pi}_0=(1,0)$, successive distributions are:

The distribution approaches the steady state but does not equal it at cycles 1–3. This chain is irreducible because both states can reach each other and aperiodic because remaining in a state has positive probability. For this two-state model, the deviation from 0.60 halves each cycle; the convergence speed is a feature of its transition probabilities, not of every Markov model.

Rewards at steady state

If state rewards are fixed per cycle, the expected reward per cycle once the distribution is stationary is $\boldsymbol{\pi}\mathbf{r}$. Suppose annual costs are 100 currency units in Stable and 300 in High Need, and annual utility weights are 0.90 and 0.50. At steady state, average cost is $0.60(100)+0.40(300)=180$ currency units per person-year, and utility-weighted survival under a full-year occupancy convention is $0.60(0.90)+0.40(0.50)=0.74$ QALYs per person-year.

Those are rates of accumulation in a maintained stationary distribution, not lifetime totals. Multiplying 180 or 0.74 by a finite horizon would be valid only if the distribution were stationary throughout that horizon, the rewards and cycle conventions stayed constant, and discounting and eligibility were handled appropriately. Starting all Stable produces different early-cycle rewards. A finite-horizon economic evaluation should generally propagate the actual initial cohort distribution rather than replace it with an asymptotic distribution.

Absorbing states and why the limit may be unhelpful

Many health economic models include an absorbing Dead state: once entered, a person never leaves. If all living states eventually lead to death with probability one, the limiting distribution of a closed cohort is entirely Dead. This is a mathematically valid stationary distribution but usually not the clinically interesting “steady level” of illness or resource use. Report finite-horizon state occupancy, survival, costs and QALYs instead of treating the all-dead limit as a useful planning result.

With several absorbing classes, the limiting distribution can depend on the initial state, and stationary distributions may not be unique. A service-delivery system with arrivals and departures might reach a stable flow or occupancy under sustained inflow, but that is an open-system model with birth, arrival or replacement terms, not necessarily the stationary distribution of a closed patient cohort. State which system is being balanced before borrowing a steady-state formula.

If transition probabilities change with age, calendar time, time since an event, treatment duration or epidemic conditions, there is no single fixed transition matrix $P$ across the model horizon. A sequence $\boldsymbol{\pi}_{t+1}=\boldsymbol{\pi}_tP_t$ may be the correct calculation even if a snapshot matrix at one time has a stationary solution. Calling that instantaneous solution the actual future distribution would require extra assumptions. Periodic forcing, evolving population mix and policy changes also can prevent a fixed steady state.

Interpretation and validation

Steady-state proportions may help plan long-run capacity in a recurrent, replenished and sufficiently stable system, or provide a mathematical check on a transition matrix. They do not show how quickly a system reaches the limit, whether the horizon is long enough, whether state rewards are affordable, or whether the underlying model matches observed flows. Examine transient trajectories and time to approach the limiting distribution; in slow-mixing chains, a distant steady state may be irrelevant to near-term decisions.

Verify each row of $P$, solve the normalised balance equations, and substitute the result back into $\boldsymbol{\pi}P$. Check nonnegativity and total one, irreducibility, aperiodicity, absorbing states and plausible parameter uncertainty. Numerical eigensolvers can return an unnormalised vector or unstable answers when eigenvalues cluster; residual checks and multiple starting distributions help diagnose interpretation and numerical issues. A stationary distribution is conditional on the chosen transition probabilities, which themselves need evidence and validation.

In health economics, assess changes to rewards and transition rates separately. A policy could alter the steady-state mix, improve health within the same named state, or both. Compare relevant finite-horizon costs and QALYs for competing strategies; a higher fraction in a favourable state at equilibrium does not by itself establish cost-effectiveness, especially if achieving it is costly or occurs only after a long delay.

Methodological sources

The ISPOR-SMDM State-Transition Modeling report sets out the health decision-model context. Carnegie Mellon University's notes on stationary distributions distinguish the balance equation from convergence conditions. The University of Cambridge Markov-chain lecture states convergence for finite irreducible aperiodic chains. The two-state calculation here is original and exactly verifiable from its displayed transition table.