Markov Cohort Model

Markov Cohort Model and When a Cohort Is Enough for the Decision

A Markov cohort model runs a Markov model on a notional group of patients rather than on separate people, so each health state holds a share of the group and every member of that share faces the same transition probabilities. Because the shares are probabilities, the model returns mean costs and quality-adjusted life years per person directly, which is the quantity a cost-effectiveness decision needs. The NICE technology appraisal manual (PMG36) names the Markov cohort model and individual patient simulation as examples of model types whose choice should be justified for each new decision problem. The choice turns on whether every piece of patient history that changes risk, cost or quality of life fits into a manageable number of states. If it does not, an individual-level model such as Markov microsimulation is the usual alternative.

What makes a Markov model a cohort model

A Markov cohort model is a state-transition model evaluated as a cohort simulation, one of three methods in Sonnenberg and Beck's guide, which redistributes a hypothetical cohort across the states at every cycle. The ISPOR-SMDM good practice report on state-transition modelling treats cohort ("Markov") and individual-level ("microsimulation") models as the two common frameworks, and NICE Decision Support Unit Technical Support Document 15 (TSD 15) defines a cohort model as one that estimates outcomes for the group without explicitly considering the outcomes of each individual patient.

The tutorial on cohort state-transition models by Alarid-Escudero and colleagues names three core components, a state vector, a cohort trace and a transition matrix. The state vector is updated once per cycle as follows.

$$\mathbf{m}_{t+1} = \mathbf{m}_t \mathbf{P}$$

where $\mathbf{m}_t$ is a row vector holding the proportion of the cohort in each of the $n_S$ states at cycle $t$, $\mathbf{m}_0$ is the starting distribution, and $\mathbf{P}$ is the $n_S \times n_S$ matrix whose element $p_{ij}$ is the one-cycle probability of moving out of state $i$ into state $j$, with every row summing to one. Stacking the vectors gives the cohort trace, whose cycle-by-cycle mechanics are on the cohort simulation page, and costs and QALYs are read off it.

$$\bar{C} = \sum_{t=0}^{T} w_t \frac{\mathbf{m}_t \mathbf{c}}{(1 + r)^{t}} \qquad \bar{E} = \sum_{t=0}^{T} w_t \frac{\mathbf{m}_t \mathbf{q}}{(1 + r)^{t}}$$

where $\bar{C}$ and $\bar{E}$ are expected costs and QALYs per person, $\mathbf{c}$ is a column vector of the cost of one cycle in each state, $\mathbf{q}$ is a column vector of the QALYs earned in one cycle in each state (utility multiplied by cycle length), $r$ is the discount rate per cycle, $T$ is the final cycle and $w_t$ is a within-cycle correction weight, such as the half-cycle correction that the ISPOR-SMDM report recommends for the first cycle and, when the horizon is not lifetime, the final cycle.

What a cohort assumes about the people in each state

The ISPOR-SMDM report states that each state in a cohort simulation is homogeneous, so every individual in a state has the same transition probabilities, and its best practice III-6 asks for states that are homogeneous with respect to both observed and unobserved characteristics that affect those probabilities. History that changes risk must therefore become part of the state definition, so in the report's example a model in which a prior myocardial infarction raises risk needs states for people with and without one. Continuous measures such as blood pressure have to be grouped into categories, and variables unknown at decision time, such as an undiagnosed infection, can create what the report calls heterogeneity bias.

Characteristics known at the start, such as age, sex and disease stage, need not enter the states. The ISPOR-SMDM report notes they can define separate starting cohorts unless they change over time in a meaningful way. TSD 15 warns that one cohort run at average values biases the mean when outcomes are non-linear, as the cohort model page shows.

The cohort is also closed and its members do not affect one another, since the ISPOR-SMDM report says both common frameworks model a single closed cohort and do not capture interactions. The ISPOR-SMDM report on dynamic transmission modelling adds that many common models, such as Markov models, ignore the indirect effects of averted infections, so an intervention that changes transmission, such as a vaccine with herd effects, needs a dynamic transmission model.

Memory, tunnel states and state explosion

Under the Markov assumption the next transition depends only on the current state. The ISPOR-SMDM report says this can be very limiting where past states or time in the current state strongly determine what happens next. A cohort model can carry memory only by adding states. TSD 15 describes two routes, replicate states for people with a particular history of previous events and chains of tunnel states that record time since the last transition, and warns that there is a limit beyond which the cohort approach becomes unwieldy.

The ISPOR-SMDM report calls the result state explosion. Each extra history variable multiplies the number of states, and each new state adds a row of the transition matrix to estimate and check. An individual-level model instead keeps a few health states plus tracker variables for each simulated person, and TSD 15 describes this as trading the need for stochastic evaluation against fewer states to enumerate.

Worked example of a fracture model as a cohort and as individuals

The model and every number in this example are hypothetical and illustrative. A model for a bone-protecting treatment must let fracture risk depend on three pieces of history, the number of prior fractures (none, one, or two or more), the time since the most recent fracture (first year, second year, third year or later) and treatment status (on or off treatment).

1. Count the states the cohort needs. People with no prior fracture have no time since fracture to record, so they need one history state. People with one fracture, or with two or more, each need three time states. Every history state is then split by treatment status, and Dead is added once.

$$S = (1 + k \times d) \times g + 1 = (1 + 2 \times 3) \times 2 + 1 = 15$$

where $S$ is the number of states, $k = 2$ is the number of prior-fracture levels above none, $d = 3$ is the number of time-since-fracture levels, $g = 2$ is the number of treatment statuses and the final 1 is Dead. The transition matrix has 15 × 15 = 225 cells, most of them zero by design.

2. Add more memory. If time since fracture is tracked year by year up to a fifth year or later level, $d$ rises to 5 and $S = (1 + 2 \times 5) \times 2 + 1 = 23$ states, with 23 × 23 = 529 cells. Adding a third treatment status for people who stopped because of side effects gives (1 + 2 × 5) × 3 + 1 = 34 states and 34 × 34 = 1,156 cells. An individual-level version keeps Alive and Dead and three tracker variables, however fine the history becomes.

3. Handle what is known at baseline outside the states. Sex and three age bands at entry define 2 × 3 = 6 starting cohorts. Each is run separately through the same 15 states and the results are weighted by the population shares, so this heterogeneity adds runs, not states.

4. Compare the precision of the two forms. Suppose the 15-state cohort model gives a ten-year probability of a first fracture of $p = 0.30$. That figure carries no sampling error. A microsimulation estimates it from $n$ simulated people, with a standard error as follows.

$$\text{SE} = \sqrt{\frac{p(1 - p)}{n}}$$

where $p$ is the true proportion and $n$ is the number of simulated individuals. With $n$ = 1,000 the variance term is 0.30 × 0.70 = 0.21 divided by 1,000, so the standard error is about 0.0145 and an approximate 95% range runs from 0.272 to 0.328. With $n$ = 100,000 the standard error falls to about 0.00145. Cutting the noise tenfold needs one hundred times as many people, which is the Monte Carlo error that the cohort form avoids.

5. Count the work in a probabilistic analysis. With 1,000 probabilistic sensitivity analysis draws and two strategies, the cohort model needs 1,000 × 2 = 2,000 runs. The microsimulation needs an inner loop of 100,000 people for each strategy and draw, or 1,000 × 100,000 × 2 = 200,000,000 simulated patient histories.

6. Interpretation. At 15 states the cohort form is manageable and gives exact expectations cheaply, so the ISPOR-SMDM recommendation points to it. At 34 states, or if risk also depended on a continuous measure such as bone density, the case for an individual-level model strengthens, and step 5 shows the computing price of that validity.

Choosing the cohort form and justifying the choice

The ISPOR-SMDM best practice III-1 favours a cohort whenever a manageable number of states holds every relevant characteristic, and warns that validity should not be sacrificed for simplicity. For NICE submissions the justification is expected in writing. PMG36 section 4.6.3 says it is not enough to state that a structure was used before, and section 4.7.20 asks for clear justification of any model structure that limits the feasibility of probabilistic sensitivity analysis. TSD 15 states that models used in NICE appraisals have sometimes been criticised on this choice. It notes that with modern computers the extra PSA computing time of a patient-level model may not be overly restrictive, so computing cost alone is a weak reason to accept an invalid cohort structure.

Reading results from a Markov cohort model

The outputs are expected values for the modelled population, as PMG36 section 4.2.16 requires when it says an ICER must be the ratio of expected additional total cost to expected additional QALYs. TSD 15 calls this analytic evaluation, since no random sampling is needed to obtain the mean for one set of inputs. When the model is not linear, PMG36 section 4.7.12 prefers the probabilistic estimate.

The mean hides how outcomes are spread across patients. The ISPOR-SMDM report gives the example of a one-year gain in life expectancy that could mean one year for everyone or three years for half and one year lost for the other half. It recommends reporting the distribution where it matters, for instance for equity, and notes that one can be derived from a cohort model by analysing the Markov trace. The report also suggests setting parameters so the trace has a predictable shape, as a way of finding programming errors and part of wider model validation.

Misreadings of Markov cohort models

Several errors recur when these models are built or read. Most come from forgetting that the cohort is an averaging device with no memory beyond its states.

• The cohort is a notional group created inside the model. It has no recruited participants.
• No sampling noise does not mean no uncertainty. The cohort result is exact for its inputs, but parameter uncertainty and structural uncertainty remain and still need probabilistic and scenario analyses.
• A cohort model is not necessarily a Markov chain. Time-dependent transition probabilities, such as age-specific mortality, are standard in cohort models, while a Markov chain in Sonnenberg and Beck's sense has constant probabilities.

Sources

• Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322-338. doi:10.1177/0272989X9301300409.
• Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. doi:10.1016/j.jval.2012.06.014.
• Alarid-Escudero F, Krijkamp E, Enns EA, Yang A, Hunink MGM, Pechlivanoglou P, Jalal H. An introductory tutorial on cohort state-transition models in R using a cost-effectiveness analysis example. Medical Decision Making. 2023;43(1):3-20. doi:10.1177/0272989X221103163.
• Davis S, Stevenson M, Tappenden P, Wailoo A. NICE DSU Technical Support Document 15: cost-effectiveness modelling using patient-level simulation. Decision Support Unit, ScHARR, University of Sheffield; April 2014. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD15_Patient-level_simulation.pdf
• Pitman R, Fisman D, Zaric GS, Postma M, Kretzschmar M, Edmunds J, Brisson M. Dynamic transmission modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-5. Value in Health. 2012;15(6):828-834. doi:10.1016/j.jval.2012.06.011.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Chapter 4, sections 4.2.16, 4.6.3, 4.7.12 and 4.7.20. https://www.nice.org.uk/process/pmg36/chapter/economic-evaluation-2