Dirichlet Distribution

Dirichlet Distribution for Transition Probabilities in Probabilistic Models

When a state in a Markov model has three or more destinations, the uncertain probabilities of moving to each have to be sampled together, because in every iteration of a probabilistic sensitivity analysis they must still add to one. The Dirichlet distribution does this by treating a whole row of a transition matrix as a single uncertain quantity. A row is fitted from observed transition counts with a prior that adds to them and is sampled from gamma draws, and an illustrative worked example with counts of 70, 20 and 10 shows every step.

Why multi-branch chance nodes need a joint distribution

A chance node or a Markov state with only two exits needs one uncertain probability, since the second is one minus the first, and the beta distribution is the usual choice for it. With three or more exits, separate distributions for each branch no longer guarantee that the branches add to one in a given iteration, so the model can create or lose part of the cohort.

Briggs, Ades and Price described the common advice to avoid this problem by restricting every chance node to two branches, and noted that the evidence may be naturally available in unconditional form, so forcing a conditional two-branch structure can complicate the sensitivity analysis rather than simplify it. They argued that the Dirichlet distribution, the multivariate equivalent of the beta, is an appropriate way to give probabilistic values to multiple branches while keeping mutually exclusive probabilities summing to one, and illustrated it by generating a fully probabilistic transition matrix for a Markov model.

Density, parameters and moments

The Dirichlet distribution is defined over vectors of $K$ non-negative proportions that add to one, which is exactly the set of valid rows of a transition matrix. It has one positive parameter for each destination, and its density is given below.

$$f(p_1, \dots, p_K \mid \alpha_1, \dots, \alpha_K) = \frac{\Gamma(\alpha_0)}{\prod_{j=1}^{K} \Gamma(\alpha_j)} \prod_{j=1}^{K} p_j^{\alpha_j - 1}, \qquad \sum_{j=1}^{K} p_j = 1$$

where $p_j$ is the probability of moving to destination $j$, $K$ is the number of possible destinations, $\alpha_j$ is the positive parameter attached to destination $j$, $\alpha_0 = \sum_{j} \alpha_j$ is the sum of the parameters, and $\Gamma(\cdot)$ is the gamma function. The means, variances and covariances follow from the parameters.

$$E(p_j) = \frac{\alpha_j}{\alpha_0}, \qquad \operatorname{Var}(p_j) = \frac{\alpha_j (\alpha_0 - \alpha_j)}{\alpha_0^2 (\alpha_0 + 1)}, \qquad \operatorname{Cov}(p_i, p_j) = -\frac{\alpha_i \alpha_j}{\alpha_0^2 (\alpha_0 + 1)}$$

where $E(p_j)$ is the expected probability of destination $j$, $\operatorname{Var}(p_j)$ its variance and $\operatorname{Cov}(p_i, p_j)$ the covariance between the probabilities of two different destinations $i$ and $j$. The ratios of the parameters fix the means, and their sum $\alpha_0$ fixes the precision, which is why statistical texts call the parameters prior sample sizes. In the Dirichlet, every pair of components is negatively correlated, so a sampled row that sends more of the cohort to one destination sends less to each of the others.

Two properties link the Dirichlet to simpler distributions. The marginal distribution of a single component is a beta, $p_j \sim \text{Beta}(\alpha_j, \alpha_0 - \alpha_j)$, so each transition probability taken on its own behaves like a beta fitted to that destination against all the others. Merging destinations keeps the Dirichlet form, with the merged parameter equal to the sum of the parameters merged, and with two destinations the Dirichlet is a beta.

Fitting a row from observed transition counts

When a cohort is observed over an interval that matches the model cycle, the evidence for one starting state is a vector of counts, the number of patients who ended the interval in each destination. The Dirichlet is the conjugate prior for the probabilities of a multinomial distribution. A Dirichlet prior distribution combined with multinomial counts therefore gives a Dirichlet posterior distribution, with each parameter increased by its count.

$$(p_1, \dots, p_K) \mid n_1, \dots, n_K \sim \text{Dirichlet}(a_1 + n_1, \dots, a_K + n_K)$$

where $n_j$ is the observed number of transitions from the starting state to destination $j$ and $a_j$ is the prior parameter for that destination. The prior parameters simply add to the counts.

The prior matters little when counts are large and a great deal when they are small. Setting every $a_j = 0$ gives a posterior whose parameters equal the raw counts and whose means equal the observed proportions, but this prior is improper and the posterior is only valid if every destination was observed at least once. Setting every $a_j = 1$ gives a uniform prior over all valid rows and adds one to each count. Briggs, Ades and Price showed that a Bayesian approach overcomes the problem of a transition of interest with a zero count, which would otherwise give a parameter of zero and no valid distribution. A transition that is clinically impossible, such as a return from death, is a different case, since it is left out of the row as a structural zero rather than given prior weight.

The counts must describe the same population, starting state and interval as the model, without irregular or censored follow-up.

Sampling a row from normalised gamma draws

The fastest standard method samples a Dirichlet row by drawing one independent gamma variable per destination and dividing each by their total. It generalises the way a beta value can be drawn from two gamma variables. Only a gamma random number generator is needed, so the method is easy to build into a spreadsheet model.

$$p_j = \frac{G_j}{\sum_{k=1}^{K} G_k}, \qquad G_j \sim \text{Gamma}(\alpha_j, 1)$$

where $G_j$ is an independent draw from a gamma distribution with shape $\alpha_j$ and a common scale of one, and $p_j$ is the resulting sampled probability for destination $j$. In Excel, each draw can be made with GAMMA.INV(RAND(), $\alpha_j$, 1), since GAMMA.INV with its third argument, beta, set to one returns the standard gamma distribution. The function returns an error if a shape parameter is zero or negative, which is how a zero count without a prior shows up in a spreadsheet.

A slower alternative draws the components in sequence as beta shares of the probability still unallocated; for three destinations, drawing Beta($\alpha_1$, $\alpha_2 + \alpha_3$) and then Beta($\alpha_2$, $\alpha_3$) is the conditional two-branch tree discussed above, and gives the same joint distribution as the Dirichlet when both are taken from the same Dirichlet parameters.

Worked example with a stable state and three exits

The figures below are illustrative. In a cohort model with a one-year cycle, 100 patients in a stable state were followed for one year, after which 70 remained stable, 20 progressed and 10 died. With the improper zero prior, the row is Dirichlet(70, 20, 10), with $\alpha_0 = 100$.

1. Means. The expected probabilities are 70/100 = 0.70 for remaining stable, 20/100 = 0.20 for progression and 10/100 = 0.10 for death, the observed proportions.

2. Spread and correlation. The variance of the progression probability is calculated first.

$$\operatorname{Var}(p_2) = \frac{20 \times 80}{100^2 \times 101} = \frac{1600}{1010000} = 0.001584$$

where $p_2$ is the progression probability and the formula is the variance given above. The standard deviation is the square root, about 0.0398. The covariance between remaining stable and progressing follows the same pattern.

$$\operatorname{Cov}(p_1, p_2) = -\frac{70 \times 20}{100^2 \times 101} = -\frac{1400}{1010000} = -0.001386$$

where $p_1$ is the probability of remaining stable. With a standard deviation for $p_1$ of about 0.0456, the correlation between the two is about minus 0.76, so in iterations where more patients progress, fewer remain stable.

3. An interval from the beta marginal. The progression probability on its own follows Beta(20, 80). Its central 95% runs from BETA.INV(0.025, 20, 80), about 0.128, to BETA.INV(0.975, 20, 80), about 0.283.

4. One sampled row. Taking fixed uniform numbers in place of RAND() so the step can be repeated, GAMMA.INV(0.30, 70, 1) returns about 65.38, GAMMA.INV(0.80, 20, 1) about 23.63 and GAMMA.INV(0.50, 10, 1) about 9.67. Their total is 98.68, and dividing each draw by it gives the sampled row.

$$p_1 = \frac{65.38}{98.68} = 0.663, \qquad p_2 = \frac{23.63}{98.68} = 0.239, \qquad p_3 = \frac{9.67}{98.68} = 0.098$$

where $p_3$ is the probability of death in this iteration. The three values add to 1.000, so the cohort is conserved.

5. What independent betas would have done. Suppose instead the three exits were sampled from separate distributions, Beta(70, 30), Beta(20, 80) and Beta(10, 90), each correct on its own. With independent draws the variance of the row total is the sum of the three variances, 0.002079 + 0.001584 + 0.000891 = 0.004554, a standard deviation of about 0.067, so about 30% of iterations would give a row total below 0.93 or above 1.07. Dividing each independent draw by the row total forces the sum to one but does not reproduce the distribution implied by the counts. In an illustrative simulation of 400,000 iterations, the rescaled progression probability had a central 95% range of about 0.136 to 0.267, narrower than the 0.128 to 0.283 of the correct marginal, so the rescaling understates uncertainty.

6. Effect of a uniform prior. With $a_j = 1$ for each destination, the posterior is Dirichlet(71, 21, 11), with means of 0.689, 0.204 and 0.107. With 100 patients the change is small. If the progressed state had been followed in only 60 patients, with none returning to stable, 45 remaining progressed and 15 dying, the zero prior would give no valid distribution, whereas the uniform prior gives Dirichlet(1, 46, 16), a mean return probability of 1/63 = 0.016 and a beta marginal Beta(1, 62) whose central 95% runs from about 0.0004 to 0.058. Whether that chance of returning is credible is a clinical judgement to be reported.

Building a probabilistic transition matrix

A full matrix is assembled row by row. Each starting state with more than one possible destination gets its own Dirichlet, fitted to the counts observed from that state; an absorbing state such as death keeps a fixed row; and structural zeros stay at zero in every iteration. Rows fitted to separate groups of patients are usually sampled independently of one another, while the components within each row are linked through the negative covariances above.

NICE's manual for technology appraisal (PMG36, paragraph 4.7.10) asks for distributions that characterise uncertainty in mean parameter values, represent the available evidence rather than being chosen arbitrarily, and are justified, with separate analyses for alternative plausible distributions. Paragraph 4.7.19 asks for evidence on correlation between parameters to be reflected in the probabilistic analysis. The manual does not name a particular distribution for probabilities. A Dirichlet fitted to counts is consistent with both requests for a row of multinomial transitions, because the counts set its parameters, and the parameters fix the means, the precision and the correlation between components.

Applying a treatment effect to one component, or converting a row to a different cycle length component by component, can break the sum to one, so the whole row is recalculated with the methods described for each transition probability.

Assumptions and limits of a single Dirichlet

A Dirichlet with $K$ parameters has only one measure of precision, $\alpha_0$. Once the means are fixed, the variance of every component follows from them and from $\alpha_0$, so the distribution cannot express, for example, a well-estimated death probability alongside a poorly estimated progression probability from a different source. In that situation a sequence of conditional betas fitted to separate evidence, or sampling the underlying hazards, can describe the evidence more faithfully. Because every pair of components in a Dirichlet is negatively correlated, it cannot express a positive correlation between two destinations of the same row, and it says nothing about correlation across rows or with other inputs such as a treatment effect.

The counts behind the parameters must be real. Multiplying a published percentage by an assumed sample size sets the precision by assumption, which sits uneasily with the requirement that distributions are not chosen arbitrarily. As with any distribution in a probabilistic analysis, the Dirichlet describes parameter uncertainty about the population row, which shrinks as more patients are observed; it does not describe how individual patients move.

Where Dirichlet rows go wrong in applied models

Several errors recur in applied models. Sampling each exit from its own beta and then ignoring or rescaling the row total understates uncertainty, as step 5 of the worked example shows. Entering zero counts directly gives an invalid parameter, and replacing them with a small arbitrary number hides the prior it implies, which should be reported. Treating the components of a sampled row as independent in later analysis is a third error. Computing the expected value of partial perfect information for one component would require the others to be drawn from their conditional distribution given it, so the row is usually treated as one group of parameters.

Sources

• Briggs AH, Ades AE, Price MJ. Probabilistic sensitivity analysis for decision trees with multiple branches: use of the Dirichlet distribution in a Bayesian framework. Medical Decision Making. 2003;23(4):341-350.
• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford: Oxford University Press; 2006. Chapter 4.
• Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. 3rd ed. Boca Raton: CRC Press; 2013. Section 3.4 and Appendix A.
• Microsoft. GAMMA.INV function. Microsoft Support; accessed 2 October 2026. https://support.microsoft.com/en-gb/office/gamma-inv-function-74991443-c2b0-4be5-aaab-1aa4d71fbb18
• Microsoft. BETA.INV function. Microsoft Support; accessed 2 October 2026. https://support.microsoft.com/en-gb/office/beta-inv-function-e84cb8aa-8df0-4cf6-9892-83a341d252eb
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. Section 4.7.