Finite Mixture Model

Finite Mixture Model in Health Care Cost, Use and Utility Analysis

A finite mixture model explains a skewed or lumpy distribution of patient data by assuming that each person belongs to one of a small, fixed number of unobserved classes, and it estimates each class's population share together with the distribution of the outcome within it. In health economics an early use is the demand for care, where a two-class mixture of count or cost regressions separates infrequent users from frequent users without anyone being labelled in advance. The same machinery is used for EQ-5D utilities in mapping. What makes the model finite is that unobserved heterogeneity is represented by a limited number of points instead of a continuous spread, and that choice shapes how it is estimated and how far its classes can be read as real patient groups.

Finite mixing as a special case of a general mixture

Any mixture model writes the distribution of an outcome as an average of component distributions. In the general form the component parameter can vary continuously across the population, and McLachlan, Lee and Rathnayake write it as an integral over a probability measure on the parameter space.

$$f(y) = \int f(y; \theta)\, dH(\theta)$$

where $y$ is the outcome, $f(y; \theta)$ is a component density with parameter $\theta$, and $H$ is the mixing distribution that says how $\theta$ is spread over the population. A finite mixture model is the case in which $H$ puts all its weight on $K$ points, so the integral becomes a weighted sum.

$$f(y_i \mid \mathbf{x}_i) = \sum_{k=1}^{K} \pi_k \, f_k(y_i \mid \mathbf{x}_i; \theta_k), \qquad \pi_k \ge 0, \qquad \sum_{k=1}^{K} \pi_k = 1$$

where $y_i$ is the outcome for person $i$ (a cost, a count of visits or a utility score), $\mathbf{x}_i$ is a vector of covariates, $K$ is the number of components, $\pi_k$ is the population share of class $k$, $f_k$ is the density for class $k$ (often negative binomial for counts or gamma for costs) and $\theta_k$ holds that class's parameters, including its own regression coefficients.

The finite form is less restrictive than it looks. The same review notes that, however general the mixing distribution is allowed to be, a likelihood-maximising mixing measure exists that is concentrated on no more points than there are distinct data values, so a finite mixture maximises the likelihood.

Why health care use and costs suit a finite mixture

Cost and use data are skewed, with many zeros, heavy right tails and sometimes several peaks. In their review of trial cost analysis, Mihaylova, Briggs, O'Hagan and Thompson describe parametric mixtures as a flexible way to accommodate excess zeros, overdispersion and heavy tails, with the further motive that covariates may act differently on rare users and high cost users. They report that mixtures often outperform single-distribution models for total resource use or costs.

Deb and Trivedi developed a finite mixture negative binomial model for six measures of medical care use by the elderly in the 1987 National Medical Expenditure Survey, and model selection criteria preferred it overall to a standard negative binomial and a hurdle model. Two points of support were enough, which they read as "healthy" and "ill" latent populations. Deb and Holmes applied the approach to behavioural health care, where a mixture can separate classes such as the "worried well" and the severely mentally ill. The finite mixture fitted spending and use much better than standard specifications, above all for high intensity users.

Mihaylova and colleagues describe the two-part model as a special case of a mixture with two components, one of them degenerate, whose parts are estimated separately because the split between users and non-users is observed. A finite mixture has unknown separation. On counts from the RAND Health Insurance Experiment, Deb and Trivedi later found strong evidence for a latent class model of infrequent and frequent users over a two-part model of users and non-users.

Mixture models for EQ-5D utilities and mapping

For utilities, Hernández Alava, Wailoo and Ara note that EQ-5D values are bounded above at 1 with a large gap below it, bounded below and multimodal, and they added a mixture for the multiple peaks to an adjusted limited dependent variable model for the bound and gap. In a rheumatoid arthritis trial three latent classes were appropriate, and the model beat linear and Tobit models on the Akaike information criterion, the Bayesian information criterion and prediction error, without predicting infeasible values. Such models are now an option for a mapping function that predicts a health state utility value. Mapping an asthma questionnaire, Gray, Hernández Alava and Wailoo reported that standard mapping approaches typically give biased estimates of health gain whereas the mixture approaches showed no such bias, and mapped utilities feed directly into QALY gains.

Estimation by maximum likelihood and the EM algorithm

Finite mixtures are usually fitted by maximum likelihood estimation, most often through the expectation maximisation (EM) algorithm. The algorithm treats each person's class as missing data. Its first step uses Bayes' theorem to give each person a posterior probability of belonging to each class.

$$\tau_{ik} = \frac{\pi_k \, f_k(y_i \mid \mathbf{x}_i; \theta_k)}{\sum_{m=1}^{K} \pi_m \, f_m(y_i \mid \mathbf{x}_i; \theta_m)}$$

where $\tau_{ik}$ is the posterior probability that person $i$ belongs to class $k$. The second step updates each mixing proportion as the average of these probabilities and re-estimates each class's parameters with every person weighted by $\tau_{ik}$.

$$\pi_k^{(t+1)} = \frac{1}{n} \sum_{i=1}^{n} \tau_{ik}^{(t)}$$

where $n$ is the number of people and $t$ counts iterations. McLachlan and colleagues report that the likelihood never falls between EM iterations and that the limit is in almost all cases a local maximum. Mihaylova and colleagues add that the log likelihood for mixtures of cost and use data often has several maxima. Refitting from several starting values and comparing the maximised log likelihoods is therefore a basic check, as is looking for spurious solutions in which one component captures a few observations with a very small variance. Class numbering carries no meaning either, since with components from one family the density is unchanged under any permutation of the labels, so classes are identified by their estimated parameters, such as mean cost.

Choosing the number of classes

McLachlan and colleagues call identifying the number of components arguably the most obdurate problem with mixture distributions. Adding a component always raises the likelihood, so one approach penalises the log likelihood for the number of parameters.

$$\text{BIC} = -2 \log L(\hat{\Psi}) + d \log n$$

where $L(\hat{\Psi})$ is the maximised likelihood, $d$ is the number of free parameters (for $K$ classes each with $p$ parameters, $d = Kp + K - 1$ because the shares sum to one) and $n$ is the sample size, with lower values preferred.

A likelihood ratio test of $K$ against $K + 1$ components cannot use its usual chi-squared reference distribution because the standard regularity conditions fail, so its null distribution is usually estimated by resampling. BIC tends to favour enough components to describe the density well and so can overestimate the number of clusters. Applied work therefore also asks whether each class is large, stable and interpretable.

Worked example with two illustrative cost classes

All numbers in this example are illustrative. Annual costs in a chronic condition are modelled as a two-class mixture of exponential distributions, which can be worked by hand (applied work would use gamma or log-normal components with covariates). The low-use class has mean GBP 800 and share 0.75, and the high-use class has mean GBP 6,000 and share 0.25. An exponential density with mean $\mu$ is $f(y) = \tfrac{1}{\mu} e^{-y/\mu}$.

1. Mean cost. The low-use class contributes 0.75 × 800 = 600 and the high-use class 0.25 × 6,000 = 1,500, and 600 + 1,500 = 2,100, so the population mean is GBP 2,100 per person.

2. Class membership for one patient. For a patient costing GBP 3,000, the class densities are $f_1 = \tfrac{1}{800} e^{-3.75} = 2.940 \times 10^{-5}$ and $f_2 = \tfrac{1}{6000} e^{-0.5} = 1.0109 \times 10^{-4}$. Weighting by the shares gives $\pi_1 f_1 = 0.75 \times 2.940 \times 10^{-5} = 2.205 \times 10^{-5}$ and $\pi_2 f_2 = 0.25 \times 1.0109 \times 10^{-4} = 2.527 \times 10^{-5}$. The posterior probability of the high-use class is 2.527 / 4.732 = 0.534.

3. The upper tail. The probability of costing more than GBP 10,000 is $\Pr(Y > 10{,}000) = 0.75 e^{-12.5} + 0.25 e^{-1.667}$, which is 0.0000028 + 0.0472, or 0.0472. A single exponential with the same mean of GBP 2,100 gives $e^{-4.762} = 0.0085$. Per 1,000 patients that is about 47 people above GBP 10,000 against about 9.

4. A change in class shares. If a programme moved 5 percentage points of the population out of the high-use class, the contributions would become 0.80 × 800 = 640 and 0.20 × 6,000 = 1,200. Since 640 + 1,200 = 1,840, the mean falls by GBP 260 per person, which equals 0.05 × 5,200, the shift times the gap between the class means.

5. Interpretation. Two models with the same mean give very different numbers of high-cost patients, which matters for budget impact analysis. Step 4 holds only if the programme changes class membership without changing the class distributions, a causal assumption the mixture itself cannot test.

Reading latent classes in economic evaluation and HTA

McLachlan and colleagues note that the component structure can be used for clustering irrespective of whether the groups exist externally, and labels such as "healthy", "ill" or "worried well" are the authors' readings of fitted classes. A class is a credible patient subgroup only when it makes clinical sense and reappears in other data.

For a cost-effectiveness analysis the main product is usually a better estimate of mean cost, use or utility. Assigning each person outright to the most probable class and treating classes as observed subgroups ignores uncertainty in the posterior probabilities, so analyses using classes downstream should carry those probabilities, and the joint uncertainty in shares and class parameters, into the probabilistic sensitivity analysis. Mihaylova and colleagues placed mixture models in their "red orbit" of methods that generally require substantial expertise in modelling and computation, while judging that mixtures could have significant advantages for skewed, multimodal cost data and calling for further applications.

Neighbouring models and where each applies

Several live pages describe special cases or close relatives. The choice depends on what the latent structure represents.

A finite mixture differs from subgroup analysis, which compares groups defined by observed characteristics.

Limitations and common misreadings

Most problems arise when classes are asked to carry more meaning than the data support. Others are computational and caught by routine checks.

1. Treating classes as real groups. A mixture can fit a skewed distribution well when no distinct subpopulations exist.
2. Choosing the number of classes on fit alone. Criteria can favour extra components, and a tiny class with a very small variance is often spurious.
3. Trusting a single run. EM stops at a local maximum that depends on the starting values.
4. Expecting many classes to be identifiable. With unknown separation and more than two components, overlap between distributions can make components hard to identify, and Mihaylova and colleagues rated mixtures as only of possible applicability in samples of tens to a few hundreds.

Sources

• Deb P, Holmes AM. Estimates of use and costs of behavioural health care: a comparison of standard and finite mixture models. Health Economics. 2000;9(6):475-489. https://pubmed.ncbi.nlm.nih.gov/10983002/
• Deb P, Trivedi PK. Demand for medical care by the elderly: a finite mixture approach. Journal of Applied Econometrics. 1997;12(3):313-336. https://ideas.repec.org/a/jae/japmet/v12y1997i3p313-36.html
• Deb P, Trivedi PK. The structure of demand for health care: latent class versus two-part models. Journal of Health Economics. 2002;21(4):601-625. https://pubmed.ncbi.nlm.nih.gov/12146593/
• Gray LA, Hernández Alava M, Wailoo AJ. Development of methods for the mapping of utilities using mixture models: mapping the AQLQ-S to the EQ-5D-5L and the HUI3 in patients with asthma. Value in Health. 2018;21(6):748-757. https://doi.org/10.1016/j.jval.2017.09.017
• Hernández Alava M, Wailoo AJ, Ara R. Tails from the peak district: adjusted limited dependent variable mixture models of EQ-5D questionnaire health state utility values. Value in Health. 2012;15(3):550-561. https://pubmed.ncbi.nlm.nih.gov/22583466/
• McLachlan GJ, Lee SX, Rathnayake SI. Finite mixture models. Annual Review of Statistics and Its Application. 2019;6:355-378. https://doi.org/10.1146/annurev-statistics-031017-100325
• Mihaylova B, Briggs A, O'Hagan A, Thompson SG. Review of statistical methods for analysing healthcare resources and costs. Health Economics. 2011;20(8):897-916. https://doi.org/10.1002/hec.1653