Between-Subject Variation

Between-Subject Variation in Cost-Effectiveness Analysis

Patients given the same treatment do not end up with the same costs or health outcomes, and part of that spread comes from stable differences between people rather than chance fluctuation from one measurement to the next. In health economic evaluation this decides how a model represents patients, whether as one average cohort, as subgroups, or as simulated individuals who differ from one another. Differences between patients are sometimes confused with uncertainty about model parameters, although the two call for different analyses. Modelling a single average patient can also bias cost-effectiveness results, and in an illustrative example a treatment that looks good value for the average patient is poor value for a low-risk subgroup at £20,000 per QALY.

Splitting total variation into between-patient and within-patient parts

When the same patients are measured more than once, for example quality-of-life scores at several trial visits or costs in successive years, the total spread of the observations can be divided into two parts. One part reflects how far each patient's own typical level sits from the population mean; the other reflects how far single measurements wander around that level. The standard expression is a linear mixed effects model with a random intercept, shown below.

$$Y_{ij} = \mu + u_i + \varepsilon_{ij}, \quad u_i \sim N(0, \sigma_b^2), \quad \varepsilon_{ij} \sim N(0, \sigma_w^2)$$

where $Y_{ij}$ is measurement $j$ on patient $i$, $\mu$ is the population mean, $u_i$ is the random effect for patient $i$ (that patient's persistent departure from the mean), $\varepsilon_{ij}$ is the deviation on occasion $j$ around the patient's own level, $\sigma_b^2$ is the between-subject variance and $\sigma_w^2$ is the within-subject variance. With independent random terms, the total variance and the share due to differences between patients are as follows.

$$\mathrm{Var}(Y_{ij}) = \sigma_b^2 + \sigma_w^2, \qquad \rho = \frac{\sigma_b^2}{\sigma_b^2 + \sigma_w^2}$$

where $\rho$ is the intraclass correlation, which also equals the correlation between two measurements on the same patient. As an illustration, if utility values have a between-patient standard deviation of 0.15 and a within-patient standard deviation of 0.08, the variances are 0.0225 and 0.0064, the total is 0.0289 and $\rho$ is about 0.78, so most of the spread reflects persistent differences between people. For a model, a persistent difference should travel with a simulated patient over time, whereas within-subject variation can be redrawn at each cycle.

Variability, heterogeneity and uncertainty in decision models

The ISPOR-SMDM Modeling Good Research Practices Task Force (working group 6, reported by Briggs and colleagues in 2012) noted that the terminology for patient differences and uncertainty is sometimes confused. Its report sets out preferred terms that separate how patients differ from how well model inputs are known.

The report links each idea to a simple regression model, shown below.

$$Y = X\beta + e$$

where $Y$ is an outcome, $X$ is a set of patient characteristics, $\beta$ is the vector of estimated coefficients and $e$ is the error term. Heterogeneity corresponds to the way predicted outcomes vary with $X$, parameter uncertainty to the standard errors of $\beta$, and stochastic uncertainty to the unexplained error $e$. The task force likens the stochastic and parameter distinction to that between a standard deviation, which describes how individual observations vary, and a standard error, which describes the precision of an estimate. Between-subject variation therefore has two parts in this scheme. The part explained by recorded characteristics is heterogeneity, and the unexplained part appears as first-order variability between otherwise identical patients.

Groot Koerkamp and colleagues give the division a computational form, in which patient heterogeneity is represented by frequency distributions, parameter uncertainty by probability distributions analysed with second-order Monte Carlo simulation, and stochastic uncertainty by first-order Monte Carlo simulation. The NICE Decision Support Unit's Technical Support Document 15 draws the practical conclusion that parameters reflecting patient heterogeneity should not be included in the probabilistic sensitivity analysis. A distribution of starting ages describes who is treated, not uncertainty about a parameter. Its mean and spread are themselves estimated from a sample, though, and uncertainty about those estimates does belong in the probabilistic sensitivity analysis.

Why the average patient can give the wrong answer

A cohort model run once for a patient with average characteristics gives the average outcome only if outcomes change linearly with those characteristics. Technical Support Document 15 states that when factors that vary between patients, such as age, have a non-linear relationship with costs and QALYs, using average characteristics gives a biased estimate of the average outcome across the population to be treated. In general the following holds.

$$\mathrm{E}[g(X)] \neq g(\mathrm{E}[X])$$

where $X$ is a patient characteristic that varies across the treated population, $g$ is the model mapping it to an outcome such as expected QALYs, and $\mathrm{E}$ denotes the population average. The two sides are equal when $g$ is linear in $X$.

The same document describes two remedies. Where the relevant factors can be identified before treatment, the population can be split into reasonably homogeneous subgroups, giving separate recommendations or a weighted average. Where too many categories would be needed, because of continuous variables or interacting factors, a patient-level simulation that samples characteristics from the population distribution, with any covariance between them reflected, gives an unbiased estimate provided enough patients are simulated.

Worked example with one treatment and two risk groups

The figures in this example are illustrative. A population is split equally between a low-risk group with an annual event risk of 2% and a high-risk group with an annual risk of 18%, so the mean annual risk is 10%. A preventive treatment halves the annual risk for 10 years and costs £6,000 per patient. A patient who has an event incurs £12,000 and loses 2.0 QALYs. Only the first event counts, annual risk is constant and independent between years, and there is no discounting. The probability of at least one event over 10 years is as follows.

$$P(r) = 1 - (1 - r)^{10}$$

where $P(r)$ is the 10-year probability of at least one event and $r$ is the annual event risk.

1. Events avoided per patient. Applying the formula with and without treatment gives these values, each rounded to four decimal places from unrounded calculations.

Across the real population, events avoided average $(0.0873 + 0.2520)/2 \approx 0.1697$ per patient. The average-patient cohort gives 0.2501, overstating the benefit by about 47%, because the event probability curve flattens at high risk.

2. Incremental costs, QALYs and ratios. For each group the increments are as follows.

$$\Delta C = 6000 - 12000 \times A, \qquad \Delta Q = 2.0 \times A$$

where $\Delta C$ is the incremental cost per patient in pounds, $\Delta Q$ is the incremental QALYs per patient and $A$ is the number of events avoided per patient. The low-risk group has $\Delta C$ of £4,952.40 and $\Delta Q$ of 0.1746; the high-risk group has £2,976 and 0.5040. The incremental cost-effectiveness ratios are as follows.

$$ICER_{low} = \frac{4952.4}{0.1746} = 28364, \qquad ICER_{pop} = \frac{(4952.4 + 2976)/2}{(0.1746 + 0.5040)/2} = \frac{3964.2}{0.3393} = 11683$$

where each ratio is the extra cost in pounds per QALY gained, for the low-risk group and for the whole population. The high-risk ratio is about £5,905 per QALY, and the average-patient cohort gives about £6,000 per QALY, roughly half the correct population figure.

At an illustrative threshold of £20,000 per QALY, the population ratio of £11,683 supports treating everyone, yet the low-risk ratio of about £28,400 is above the threshold, a subgroup the average-patient cohort gives no sign of; restricting treatment to the high-risk group is only possible if risk can be identified before treatment starts.

Representing patient differences in simulation models

An individual patient simulation represents between-subject variation directly. Technical Support Document 15 describes models in which starting characteristics are sampled from distributions and mean outcomes are taken across the simulated group, separately from random variation in each patient's pathway, which it calls stochastic uncertainty. A persistent unexplained difference, such as a patient-level random effect or a term in a frailty model, can be drawn once per simulated patient and kept for the whole run, so the patient's outcomes stay correlated over time.

Simulating individuals brings its own noise. The document notes that patient-level models carry sampling error from variability in simulated experiences between patients, often called first-order uncertainty (the task force lists Monte Carlo error as another name), which falls as more patients are simulated; the authors knew of no widely used algorithm in health economics for the optimal number and expect modellers to justify the number chosen. The task force report adds that assessing parameter uncertainty in such models requires stochastic uncertainty to be eliminated, a concern cohort models do not have.

Subgroups and modelling choices that change the answer

The task force report places the relevance of heterogeneity in identifying subgroups for separate cost-effectiveness analyses, and the NICE manual (PMG36, section 4.9) expects cost-utility analyses to give estimates separately for each relevant subgroup when the level of benefit differs between patients, so that a decision need not rest on the average treatment effect alone. Other choices about between-subject variation also alter results when the data are unchanged. A characteristic left out of the model cannot support a stratified decision and, if it acts non-linearly, biases the population average. Treating persistent differences as fresh noise in every cycle leaves means unchanged in a linear model but understates how widely lifetime costs and QALYs spread across patients. The NICE manual also asks analysts to specify the scale on which effect modification is defined. A constant relative effect, as in the worked example, still gives different absolute benefits when baseline risk varies.

How it differs from nearby ideas

Between-subject variation is often confused with concepts that describe variation at a different level or of a different kind.

• Within-subject variation is fluctuation in repeated measurements on one person; a crossover trial uses each patient as their own control, removing between-subject variation from the comparison.
• Between-study heterogeneity is variation in true effects across studies in a meta-analysis. A random effects model can describe either level, but the units are studies in one case and patients in the other.
• Heterogeneous treatment effect refers to variation in the effect of treatment itself. Between-subject variation also covers differences in baseline risk, costs and utilities that change absolute benefit when the relative effect is the same for everyone.
• Parameter uncertainty shrinks as more data are collected. Between-subject variation does not, since a larger sample estimates the between-subject variance more precisely but does not make patients more alike.

Common misreadings

Several recurring errors come from treating patient differences as something else. Reading the spread of individual outcomes from a microsimulation as uncertainty about the decision confuses first-order with second-order uncertainty; the decision rests on expected values and their parameter uncertainty. The bias from using mean characteristics in a non-linear model can run in either direction, so it cannot be assumed to be conservative. A subgroup difference on its own is weak evidence of heterogeneity, since small subgroups give imprecise estimates and differences can arise by chance.

Sources

• Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical Decision Making. 2012;32(5):722-732. https://eprints.gla.ac.uk/73438/
• Davis S, Stevenson M, Tappenden P, Wailoo A. NICE DSU Technical Support Document 15: Cost-effectiveness modelling using patient-level simulation. Sheffield: Decision Support Unit, ScHARR, University of Sheffield; 2014. Sections 2.2, 3.1 and 5.2. https://www.sheffield.ac.uk/nice-dsu/tsds/patient-level-simulation
• Fitzmaurice GM, Laird NM, Ware JH. Applied Longitudinal Analysis. 2nd ed. Hoboken, NJ: Wiley; 2011.
• Groot Koerkamp B, Stijnen T, Weinstein MC, Hunink MGM. The combined analysis of uncertainty and patient heterogeneity in medical decision models. Medical Decision Making. 2011;31(4):650-661.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. Section 4.9.