Within Subject Variation

Within Subject Variation in Repeated Health Measurements

Within-subject variation is the spread that appears when the same patient is measured more than once, so that an EQ-5D index recorded at three trial visits, or a person's healthcare costs in two successive years, differ even when nothing systematic has changed. Health economists meet it in most trial-based cost-utility analyses and in before-and-after evaluations of services. Its size decides how much one reading says about a patient, how far patients picked for an extreme value drift back on re-measurement, and how many participants a trial needs to detect a difference in utility or cost. The counterpart is between-subject variation, the persistent difference between one patient's usual level and another's.

A patient's usual level and the scatter around it

The simplest description gives each patient a usual level around which readings scatter. A random intercept mixed effects model writes this as follows.

$$Y_{ij} = \mu + u_i + e_{ij}, \qquad u_i \sim N(0, \sigma_b^2), \qquad e_{ij} \sim N(0, \sigma_w^2)$$

where $Y_{ij}$ is reading $j$ on patient $i$, $\mu$ is the population mean, $u_i$ is patient $i$'s persistent departure from that mean, $e_{ij}$ is the departure of reading $j$ from the patient's own level, $\sigma_b^2$ is the between-subject variance and $\sigma_w^2$ is the within-subject variance. With the two terms independent, the total variance and the correlation between two readings on the same patient are as follows.

$$\sigma^2 = \sigma_b^2 + \sigma_w^2, \qquad \rho = \frac{\sigma_b^2}{\sigma_b^2 + \sigma_w^2}$$

where $\sigma^2$ is the variance of a single reading across the population and $\rho$ is the intraclass correlation.

The within-subject term has several sources. Ariel Linden, writing on health care evaluations, attributes the difference between two readings taken even minutes apart to natural biologic variability or measurement error, and calls this within-subject variability at the individual level. In health economic data the occasion itself adds more, since a utility score depends on the day the questionnaire is completed and an annual cost depends on whether an admission happened to fall in that year. Measurement error is therefore one part of within-subject variation, not a synonym for it.

Estimating the within-subject standard deviation

Estimation needs patients measured at least twice under the same conditions. Martin Bland's notes on measurement studies give the within-subject variance for a pair of readings as the squared difference divided by 2, averaged across patients. The within-subject standard deviation $s_w$ is the square root of that average and keeps the units of the outcome, so 0.07 on a utility index means a typical reading sits about 0.07 from the patient's own level.

$$s_w^2 = \frac{1}{n} \sum_{i=1}^{n} \frac{(y_{i1} - y_{i2})^2}{2}$$

where $s_w^2$ is the estimated within-subject variance, $n$ is the number of patients and $y_{i1}$ and $y_{i2}$ are patient $i$'s two readings.

The estimate assumes one within-subject standard deviation for everybody. Bland describes within-subject variation that is proportional to the size of the measurement and then estimates a within-subject coefficient of variation instead, by taking logarithms, finding the within-subject standard deviation on the log scale, taking the antilog and subtracting one. Skewed outcomes such as costs deserve this check, since a fixed scatter in pounds is implausible when usual costs differ widely.

What one reading can and cannot show

The difference between two readings on the same patient has variance equal to twice $\sigma_w^2$, so with Normal errors 95% of such differences lie within 1.96 × 1.4142 × $\sigma_w$, about 2.77 × $\sigma_w$. Bland's notes call this limit the repeatability, the value within which 95% of differences between two readings on the same subject will lie, and note that some round 1.96 to 2. A change smaller than the repeatability coefficient in one patient cannot be told apart from ordinary scatter. Averaging $k$ readings divides the within-subject variance by $k$. The smallest detectable change used in test-retest reliability studies rests on the same arithmetic, and a minimal clinically important difference only helps judge an individual patient when it is larger than this scatter.

Regression to the mean after selection on one reading

When patients are chosen because a single reading is extreme, part of what made the reading extreme was within-subject scatter, and the next reading will on average be closer to the mean even with no intervention. Barnett, van der Pols and Dobson report that the effect becomes more noticeable as measurement error increases and when follow-up is examined only in a sub-sample selected on baseline values. Linden gives the expected size of the effect for Normal data as follows.

$$\text{RTM} = \frac{\sigma_w^2}{\sigma} \, C(z), \qquad z = \frac{\mu - \kappa}{\sigma}, \qquad C(z) = \frac{\phi(z)}{1 - \Phi(z)}$$

where RTM is the expected shift towards the mean in the selected group, $\sigma_w^2$ is the within-subject variance, $\sigma$ is the standard deviation of the baseline readings, $\mu$ is the population mean at baseline, $\kappa$ is the cut-off used to select patients with low values (for high values $z$ is $(\kappa - \mu)/\sigma$), and $\phi$ and $\Phi$ are the standard Normal density and cumulative distribution functions. The shift vanishes when readings are perfectly repeatable.

Linden illustrates the effect with costs. In a cohort of chronically ill health plan members followed for two years with no chronic disease programme in place, the highest cost quintile averaged roughly USD 20,000 to USD 27,000 in the first year across three conditions and roughly USD 7,000 to USD 10,000 in the second. In a control group that received no intervention, members scoring below 44.25 on the SF-12 physical component summary improved by over 8 points within three months. An uncontrolled before-and-after study of a case management programme would book such changes as savings or health gains. Linden calls the randomised controlled trial the obvious remedy, because randomly assigned groups should be equally affected. He also suggests basing assignment on the mean of several pre-tests rather than a single one, which stabilises the mean and reduces within-subject variability.

Baselines, change scores and adjustment in trial-based evaluation

Trial-based economic evaluations usually measure utility at baseline and during follow-up, and within-subject variation decides which analysis of those readings is most precise. Under the random intercept model with variances constant over time, the correlation between baseline and follow-up equals $\rho$, and the variance of one patient's outcome under three common analyses is as follows.

$$V_{\text{post}} = \sigma^2, \qquad V_{\text{change}} = 2\sigma^2(1 - \rho) = 2\sigma_w^2, \qquad V_{\text{ANCOVA}} = \sigma^2(1 - \rho^2)$$

where $V_{\text{post}}$ applies to an analysis of follow-up values alone, $V_{\text{change}}$ to the change from baseline, and $V_{\text{ANCOVA}}$ to analysis of covariance, a regression of follow-up on treatment and baseline. Change scores beat follow-up values only when $\rho$ exceeds 0.5, that is when within-subject variance is smaller than between-subject variance, while covariance adjustment is never worse than either. Frison and Pocock quantify the superiority of analysis of covariance in reduced variance and in avoidance of bias. Borm, Fransen and Lemmens show that analysis of covariance in groups of $(1 - \rho^2)n$ patients has the same power as a t-test in groups of $n$, which feeds into sample size calculation.

The same logic applies to QALYs built from repeated utility readings. Manca, Hawkins and Sculpher note that baseline mean utility is invariably imbalanced between trial arms and is likely to be highly correlated with a patient's QALYs over follow-up, not least because baseline utility usually contributes to the QALY calculation. They show that widely used methods give incorrect differential QALYs under baseline imbalance, whether or not the imbalance is statistically significant, and that multiple regression controlling for baseline utility gives appropriate estimates.

Designs that compare each patient with themselves

A crossover design gives each participant every treatment in turn, so the treatment comparison is made within patients and, when the treatment effect is the same for every patient, its variance depends on $\sigma_w^2$ alone. The gain over a parallel trial grows as between-subject variance dominates. In bioequivalence studies for generic medicines, the EMA guideline on the investigation of bioequivalence (CPMP/EWP/QWP/1401/98 Rev. 1) treats products whose intra-subject variability for a parameter is larger than 30% as highly variable and permits a replicate crossover design for them.

Worked example with repeated utility scores

All numbers in this example are illustrative. A preference-based utility index in a chronic condition has population mean 0.70, between-subject variance 0.0200 and within-subject variance 0.0050.

1. Total variance and correlation. The total variance is 0.0200 + 0.0050 = 0.0250, so $\sigma$ = 0.1581 and $\rho$ = 0.0200/0.0250 = 0.80. The within-subject standard deviation is the square root of 0.0050, or 0.0707.

2. Repeatability. The repeatability coefficient is 1.96 × 1.4142 × 0.0707 = 0.196, so a single patient's change of 0.15 between two visits is within ordinary scatter. The mean of three readings has within-subject variance 0.0050/3 = 0.00167, a standard deviation of 0.041.

3. Regression to the mean. A self-management programme enrols patients whose single baseline score is below 0.55. Then $z$ = 0.15/0.1581 = 0.949, $\phi(z)$ = 0.2544 and the upper tail probability is 0.1714, so $C$ = 0.2544/0.1714 = 1.484. Since 1.484 × 0.1581 = 0.2346, the expected baseline mean of those enrolled is 0.70 minus 0.2346, or 0.465. With no effect, their expected follow-up mean is 0.70 minus 0.2346 × 0.80, or 0.70 minus 0.1877, which is 0.512. The shift of 0.047 matches the formula, since 0.0050/0.1581 × 1.484 = 0.047.

4. The false economic case. If the programme costs GBP 500 per patient and the 0.047 gain were credited as lasting a year, an uncontrolled evaluation would report an apparent ratio of 500/0.0469, about GBP 10,660 per QALY gained, for an ineffective programme.

5. Trial size under three analyses. A trial seeks to detect a difference of 0.05 with two-sided 5% significance and 80% power, so the squared sum of Normal quantiles is 2.8016 squared, or 7.849, and each arm needs 2 × $V$ × 7.849/0.0025 patients, where $V$ is the variance of the analysed outcome. Follow-up values alone, with $V$ = 0.025, need 2 × 0.025 × 7.849/0.0025 = 157 per arm. Change scores, with $V$ = 2 × 0.0050 = 0.010, need 63. Covariance adjustment, with $V$ = 0.025 × 0.36 = 0.009, needs 57.

6. Interpretation. With small within-subject scatter, adjustment for baseline cuts the trial by almost two thirds. Reversing the split, to between-subject variance 0.010 and within-subject variance 0.015, gives $\rho$ of 0.40, and change scores would then need 189 per arm against 157 for follow-up values and 132 for covariance adjustment. Large scatter makes change scores wasteful and lets selection on one reading manufacture apparent gains.

Neighbouring ideas and decision models

Reliability coefficients such as the intraclass correlation express repeatability as a share of total variance, so the same $s_w$ gives a high coefficient in a varied sample and a low one in a homogeneous sample. The standard deviation of single readings across patients mixes both components. A standard error shrinks as patients are added, whereas within-subject variation is a property of the outcome that a larger sample estimates better without reducing. In an individual patient simulation, persistent differences are drawn once per simulated patient and short-lived fluctuation can be redrawn each cycle, so the split between the two variance components shapes the spread of simulated lifetime costs and QALYs.

Common misreadings

The most frequent error is reading a change in patients selected for an extreme value as an effect of whatever was done to them. A second is assuming change from baseline is always the better analysis, which holds only when within-subject variance is the smaller component. A third is treating within-subject variation as pure measurement error and expecting a better instrument to remove it, when much of the scatter in utilities and costs reflects genuine fluctuation in patients' lives.

Sources

• Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology. 2005;34(1):215-220. https://doi.org/10.1093/ije/dyh299
• Bland JM. How should I calculate a within-subject coefficient of variation? Measurement studies notes, University of York; last updated 16 October 2006. https://www-users.york.ac.uk/~mb55/meas/cv.htm
• Bland JM. What is the origin of the formula for repeatability? Measurement studies notes, University of York; last updated 18 February 2005. https://www-users.york.ac.uk/~mb55/meas/repeat.htm
• Borm GF, Fransen J, Lemmens WA. A simple sample size formula for analysis of covariance in randomized clinical trials. Journal of Clinical Epidemiology. 2007;60(12):1234-1238. https://doi.org/10.1016/j.jclinepi.2007.02.006
• European Medicines Agency. Guideline on the investigation of bioequivalence (CPMP/EWP/QWP/1401/98 Rev. 1/Corr). 20 January 2010, effective 1 August 2010. Section 4.1.10. https://www.ema.europa.eu/en/scientific-guidelines/investigation-bioequivalence
• Frison L, Pocock SJ. Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Statistics in Medicine. 1992;11(13):1685-1704. https://doi.org/10.1002/sim.4780111304
• Linden A. Assessing regression to the mean effects in health care initiatives. BMC Medical Research Methodology. 2013;13:119. https://doi.org/10.1186/1471-2288-13-119
• Manca A, Hawkins N, Sculpher MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Economics. 2005;14(5):487-496. https://doi.org/10.1002/hec.944