Linear Regression

Linear Regression in Health Economic Analysis

Linear regression describes the expected value of a continuous outcome as an intercept plus a coefficient times each explanatory variable, so every coefficient reads as a change in the outcome's own units per unit of its predictor. In health economics the outcome is usually a patient's cost, QALYs or utility, and a coefficient might give the extra annual hospital spending linked to one more chronic condition. Predictions stay on the natural scale, giving the arithmetic mean costs and mean differences that budgets and incremental analyses need. Its limits follow from the same simplicity, since patient costs are skewed and utilities are bounded, and the NICE Decision Support Unit regards the linear model as unsuitable for mapping EQ-5D values. The wider family is described under regression analysis, and adjustment of trial costs and QALYs for baseline covariates under multiple regression.

The model and the least squares estimator

Each observation is a systematic part, linear in unknown coefficients, plus a random error. Linearity refers to the coefficients, so a predictor can enter as a square or a logarithm and the model is still a linear regression.

$$y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + \dots + \beta_k x_{ki} + \varepsilon_i$$

where $y_i$ is the outcome for patient $i$, $x_{1i}$ to $x_{ki}$ are that patient's values of the $k$ explanatory variables, $\beta_0$ is the intercept, $\beta_j$ is the coefficient on $x_j$ and $\varepsilon_i$ is an error term with mean zero.

Ordinary least squares (OLS) picks the coefficients that minimise the sum of squared residuals, the gaps between observed and fitted values. In matrix form the solution is below, although software uses matrix factorisation instead of an explicit inverse, as the page on least squares approximation explains.

$$\hat{\boldsymbol\beta} = (X^{\mathsf T}X)^{-1}X^{\mathsf T}\mathbf{y}$$

where $\hat{\boldsymbol\beta}$ is the vector of estimated coefficients, $X$ is the design matrix with one row per patient and one column per coefficient (including a column of ones for the intercept), $X^{\mathsf T}$ is its transpose and $\mathbf{y}$ is the vector of outcomes.

With a single predictor the estimates reduce to two short formulas.

$$\hat\beta_1 = \frac{\sum_i (x_i - \bar x)(y_i - \bar y)}{\sum_i (x_i - \bar x)^2}, \qquad \hat\beta_0 = \bar y - \hat\beta_1 \bar x$$

where $\bar x$ and $\bar y$ are the sample means of the predictor and the outcome, and the sums run over all patients. The fitted line therefore passes through the point of means, and with an intercept the residuals sum to zero.

What each assumption buys

Least squares can always be computed, but what its numbers mean depends on assumptions, each supporting a different claim. If the error has mean zero given the predictors, sampling is random and no predictor is an exact linear combination of the others, the OLS coefficients are unbiased. Adding constant error variance makes OLS the most precise linear unbiased estimator, the Gauss-Markov result, and makes the textbook standard errors correct. Normal errors matter only for exact small-sample t and F tests, since in larger samples the central limit theorem makes the estimates approximately normal.

Linear regression on patient-level costs

Thompson and Barber argue that for policy decisions the arithmetic mean cost is the informative measure, because multiplied by the number of patients it gives the total cost of treating them, which medians and geometric means do not. OLS on untransformed costs targets that quantity. With a treatment indicator as the only predictor, the slope equals the difference in arithmetic mean costs between arms and its test is the pooled-variance t test, so the regression extends the t test to allow covariates.

Mihaylova and colleagues, reviewing methods for healthcare costs, note that OLS presents results on the scale of interest and gives unbiased estimates for randomised data, but is sensitive to extreme values and likely to be inefficient in small to medium samples when the distribution is not normal. Generalised least squares or a Huber-White sandwich estimator of the variance is often used for such data.

Jones calls linear regression on the level of costs a natural starting point, quick to estimate even with hundreds of regressors and millions of observations, as in risk adjustment models built on administrative data. He notes that individual cost data typically show a high degree of heteroskedasticity and that the norm is robust standard errors for inference. In his worked application the linear model produced some negative predicted costs, yet in a comparison by Hill and Miller on US Medical Expenditure Panel Survey data, linear OLS was among the best performing models and OLS on log costs performed poorly. The skewness of the data and the purpose of the analysis guide the choice between OLS and a generalized linear model.

Worked example with five patients and one predictor

All numbers in this example are illustrative, and the sample is tiny so every step can be checked by hand. Annual hospital cost is regressed on the number of chronic conditions.

1. Compute the means. The mean number of conditions is (0 + 1 + 2 + 3 + 4) / 5 = 2, and the mean cost is 9,500 / 5 = GBP 1,900.

2. Compute the slope. Deviations in conditions are -2, -1, 0, 1 and 2, with squares summing to 10. Cost deviations are -1,300, -200, -400, 700 and 1,200, and the cross-products are 2,600, 200, 0, 700 and 2,400, summing to 5,900. The slope is 5,900 / 10 = GBP 590 per condition.

3. Compute the intercept. The intercept is 1,900 minus 590 × 2 = GBP 720, the predicted cost for a patient with no chronic conditions.

4. Check the fit. Fitted costs are 720, 1,310, 1,900, 2,490 and 3,080, giving residuals of -120, 390, -400, 110 and 20, summing to zero. The residual sum of squares is 14,400 + 152,100 + 160,000 + 12,100 + 400 = 339,000 and the total is 1,690,000 + 40,000 + 160,000 + 490,000 + 1,440,000 = 3,820,000, so R-squared is 1 minus 339,000 / 3,820,000, or 0.911.

5. Quantify uncertainty. With five patients and two coefficients there are 3 degrees of freedom, so the residual variance is 339,000 / 3 = 113,000 and the slope's standard error is the square root of 113,000 / 10, or 106.3. The 97.5th percentile of the t distribution with 3 degrees of freedom is 3.182, and 3.182 × 106.3 = 338.3, so the 95% confidence interval for the slope lies between about GBP 252 and GBP 928.

6. Predict for a model cohort. A cohort averaging 2.6 conditions has a predicted mean cost of 720 + 590 × 2.6 = GBP 2,254, which in a linear model equals the average of the individual predictions, a property a log-link model lacks. A prediction for six conditions, 720 + 590 × 6 = GBP 4,260, lies outside the observed range and assumes the line holds there.

7. Interpret. Each extra chronic condition goes with about GBP 590 more annual cost, with a wide interval. The slope is an association, so it does not show what preventing one condition would save unless confounding, for example by age, is ruled out.

Mapping utilities, where the linear model fails

A mapping function predicts a preference-based health state utility value, usually EQ-5D, from a clinical or condition-specific measure. NICE DSU Technical Support Document 22 describes EQ-5D data as bounded above at full health, with many observations at full health, a substantial gap to the next feasible value (0.883 in the UK 3L value set) and a multimodal, skewed remainder. It states that linear regression, widely used in the mapping literature, is considered inappropriate because it underestimates health benefits for people in good health and overestimates them for people in poor health, which leads to misleading cost-effectiveness estimates.

Its recommendations add that unsuitable models generate infeasible values and misrepresent uncertainty, and it cites a study by Lamu and Olsen in which linear regression gave predictions above 1 when mapping from the QLQ-C30 in cancer patients, and asks analyses using such models to report how often out-of-range predictions occurred and how they were handled. Mixture model approaches performed very well in the case studies it draws on.

Regression coefficients as inputs to decision models

Decision models often take a cost or utility from a fitted regression. The ISPOR-SMDM task force on parameter estimation likens coefficient standard errors to parameter uncertainty and the form of the regression to structural uncertainty. It notes that the covariance matrix of the coefficients defines the uncertainty in such a derived parameter and that multivariate normality is appropriate for the linear predictor. Its best practice VI-10 adds that correlation between jointly estimated parameters, such as those from a regression, should be reflected in the analysis. In probabilistic sensitivity analysis the coefficients are therefore drawn together, usually through a Cholesky decomposition of the covariance matrix. When the mean of the predictor is positive, the intercept and slope estimates are negatively correlated, so drawing them independently overstates the spread of predicted costs near the centre of the data.

Linear regression metamodels of simulation results

Linear regression can also be fitted to the output of a decision model. Jalal and colleagues proposed regressing the outcomes of a probabilistic analysis on standardised inputs to summarise and communicate the model.

$$y_s = \beta_0 + \sum_{j=1}^{m} \beta_j z_{js} + u_s, \qquad z_{js} = \frac{x_{js} - \bar x_j}{\sigma_j}$$

where $y_s$ is a model outcome such as cost or net health benefit in simulation $s$, $x_{js}$ is the sampled value of input parameter $j$, $\bar x_j$ and $\sigma_j$ are that input's mean and standard deviation, $z_{js}$ is its standardised value, $m$ is the number of inputs and $u_s$ is a residual.

In their cancer-cure example with 10,000 simulated cohorts the intercept equals the base-case outcome with all parameters at their means, and each coefficient is the change in outcome for a one standard deviation change in its input, so absolute coefficients rank inputs by relative uncertainty. The metamodel gave results similar to one-way and two-way deterministic sensitivity analysis but was more accurate because it used all parameter values.

Linear regression beside neighbouring models

Two close relatives keep the linear predictor but change what it predicts. Logistic regression applies it to the log odds of a binary event, such as a hospital admission, and is fitted by maximum likelihood because a binary outcome has no normal error. A generalized linear model links the predictor to the mean of a skewed cost or a count, and Jones notes that a Gaussian distribution with an identity link is comparable to linear regression.

Misreadings and limits

Most errors with linear regression come from reading the coefficients as more than the model supports. Five recur often.

1. Reading an association as an effect. Outside a randomised comparison a coefficient carries the influence of any omitted confounder.
2. Trusting default standard errors for cost data. Patient costs are usually heteroskedastic, so robust or bootstrap errors are the usual basis for inference.
3. Using the linear model for bounded outcomes. For EQ-5D mapping TSD 22 judges it inappropriate, and predictions can exceed full health.
4. Treating R-squared as a verdict on the model. A high value says nothing about bias in the coefficient of interest, and a low one is common for individual costs.
5. Extrapolating the line. Predictions outside the observed range, including negative costs, rest on a linearity the data cannot check.

Sources

• Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical Decision Making. 2012;32(5):722-732. https://doi.org/10.1177/0272989X12458348
• Jalal H, Dowd B, Sainfort F, Kuntz KM. Linear regression metamodeling as a tool to summarize and present simulation model results. Medical Decision Making. 2013;33(7):880-890. https://doi.org/10.1177/0272989X13492014
• Jones AM. Models for health care. HEDG Working Paper 10/01. University of York; January 2010. https://www.york.ac.uk/media/economics/documents/herc/wp/10_01.pdf
• Mihaylova B, Briggs A, O'Hagan A, Thompson SG. Review of statistical methods for analysing healthcare resources and costs. Health Economics. 2011;20(8):897-916. https://doi.org/10.1002/hec.1653
• Thompson SG, Barber JA. How should cost data in pragmatic randomised trials be analysed? BMJ. 2000;320(7243):1197-1200. https://doi.org/10.1136/bmj.320.7243.1197
• Wailoo A, Hernández Alava M, Pudney S. NICE DSU Technical Support Document 22: Mapping to estimate health state utilities (updates and replaces TSD 10). Decision Support Unit; June 2023, updated September 2026. Sections 4.2, 5.2 and 7. https://doi.org/10.15131/shef.data.33849340