Logistic Regression

Logistic Regression for Event Probabilities in Health Economic Models

Logistic regression is the standard regression for an outcome that either happens or does not within a set period, such as an emergency admission, a treatment response or a death. It links patient characteristics to the probability of that event through the log odds, so every prediction stays between 0 and 1 and every coefficient converts to an odds ratio. In health economics it supplies event probabilities to decision models and adjusts comparisons for differences between patients. The harder step is turning a fitted odds ratio into the absolute probabilities a Markov model needs.

The logit link and the fitted equation

Each outcome is coded 1 if the event occurs during follow-up and 0 if not. A straight line fitted to such data can predict values outside 0 and 1, one reason linear regression suits continuous outcomes better than binary ones. Logistic regression instead models the log of the odds of the event as a linear function of the covariates.

$$\ln\left(\frac{p_i}{1 - p_i}\right) = \beta_0 + \beta_1 x_{1i} + \cdots + \beta_k x_{ki} = \eta_i$$

where $p_i$ is the probability that patient $i$ has the event within the follow-up period of the data, $x_{1i}$ to $x_{ki}$ are that patient's $k$ covariates, $\beta_0$ is the intercept (the log odds when all covariates are zero), $\beta_1$ to $\beta_k$ are the coefficients, each the change in log odds per unit of its covariate with the others fixed, and $\eta_i$ is the linear predictor. Solving for the probability gives the logistic curve.

$$p_i = \frac{1}{1 + \exp(-\eta_i)}$$

where $\exp$ is the exponential function. The curve is S-shaped, so a fixed change in the linear predictor moves the probability most for patients near 0.5.

NICE Decision Support Unit Technical Support Document 2 (TSD 2) describes the familiar logistic model for binomial data as a generalised linear model with a logit link function, appropriate for probability outcomes. The coefficients are estimated by maximum likelihood estimation, which maximises the following log-likelihood iteratively, since there is no closed-form solution.

$$\ln L(\boldsymbol{\beta}) = \sum_{i=1}^{n} \left[ y_i \ln p_i + (1 - y_i) \ln(1 - p_i) \right]$$

where $\boldsymbol{\beta}$ is the vector of coefficients, $n$ is the number of patients, $y_i$ is 1 if patient $i$ had the event and 0 otherwise, and $p_i$ depends on $\boldsymbol{\beta}$ through the linear predictor. Nested models are compared with a likelihood ratio test.

Coefficients, odds ratios and risk

Exponentiating a coefficient gives the odds ratio for a one-unit change in that covariate. Gidwani and Russell, in a tutorial for decision modellers, note that the coefficients of logistic regressions are logged odds ratios.

$$\text{OR}_j = \exp(\beta_j)$$

where $\text{OR}_j$ is the odds ratio for covariate $j$ and $\beta_j$ is its coefficient. Zhang and Yu warn that when the outcome is common in the study population (more than 10%), the adjusted odds ratio from a logistic regression no longer approximates the risk ratio, overstating it when above 1 and understating it when below 1. The relative risk that matches an odds ratio depends on the baseline probability.

$$\text{RR} = \frac{\text{OR}}{1 - p_0 + p_0 \times \text{OR}}$$

where $\text{RR}$ is the relative risk, $\text{OR}$ is the odds ratio and $p_0$ is the probability of the event in the unexposed group. Gidwani and Russell show an odds ratio of 1.5 matching a relative risk of 1.429 when $p_0$ is 0.1 but 1.071 when $p_0$ is 0.8. They add that a multivariable model has a different baseline risk for each combination of covariates, and that papers rarely publish the intercept needed to recover one.

Turning a fitted model into decision model inputs

TSD 5 describes most cost-effectiveness analyses as a baseline model for the comparator combined with relative treatment effects. It applies the odds ratio to the baseline on the logit scale.

$$\text{logit}(p_1) = \text{logit}(p_0) + \ln(\text{OR})$$

where $\text{logit}(x) = \ln(x/(1 - x))$, $p_0$ is the baseline risk on the comparator and $p_1$ is the probability on the new treatment. Working on this scale keeps the result inside the unit interval, which multiplying a probability by a relative risk does not guarantee. TSD 5 calls risk equations fitted to individual patient data a far more reliable basis for a baseline model than aggregate data, with trial treatment effects added to the equations as if they were another risk factor.

A predicted probability carries the time frame of the data, so a model fitted to one-year outcomes predicts one-year probabilities. For a state with only two possible transitions, Gidwani and Russell give the standard relationship for changing it to the model's cycle length.

$$r = \frac{-\ln(1 - p)}{t}, \qquad p = 1 - \exp(-rt)$$

where $p$ is the probability of the event over time $t$ and $r$ is the corresponding rate, assumed constant over that time. They note that these two-state formulas are not appropriate when a state has three or more exits, a case covered on the transition probability and rate to probability conversion pages.

Regression coefficients are estimated jointly, so their uncertainty is shared. The ISPOR-SMDM task force report on parameter estimation states that the covariance matrix defines the uncertainty and correlation in the coefficients and that multivariate normality is appropriate for the regression's linear predictor. In a probabilistic sensitivity analysis, each iteration therefore draws a whole coefficient vector from a multivariate normal distribution and recomputes every probability from that draw.

Worked example with an admission risk model and a care programme

All numbers in this example are illustrative. A logistic regression fitted to registry data predicts the one-year probability of an emergency admission in adults with heart failure. The intercept is minus 1.20 for a patient aged 70 with no admission in the previous year on usual care. Age adds 0.04 per year over 70, a previous admission adds 0.70 and a new community care programme adds minus 0.36.

1. Predict for a higher-risk patient. For a patient aged 75 with a previous admission on usual care, $\eta = -1.20 + 0.04 \times 5 + 0.70 = -0.30$, so the probability is $p = 1/(1 + e^{0.30}) = 1/(1 + 1.3499) = 0.4256$.

2. Add the programme. The linear predictor becomes $-0.30 - 0.36 = -0.66$, so $p = 1/(1 + e^{0.66}) = 1/(1 + 1.9348) = 0.3407$. The odds ratio is $e^{-0.36} = 0.698$, while the relative risk is $\text{RR} = 0.3407/0.4256 = 0.801$, matching the conversion formula, $\text{RR} = 0.698/(1 - 0.4256 + 0.4256 \times 0.698) = 0.801$.

3. Repeat for a lower-risk patient. For a patient aged 70 with no previous admission, $\eta = -1.20$ and $p = 1/(1 + 3.3201) = 0.2315$. With the programme, $\eta = -1.56$ and $p = 1/(1 + 4.7588) = 0.1736$, a relative risk of 0.750. The same odds ratio gives an absolute risk reduction of 0.0578 here against 0.0848 for the higher-risk patient, about 58 against 85 admissions prevented per 1,000 patients a year.

4. Convert to a three-month cycle. For the higher-risk patient on usual care, the annual rate is $-\ln(1 - 0.4256) = 0.5544$, and the three-month probability is $p = 1 - \exp(-0.5544/4) = 1 - 0.8706 = 0.1294$. Dividing the annual probability by four would give 0.1064, too low.

5. Average over a cohort. A cohort split equally between the two usual-care patients has a mean annual probability of $\bar{p} = (0.4256 + 0.2315)/2 = 0.3285$. Using the mean covariates instead, an age of 72.5 and half a previous admission, gives $\eta = -1.20 + 0.10 + 0.35 = -0.75$ and $p = 1/(1 + 2.1170) = 0.3208$. Because the curve is not a straight line, the probability at the average patient is not the average probability, and a cohort model built on one set of mean covariates misstates the expected number of events. The programme's value in a cost-effectiveness analysis therefore depends on the case mix it reaches.

Logistic models in evidence synthesis, costs and observational data

TSD 2 builds its core models for pairwise and network meta-analysis on binomial data with a logit link, in which each trial has its own baseline log odds and the pooled treatment effect is a log odds ratio. For patient-level resource use and costs, Mihaylova and colleagues describe two-part models for data with many non-users, in which a logit or probit model estimates the probability of any use or cost and a second model estimates the mean among users.

In observational comparisons, a logistic regression of treatment received on baseline characteristics estimates the propensity score. Austin states that the propensity score is most often estimated this way. NICE DSU TSD 17 asks for a sufficiently flexible propensity score model and suggests a probit or logit including squares, polynomials and interactions of the covariates. Austin adds that the model is built to balance baseline covariates and reduce confounding, not to achieve statistically significant coefficients.

Checking a logistic model before it feeds a decision model

A risk equation is only as useful as its predictions in the modelled population. Steyerberg and colleagues list the concordance (c) statistic, equal to the area under the ROC curve, for discrimination and goodness-of-fit statistics for calibration, and suggest that reporting both will always be important for a prediction model. Calibration, whether predicted probabilities match observed event rates, matters directly for a decision model because the probabilities become inputs. The Hosmer-Lemeshow test is one such check.

Small datasets give unreliable coefficients. In a simulation study, Peduzzi and colleagues found no major problems with 10 or more events per variable, but below 10 the coefficients were biased in both directions and confidence limits lacked proper coverage. TSD 5 names the main difficulty with risk equations as justifying the choice of data source and its relevance to the target population, and calls for sensitivity analyses where necessary.

Where logistic regression sits among nearby models

Logistic regression is one member of regression analysis, chosen because the outcome is binary. TSD 17 suggests logit or probit models as possible candidates for binary outcomes. A probit model uses the standard normal distribution function in place of the logistic curve, and its coefficients do not convert to odds ratios. More than two unordered categories need the multinomial logit model. When the time to an event matters, or patients leave follow-up early, a binary outcome at a fixed time discards information and mishandles censoring, and survival regression fits better. Repeated events per patient are counts, which suit Poisson regression.

Common misreadings

The errors below arise when a fitted model is carried into a decision model. Each changes the event counts that drive costs and QALYs.

1. Reading an odds ratio as a relative risk. With common events the two differ, as the worked example shows.
2. Applying one odds ratio to one baseline for everyone. Absolute benefit varies with baseline risk, so patient-level or subgroup predictions are needed when case mix differs.
3. Dividing a probability to fit a shorter cycle. The conversion goes through the rate, and only for states with two exits.
4. Sampling coefficients independently in a PSA. Ignoring their correlation misstates the spread of the derived probabilities.

Sources

• Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research. 2011;46(3):399-424. https://doi.org/10.1080/00273171.2011.568786
• Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical Decision Making. 2012;32(5):722-732. https://doi.org/10.1177/0272989X12458348
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. Decision Support Unit; August 2011, last updated September 2016. https://www.sheffield.ac.uk/sites/default/files/2022-02/TSD2-General-meta-analysis-corrected-2Sep2016v2.pdf
• Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 5: Evidence synthesis in the baseline natural history model. Decision Support Unit; August 2011, last updated April 2012. https://www.sheffield.ac.uk/nice-dsu/tsds/evidence-synthesis
• Faria R, Hernandez Alava M, Manca A, Wailoo AJ. NICE DSU Technical Support Document 17: The use of observational data to inform estimates of treatment effectiveness in technology appraisal: methods for comparative individual patient data. Decision Support Unit; May 2015. https://www.sheffield.ac.uk/nice-dsu/tsds/observational-data
• Gidwani R, Russell LB. Estimating transition probabilities from published evidence: a tutorial for decision modelers. PharmacoEconomics. 2020;38(11):1153-1164. https://doi.org/10.1007/s40273-020-00937-z
• Mihaylova B, Briggs A, O'Hagan A, Thompson SG. Review of statistical methods for analysing healthcare resources and costs. Health Economics. 2011;20(8):897-916. https://doi.org/10.1002/hec.1653
• Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology. 1996;49(12):1373-1379. https://doi.org/10.1016/s0895-4356(96)00236-3
• Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
• Zhang J, Yu KF. What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA. 1998;280(19):1690-1691. https://doi.org/10.1001/jama.280.19.1690