Imputation

Imputation of Missing Costs and Outcomes in Health Economic Analysis

Imputation fills each gap in a dataset with a value worked out from the information that was recorded, so that people with incomplete records can stay in the analysis. In health economics the gaps usually sit in trial resource-use questionnaires and EQ-5D responses at follow-up visits, and one unanswered item can leave a participant's total cost or QALY undefined. The method chosen can move the incremental cost, the incremental QALYs and the probability that a treatment is judged cost-effective, so the choice belongs within the economic evaluation and is reported with its results. Single imputation puts one value in each gap and treats it as observed, while multiple imputation draws several plausible values and carries the doubt about them into the standard errors. Why data go missing is treated on the missing data page.

What an imputed value stands for

An imputed value is a prediction or a random draw from a model of the missing data given the observed data. It is never recovered information. Its validity rests on an assumption about the missing data mechanism, which can be missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR). Sterne and colleagues describe MAR as an assumption that justifies the analysis rather than a property of the data, and Faria and colleagues note that a descriptive analysis of trial data can separate MCAR from MAR but usually cannot rule out MNAR, because the unobserved values are unknown.

Single imputation methods and the assumptions behind them

Single imputation produces one completed dataset that is analysed as if complete. The methods differ in how the replacement value is chosen, and most need a particular missing data mechanism before even the mean is estimated without bias, while last observation carried forward is biased even under MCAR. All of them understate the standard error, because standard calculations make no distinction between observed and imputed values.

The table follows the summary of ad hoc methods in van Buuren's textbook. Mean imputation shrinks the spread of the variable and weakens its correlations with other variables, while regression imputation strengthens correlations artificially because every imputed point sits on the fitted line. Stochastic regression imputation adds a residual draw and preserves correlations, but one draw still gives too small a standard error. Last observation carried forward assumes the outcome stays constant after dropout.

For within-trial cost-effectiveness analysis, Faria and colleagues conclude that single imputation methods are not appropriate for missing outcomes. Missing baseline covariates are a different case. Mean imputation of a baseline value keeps the imputed values independent of treatment allocation, and they list it beside multiple imputation as a good option there.

Multiple imputation and Rubin's rules

Multiple imputation replaces each missing value with a set of values drawn from the posterior predictive distribution of the missing data given the observed data. The regression parameters are first drawn from their posterior distribution and the missing values are then drawn given those parameters, and repeating this $m$ times gives $m$ completed datasets. Each is analysed in the usual way and the results are combined by the rules Rubin set out in 1987.

$$T = \bar{U} + \left(1 + \frac{1}{m}\right) B$$

Here $T$ is the total variance of the pooled estimate, $\bar{U}$ is the average of the complete-data variances across the imputed datasets, $B$ is the variance of the $m$ point estimates around their mean and $m$ is the number of imputations. The pooled estimate is the simple average of the $m$ estimates. The term $B$ is the extra variance caused by the missing values and $B/m$ is simulation error from using a finite number of imputations. A single imputation has no way to estimate either.

Sterne and colleagues suggest at least 20 imputations may be preferable to the five once thought sufficient, and Faria and colleagues give a rule of thumb that the number should be similar to the percentage of incomplete cases. Joint modelling assumes the variables are multivariate normal, while multiple imputation by chained equations fits one model per variable, such as Poisson regression for counts, and can derive costs passively from imputed resource use.

Modelling choices for trial cost and outcome data

Costs and QALYs are cumulative totals built from components collected at several visits, they are rarely normally distributed and they tend to be correlated. Faria and colleagues describe trial QALYs as typically bimodal, left skewed and with a spike at 1, and note that costs can have a large proportion of zeros.

Variables in the imputation model. The model should include every covariate in the analysis model, the outcome itself and variables that predict missingness, even when those predictors play no part in the final analysis. Sterne and colleagues show that leaving the outcome out of the model used to impute a predictor falsely weakens the association.

Imputing by randomised arm. Faria and colleagues recommend running the imputation separately in each randomised group. Imputing the groups together with interaction terms lets the means differ by arm but not the covariance structure.

Level of aggregation. Costs can be imputed as resource-use counts or as total cost, and health outcomes as EQ-5D domains, index scores or QALYs. Faria and colleagues suggest total cost when all resource items share one missing data pattern, and resource-level imputation, at least for the main cost drivers, when patterns differ. A simulation by Simons and colleagues found domain imputation more accurate when individual EQ-5D-3L items were missing and index imputation more accurate in samples under 100.

Skewness and zeros. A normal imputation model can produce negative costs or utilities above 1. Remedies are imputing on a log scale and back-transforming, or predictive mean matching, which fills each gap with the observed value of a donor whose predicted value is close, so imputations never fall outside the observed range. Testing up to 50% missing trial data, MacNeil Vroomen and colleagues found predictive mean matching on log-transformed costs the most stable strategy and recommended it above 35% missing.

Assumptions made before imputation. In the REFLUX trial analysis used by Faria and colleagues, a blank resource item in an otherwise completed questionnaire was taken to mean no use. Such assumptions sit outside the imputation model and should be stated.

Illustrative example of mean and conditional imputation of trial costs

The numbers below are illustrative. A trial arm has eight participants. Four have mild disease and observed 12-month costs of GBP 1,000, 1,200, 1,400 and 1,000. Four have severe disease, two with observed costs of GBP 3,000 and 3,400 and two with missing costs. Missingness depends on severity only, so the data are MAR given severity.

1. Complete-case mean. The six observed costs sum to GBP 11,000, so the complete-case mean is 11,000 / 6 = 1,833.33 pounds.

2. Mean imputation. Each missing cost is set to GBP 1,833.33, so the arm mean stays at GBP 1,833.33. The sum of squared deviations from the mean is 5,793,333 in both datasets, but the divisor grows, so the standard deviation falls.

$$s_{\text{obs}} = \sqrt{\frac{5{,}793{,}333}{5}} = 1{,}076.4 \qquad s_{\text{mean-imp}} = \sqrt{\frac{5{,}793{,}333}{7}} = 909.7$$

Here $s_{\text{obs}}$ is the sample standard deviation of the six observed costs and $s_{\text{mean-imp}}$ is the standard deviation after mean imputation, both in GBP. Dividing each by the square root of its number of values gives a naive standard error of 321.6 pounds with eight values against 439.4 pounds for the six observed values. Precision has apparently improved although no information was added.

3. Conditional mean imputation on severity. The observed severe costs average (3,000 + 3,400) / 2 = 3,200 pounds, and each missing severe cost is set to that value. The arm mean becomes (11,000 + 6,400) / 8 = 2,175 pounds.

4. Size of the bias. The gap between 2,175 and 1,833.33 is 341.67 pounds, so mean imputation understates the arm mean by about 15.7% of the conditional estimate, because the people with missing costs were disproportionately severe.

5. What conditional imputation still gets wrong. The two imputed severe costs are identical, so the standard deviation within the severe group drops to 163.3 with the imputed values included, against 282.8 for the two observed severe costs. A stochastic version adds a residual draw. Illustrative residuals of minus 150 and plus 250 give imputed costs of GBP 3,050 and 3,450 and an arm mean of (11,000 + 6,500) / 8 = 2,187.5 pounds, while residuals of plus 100 and minus 300 give an arm mean of (11,000 + 6,200) / 8 = 2,150 pounds. The spread between such completed-data estimates is what $B$ measures.

If more severe patients were missing in one arm than the other, mean imputation would bias the incremental cost as well. Conditional or multiple imputation corrects that bias only if severity explains the missingness, which is the MAR assumption.

How imputation changes cost-effectiveness results

In the non-small cell lung cancer trial described by Sterne and colleagues, costs were complete for 82 of 115 patients. Complete-case analysis gave a mean cost difference of GBP 2,804 and a net monetary benefit of minus GBP 3,346, while multiple imputation gave GBP 2,384 and GBP 1,186, reversing the conclusion that chemotherapy was not cost-effective. In the REFLUX reanalysis by Faria and colleagues, the cost difference was GBP 1,668 by complete-case analysis and GBP 1,305 by multiple imputation, the QALY difference was 0.301 against 0.244, and the probability that surgery was cost-effective at GBP 20,000 per QALY gained was 0.98 against 0.96. The REFLUX decision held, although both incremental estimates shrank.

After multiple imputation, incremental costs and QALYs can be pooled with Rubin's rules from a seemingly unrelated regression in each dataset, or each imputed dataset can be bootstrapped and the probability of cost-effectiveness averaged across imputations. Either route feeds the cost-effectiveness acceptability curve and the incremental net benefit.

Leurent and colleagues reviewed 52 trial-based cost-effectiveness analyses published in the Health Technology Assessment journal between 2013 and 2015. The median proportion of participants with complete cost-effectiveness data was 63%, complete-case analysis was the most common primary approach at 43%, multiple imputation followed at 30%, and only 2 studies (4%) examined departures from MAR.

Imputation beside deletion, weighting and likelihood methods

Complete-case analysis is valid under MCAR, or when the analysis model includes the baseline variables that predict both outcome and missingness, but it discards information. Inverse probability weighting suits a monotone pattern in which people who leave do not return. Likelihood-based models, such as mixed (multilevel) models, use all observed data under MAR without an imputation step, but they can condition only on covariates in the analysis model, while an imputation model can also use auxiliary variables such as post-randomisation measures.

Imputation is also a common vehicle for MNAR sensitivity analysis. In the pattern-mixture model approach, values are imputed under MAR and then shifted by a chosen amount, for example lowering imputed QALYs in one arm, while a selection model specifies how the probability of being missing depends on the unobserved value.

Misreadings of imputed data

Most errors in applied work come from treating imputed values as data rather than as model output. Three recur beyond the understated standard errors of single imputation.

• Reporting complete-case analysis and multiple imputation side by side does not show that a conclusion withstands the missing data. Leurent and colleagues call this misleading, because both rely on missingness being independent of the unobserved values.
• Building a ratio from separately imputed components can create extreme values, one of the causes Sterne and colleagues identify for a null cholesterol effect in an early version of the QRISK cardiovascular risk tool.
• Plausible-looking imputed values say nothing about whether MAR holds. Comparing observed and imputed distributions checks the imputation model, not the mechanism.

Sources

• Faria R, Gomes M, Epstein D, White IR. A guide to handling missing data in cost-effectiveness analysis conducted within randomised controlled trials. PharmacoEconomics. 2014;32(12):1157-1170. https://doi.org/10.1007/s40273-014-0193-3
• Leurent B, Gomes M, Carpenter JR. Missing data in trial-based cost-effectiveness analysis: an incomplete journey. Health Economics. 2018;27(6):1024-1040. https://doi.org/10.1002/hec.3654
• MacNeil Vroomen J, Eekhout I, Dijkgraaf MG, van Hout H, de Rooij SE, Heymans MW, Bosmans JE. Multiple imputation strategies for zero-inflated cost data in economic evaluations: which method works best? European Journal of Health Economics. 2016;17(8):939-950. https://doi.org/10.1007/s10198-015-0734-5
• Simons CL, Rivero-Arias O, Yu LM, Simon J. Multiple imputation to deal with missing EQ-5D-3L data: should we impute individual domains or the actual index? Quality of Life Research. 2015;24(4):805-815. https://doi.org/10.1007/s11136-014-0837-y
• Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, Wood AM, Carpenter JR. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393. https://doi.org/10.1136/bmj.b2393
• van Buuren S. Flexible Imputation of Missing Data. 2nd ed. Chapman and Hall/CRC; 2018. Sections 1.3, 2.1, 2.3 and 3.4. https://stefvanbuuren.name/fimd/