Missing Completely at Random

Missing Completely at Random in Trial-Based Economic Evaluation

Missing completely at random (MCAR) describes data in which the missing records, such as unreturned resource-use questionnaires or EQ-5D forms in a trial, form a purely random subset of all the records that should exist. The people with complete data then look, on average, like everyone who was randomised, so the gaps cost precision but do not distort the means. It is the most restrictive of the three missing data mechanisms in Rubin's 1976 classification, beside missing at random (MAR) and missing not at random (MNAR). In a trial-based cost-effectiveness analysis, MCAR is the condition under which a simple complete-case comparison of mean costs and QALYs is unbiased, and it is the only one of the three that observed data can sometimes refute. Observed data can never confirm it, because any test can compare only the values that were recorded.

The formal condition in Rubin's framework

Rubin treated each data value as having some probability of being missing and called the process behind these probabilities the missing data mechanism. Van Buuren writes the missing data model as the probability of a pattern of missingness given the observed values, the unobserved values and the model's parameters. MCAR is the case in which that probability depends on none of the data.

$$\Pr(R = 0 \mid Y_{\mathrm{obs}}, Y_{\mathrm{mis}}, \psi) = \Pr(R = 0 \mid \psi)$$

Here $R$ is the response indicator, equal to 1 when a value is observed and 0 when it is missing, $Y_{\mathrm{obs}}$ is the observed part of the data, $Y_{\mathrm{mis}}$ is the missing part and $\psi$ holds the parameters of the missing data model, so the probability depends only on the overall missingness rate. MAR lets the right-hand side depend on $Y_{\mathrm{obs}}$, and under MNAR it still depends on $Y_{\mathrm{mis}}$. Faria and colleagues give the practical reading for trials, since under MCAR the outcomes of the individuals with data are a representative sample of those of the whole trial population.

Rubin's 1976 paper showed that likelihood and Bayesian inference can ignore the process causing missing data under MAR with distinct parameters, while sampling distribution inference also needs the observed data to be observed at random. MCAR data are both missing at random and observed at random, so they meet the sampling distribution condition, and van Buuren notes that most simple fixes for missing data work only under the restrictive and often unrealistic MCAR assumption.

Covariate-dependent missingness

Trials record baseline covariates for almost everyone, and the strict definition counts them as data. Li, following Little in 1995, defines MCAR as missingness independent of the missing values, the observed values and fully observed covariates, and treats missingness that depends only on those covariates as covariate-dependent missingness, a special case of MAR. Faria and colleagues label the same covariate-dependent missingness CD-MCAR, a distinction useful in trials because missingness may depend on baseline characteristics such as age without depending on earlier outcomes.

Where the assumption is credible in trial data

The standard examples share a cause of missingness unrelated to the patient. The European Medicines Agency guideline on missing data in confirmatory clinical trials (EMA/CPMP/EWP/1776/99 Rev. 1, 2010) gives a patient moving to another city for non-health reasons, whose dropout could be considered a random and representative sample of the study population. Van Buuren adds that units left out of a simple random sample are MCAR, so a trial that collected detailed resource use in a random subsample would create missing data of this kind.

In health economic data the assumption rarely holds. Leurent and colleagues regard methods valid under MAR as more plausible than the more restrictive MCAR assumption. In the REFLUX trial of surgery for gastro-oesophageal reflux disease, reanalysed by Faria and colleagues, a higher baseline EQ-5D score went with lower odds of missing cost data (odds ratio 0.38, 95% confidence interval 0.16 to 0.90) and of missing QALY data (odds ratio 0.46, 95% confidence interval 0.19 to 1.09, significant at the 10% level). The authors judged the data unlikely to be MCAR and analysed them under MAR.

What MCAR permits in an economic evaluation

Faria and colleagues state that complete case analysis and available case analysis are valid under MCAR, and under CD-MCAR if the analysis model includes the baseline predictors of both outcome and missingness. Under MCAR, van Buuren adds, listwise deletion gives unbiased means, variances and regression weights, with standard errors correct for the smaller subset. Faria and colleagues place inverse probability weighting with no predictors of missingness under the same assumption.

MCAR does not make every simple method safe. Faria and colleagues note that last value carried forward can bias estimates even when data are MCAR, and Gabrio and colleagues point out that complete case analysis stays inefficient under MCAR because it ignores the information in partially observed cases. Hughes and colleagues state that multiple imputation gives unbiased results under MCAR or MAR and can use incomplete cases to improve precision.

Hughes and colleagues call it a widespread belief that complete-case analysis needs MCAR, since for most regression models it is unbiased when the chance of being a complete case does not depend on the outcome given the covariates. Little and colleagues still advise against complete-case analysis because it requires the unrealistic MCAR assumption. The difference lies in the estimand. Van Buuren notes that quantities depending on the marginal distribution of the outcome, such as its mean, require MCAR, so an unadjusted complete-case arm mean cost or QALY needs it even where a covariate-adjusted coefficient does not.

Testing MCAR with the observed data

Because the assumption covers the observed data too, part of it can be checked. Faria and colleagues propose a descriptive analysis before any method is chosen, and three of its checks can count against MCAR.

1. Missingness by arm and time point. Data are unlikely to be MCAR if the proportion missing differs by treatment allocation.
2. Missingness and baseline variables. A logistic regression of each missingness indicator on baseline covariates shows which predict missingness, and data are not MCAR if any does.
3. Missingness and earlier observed outcomes. An association with previously observed costs or quality of life rules out CD-MCAR and points towards MAR.

Little's test, proposed in 1988, combines such comparisons for continuous variables. As Li describes it, the test asks whether the means of the observed variables differ between groups of participants who share a missing value pattern.

$$d^2 = \sum_{j=1}^{J} n_j \left(\bar{y}_{oj} - \hat{\mu}_{oj}\right)^{\top} \tilde{\Sigma}_{oj}^{-1} \left(\bar{y}_{oj} - \hat{\mu}_{oj}\right)$$

Here $J$ is the number of missing value patterns, $n_j$ is the number of participants with pattern $j$, $\bar{y}_{oj}$ is the vector of sample means of the variables observed in pattern $j$, $\hat{\mu}_{oj}$ is the matching part of the maximum likelihood estimate of the mean vector under MCAR, usually by the EM algorithm, and $\tilde{\Sigma}_{oj}$ is the matching block of the estimated covariance matrix, scaled by $n/(n-1)$, where $n$ is the total number of participants. Under MCAR, $d^2$ approximately follows a chi-square distribution whose degrees of freedom are the number of observed variables summed over the patterns minus $p$, the number of variables.

Li notes that the test relies on multivariate normality or large samples and does not suit categorical variables. Rejecting the test's null is sufficient to reject MCAR, but MCAR can fail while the pattern means agree. Van Buuren observes that such tests are not widely used and their practical value is unclear, and that no test can separate MAR from MNAR, since the information needed is missing.

Worked example with lost cost questionnaires

All numbers in this example are illustrative. A trial randomises 200 patients, 100 to each arm, and records baseline EQ-5D index for everyone. The 12-month resource-use questionnaire, from which cost is calculated, is missing for 40 patients. The test has $p = 2$ variables and $J = 2$ patterns, 160 patients with both values and 40 with baseline utility only.

1. Missingness by arm. Twenty patients lack cost data in each arm. Every cell of the two by two table matches its expected count, so the chi-square statistic is 0 and gives no evidence against MCAR.

2. Baseline utility by pattern. The 160 complete cases have mean baseline utility 0.72 and the 40 incomplete cases 0.64. The overall mean is (160 × 0.72 + 40 × 0.64) / 200 = 0.704. The maximum likelihood variance of baseline utility over all 200 patients is 0.04, so the scaled variance is 200 / 199 × 0.04 = 0.0402.

3. Little's statistic. With baseline utility always observed and cost missing in one block, the cost part of each term cancels, because the maximum likelihood mean cost is adjusted through the regression of cost on baseline utility. Only the baseline means remain. The complete cases sit 0.016 above the overall mean, and 0.016 × 0.016 = 0.000256, so their term is 160 × 0.000256 / 0.0402 = 1.019. The incomplete cases sit 0.064 below it, and 0.064 × 0.064 = 0.004096, so their term is 40 × 0.004096 / 0.0402 = 4.076. With the unrounded variance 0.040201, the two terms add to 5.094. The degrees of freedom are the 2 + 1 = 3 observed variables across patterns less the 2 variables, which leaves 1.

4. Decision on MCAR. The 5% critical value of the chi-square distribution with 1 degree of freedom is 3.841, and the p-value for 5.094 is 0.024. The test rejects MCAR at the 5% level, although missingness was balanced across arms in step 1, because patients in poorer health at baseline returned fewer questionnaires.

5. Effect on the cost estimate. Suppose the complete cases have mean 12-month cost GBP 1,500, and cost falls by GBP 2,000 per unit of baseline utility. Under MAR given baseline utility, the mean cost estimate rises by 2,000 × 0.016 = 32, giving 1,500 + 32 = 1,532, or GBP 1,532. The complete-case mean understates it by GBP 32 per patient, and any difference in this shift between arms would carry into the incremental cost.

6. What the test cannot see. Had the incomplete cases averaged 0.71, the statistic would be 32 × 0.0001 / 0.0402 = 0.080, with p = 0.78, where 160 × 40 / 200 = 32 and 0.0001 is the squared gap. That would not establish MCAR. Patients with high costs who skipped the questionnaire for reasons unrelated to baseline utility would be invisible to any comparison of observed variables, which is the case for an MNAR sensitivity analysis.

MCAR beside the other mechanisms

Each weaker mechanism admits more causes of missingness. The table follows Faria and colleagues and van Buuren.

The EMA guideline states that it is not possible to know whether the MAR, never mind MCAR, assumptions are appropriate in any practical situation. A pattern-mixture model or selection model can explore departures from whichever assumption is primary.

Misreadings of MCAR

Most errors treat a convenient default as if the data had confirmed it. Three recur in trial-based evaluations.

1. A non-significant test proves MCAR. Li notes that rejection is sufficient to reject MCAR but not necessary, and step 6 shows a small baseline gap passing unnoticed.
2. Balanced missingness across arms implies MCAR. Faria and colleagues treat unequal proportions as evidence against MCAR, but steps 1 to 4 show equal proportions hiding missingness linked to health.
3. MCAR is required for any valid complete-case result. Hughes and colleagues show complete-case regression can be unbiased under some MAR and MNAR mechanisms, while arm means still need MCAR.

Sources

• European Medicines Agency, Committee for Medicinal Products for Human Use. Guideline on Missing Data in Confirmatory Clinical Trials. EMA/CPMP/EWP/1776/99 Rev. 1. 2 July 2010. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-missing-data-confirmatory-clinical-trials_en.pdf
• Faria R, Gomes M, Epstein D, White IR. A guide to handling missing data in cost-effectiveness analysis conducted within randomised controlled trials. PharmacoEconomics. 2014;32(12):1157-1170. https://doi.org/10.1007/s40273-014-0193-3
• Gabrio A, Mason AJ, Baio G. Handling missing data in within-trial cost-effectiveness analysis: a review with future recommendations. PharmacoEconomics Open. 2017;1(2):79-97. https://doi.org/10.1007/s41669-017-0015-6
• Hughes RA, Heron J, Sterne JAC, Tilling K. Accounting for missing data in statistical analyses: multiple imputation is not always the answer. International Journal of Epidemiology. 2019;48(4):1294-1304. https://doi.org/10.1093/ije/dyz032
• Leurent B, Gomes M, Carpenter JR. Missing data in trial-based cost-effectiveness analysis: an incomplete journey. Health Economics. 2018;27(6):1024-1040. https://doi.org/10.1002/hec.3654
• Li C. Little's test of missing completely at random. Stata Journal. 2013;13(4):795-809. https://ideas.repec.org/a/tsj/stataj/v13y2013i4p795-809.html
• Little RJ, D'Agostino R, Cohen ML, et al. The prevention and treatment of missing data in clinical trials. New England Journal of Medicine. 2012;367(14):1355-1360. https://doi.org/10.1056/NEJMsr1203730
• Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581-592. https://doi.org/10.1093/biomet/63.3.581
• Van Buuren S. Flexible Imputation of Missing Data. 2nd ed. CRC Press; 2018. Sections 1.2, 1.3, 2.2.3, 2.2.4 and 2.7. https://stefvanbuuren.name/fimd/