Missing Not at Random

Missing Not at Random: Concept Architecture

Orientation and learning roadmap

This page starts with the missing-data mechanism and its contrast with missing at random, then shows how an unobserved outcome can change a health-utility estimate. It follows with modelling and sensitivity strategies, trial reporting, and the implications for cost-effectiveness analysis.

The dependence that matters

Whether an outcome is missing may depend on the outcome value that would have been observed, even after accounting for the data that are available. For example, patients with the poorest unrecorded health-related quality of life may be less likely to return a questionnaire despite having similar recorded baseline characteristics to responders.

Let $R=1$ indicate that the outcome $Y$ is observed, let $X$ contain observed covariates and prior outcomes, and let $Y_{\mathrm{mis}}$ denote unobserved values. A missing-not-at-random (MNAR) mechanism allows the response probability to depend on those unobserved values even conditional on the observed information, schematically as

$$P(R=1\mid Y_{\mathrm{mis}},Y_{\mathrm{obs}},X) \text{ varies with }Y_{\mathrm{mis}}.$$

This dependence is a property of a specified data-generating process and conditioning set, not a label proved by a high percentage of missing observations. Adding informative observed predictors can sometimes make a missing-at-random (MAR) assumption more plausible, but it cannot prove that remaining dependence on unobserved values is absent.

Distinguishing the mechanisms

The distinction hinges on which variables still predict missingness after conditioning on observed information. It is especially important for longitudinal utility, costs, and patient-reported outcomes, because prior measurements may explain some but not all later nonresponse.

MAR and MNAR refer to the conditioning information available to the analysis, so an initial suspicion about reasons for dropout is not by itself a statistical test of either mechanism. Differential missingness by treatment group is important to inspect but does not alone establish MNAR. Equally, similar observed responders and nonresponders do not rule it out.

Worked example: unobserved utility

Suppose 80 of 100 patients provide a one-year health-utility measure and the observed mean is $0.70$. Assume for this teaching example that, conditional on the recorded data, the remaining 20 would average $0.50$ because their poorer unrecorded health contributed to nonresponse.

The illustrative full-sample mean would then be

$$\overline{U}_{\mathrm{all}}= \frac{80(0.70)+20(0.50)}{100}=0.66.$$

If the 20 missing values were instead assigned a mean of $0.70$, the full-sample mean would be $0.70$, a difference of $0.04$ utility units. Over a hypothetical full year at constant utility and survival for every patient, the corresponding difference is $0.04$ QALYs per patient; actual QALYs also require appropriate timing and survival information. The value $0.50$ is an explicit sensitivity assumption, not a value recoverable from these observed data alone.

Identification and sensitivity analysis

The distribution of unobserved values cannot generally be identified from observed outcomes alone without restrictions on the missing-data process or outcome model. An MNAR analysis therefore states its unverifiable assumptions, relates them to clinical knowledge, and explores how conclusions change across credible alternatives.

Selection models specify a joint outcome and response process, whereas pattern-mixture models stratify or parameterise outcomes by missingness pattern. A delta-adjusted pattern-mixture analysis might shift predicted missing utilities downward relative to an MAR-based imputation, with the shift and its interpretation specified in utility units. Other approaches can include reference-based assumptions, bounds, and tipping-point analysis, provided they align with the target estimand and clinical events.

Trial conduct and reporting

Preventing missing data is preferable to depending entirely on a later statistical adjustment. Trial teams should keep following participants after treatment discontinuation when the target outcome remains meaningful and collect reasons, timing, and predictors of missingness without treating those reasons as perfect proof of the mechanism.

Reports should present missingness by assigned group, visit, and outcome, and distinguish discontinuation of treatment from absence of outcome data. A primary analysis may rely on an MAR model, but the credibility of conclusions should be examined with prespecified or well-justified MNAR sensitivity analyses. Imputing every missing value with a single fixed “worst” value is not a generally valid substitute for a clinically grounded analysis.

Health economic consequences

Unobserved costs and utilities can affect both incremental costs and incremental QALYs, and their missingness mechanisms may differ. In an economic evaluation, assumptions should preserve relevant relationships among outcomes, resource use, treatment group, baseline severity, and time, then propagate uncertainty into the decision model.

The direction of bias is not determined merely by the phrase MNAR: patients with worse unobserved utility might have higher or lower unobserved costs depending on care pathways. Sensitivity results should show how incremental outcomes and the decision change under plausible mechanisms, including a threshold at which a conclusion reverses. A seemingly precise base case under one imputation model does not eliminate the uncertainty about values that were never measured.

Source basis and further learning

The linked primary guidance and methodological sources provide the basis for prevention, mechanism-based models, and sensitivity analysis. They also stress that estimands and follow-up rules should be specified before choosing a missing-data procedure.

• National Research Council (2010), The Prevention and Treatment of Missing Data in Clinical Trials. This panel report covers prevention, analysis, and sensitivity to MNAR assumptions.
• National Research Council (2010), “Principles and Methods of Sensitivity Analyses”. This chapter develops sensitivity analysis for departures from primary assumptions.
• International Council for Harmonisation (2019), ICH E9(R1), Addendum on Estimands and Sensitivity Analysis in Clinical Trials. This connects treatment-effect questions, missing data, and sensitivity analysis.
• Little (1995), “Modeling the Drop-Out Mechanism in Repeated-Measures Studies”, Journal of the American Statistical Association, 90. This describes selection and pattern-mixture formulations.