Misclassification

Misclassification in Real-World Evidence and Economic Models

Misclassification is measurement error in a categorical variable, so that a person is counted as exposed when unexposed, as a case when not a case, or in the wrong level of a covariate or health state. It matters in health economics because claims and hospital records now supply many of the event rates, relative effects and resource use that populate cost-effectiveness models, and coding errors in those sources carry through to the ICER. Whether the resulting bias has a predictable direction depends mainly on whether the errors differ between the groups compared. Its size is described with diagnostic accuracy measures, so it can be quantified and corrected.

Misclassification as measurement error in categorical data

The NICE real-world evidence framework (ECD9) defines measurement error as the extent to which measured study variables deviate from the truth and calls it misclassification for categorical variables. Misclassification is therefore the categorical form of measurement error, and the bias it produces belongs to information bias, which the Cochrane Handbook describes as bias from misclassified intervention status or misclassified or mismeasured outcomes.

Against a reference standard, sensitivity is the proportion of people truly in a category who are recorded there, specificity the proportion truly outside it who are recorded outside it, and the positive predictive value and negative predictive value are the shares of recorded positives and negatives that are correct. ECD9 lists these four as the accuracy metrics for categorical variables, and adds that an event that occurred but was never recorded may be best treated as a false negative rather than as missing data.

Differential and non-differential errors

Van Smeden, Lash and Groenwold define exposure misclassification as non-differential when its error is independent of the true outcome, and outcome misclassification as non-differential when its error is independent of the true exposure. Anything else is differential. For a binary exposure, non-differential error means that sensitivity and specificity are the same whatever the outcome.

The classic differential case is recall bias in a case-control study, in which cases may recall or report past exposures differently from controls. Measuring exposure before the outcome does not guarantee non-differential error, since people with a strong family history of a disease may also recall that history more accurately. Outcomes can be misclassified differentially too. Funk and Landi note that people taking a drug may be screened and tested more often, so fewer of their true cases go unrecorded, a mechanism shared with detection bias. NICE's manual for technology evaluations (PMG36) warns that non-randomised studies may suffer informational biases from limitations of the data or differential data collection.

Usual directions of bias and their exceptions

Copeland and colleagues showed in 1977 that the departure of a relative risk or odds ratio from its true value depends on sensitivity, specificity, disease frequency and exposure frequency. Equal misclassification in the compared groups biased the estimate towards the null, and sometimes beyond it, whereas differential misclassification could bias it in either direction and by a large amount. Later work added exceptions.

These rules describe averages, and in a simulation cited by Funk and Landi single estimates were biased both towards and away from the null. Imperfect specificity of an outcome biases the relative risk even when sensitivity is perfect, which is why outcome definitions in coded data are usually built for high specificity, a practice ECD9 describes as often recommended.

Comparative effectiveness research adds a trap. In a hypothetical example from Funk and Landi, two drugs each double the risk of an outcome relative to non-users. Non-differential misclassification of each drug against non-use biases each comparison towards the null by a different amount, so the head-to-head relative risk appears as 1.2 when the true value is 1.0. With an active comparator, non-differential error cannot be relied on to be conservative. Large datasets do not help, because more observations bring an estimate closer to its biased expected value rather than to the truth.

Coding errors in claims and administrative data

Claims data and other administrative data record what was billed rather than what happened. Funk and Landi list how pharmacy claims misclassify exposure. Users appear as non-users when they pay out of pocket, receive free samples, are treated in hospital or buy medicines over the counter, and non-users appear as users when prescriptions are filled but never taken. In a Canadian example they cite, a policy change that automated prior authorisation for thiazolidinediones suggested that perhaps 20% of earlier use had been recorded as non-use.

For diagnoses, the absence of a code is taken to mean the absence of the condition, so patients with few healthcare contacts have misclassified confounders. Outcome algorithms therefore combine codes, prescriptions and procedures to maximise specificity. Chubak, Pocobelli and Weiss argue that specificity matters most when classifying outcomes and positive predictive value when an algorithm defines a cohort, and that at very low prevalence even high specificity leaves many false positives. ECD9 says the validity of an existing code list should be reviewed before use.

The same logic reaches costs. A cost-of-illness study that identifies cases from codes with a low positive predictive value counts non-cases as cases and so mixes their costs into the average cost per case. Providers can be misclassified as well. Adams and colleagues estimated from Massachusetts claims that 22% of physicians would be misclassified in a two-tier insurance product labelling the lowest-cost quartile as lower cost, because many physicians' cost profiles had low reliability.

Misclassified health states in Markov models

The manual of the msm package for R uses heart transplant recipients screened about yearly by angiography for coronary allograft vasculopathy. Progression is thought irreversible, yet the recorded states show apparent recoveries because the angiogram is subject to error. A hidden Markov model treats the true state as unobserved and links it to the recorded state through a misclassification matrix, so that progression and the observation process are estimated separately.

In a Markov model each state carries its own cost and utility, so people placed in the wrong state accrue the wrong costs and QALYs, and a transition probability estimated from misclassified states distorts time spent in each state. ECD9 notes that real-world data are widely used to populate economic models, including resource use, quality of life, event rates, prevalence and incidence.

Validation studies, correction and bias analysis

Misclassification can be corrected when its size is known from an internal validation subsample or an external validation study. For a binary variable, the observed proportion relates to the true one as follows.

$$p^{*} = S_e \, p + (1 - S_p)(1 - p)$$

where $p^{*}$ is the observed proportion recorded as positive, $p$ is the true proportion, $S_e$ is sensitivity and $S_p$ is specificity. Rearranging gives the corrected proportion.

$$p = \frac{p^{*} - (1 - S_p)}{S_e + S_p - 1}$$

where the symbols are as above and the denominator must be positive. With the same accuracy in both groups, the observed risk difference equals the true one multiplied by $S_e + S_p - 1$. Copeland and colleagues derived formulas of this kind to recover the true relative risk or odds ratio.

Simple bias analysis reallocates the cells of an observed table using point values of sensitivity and specificity, but ignores uncertainty in them. Probabilistic bias analysis repeats the correction with values drawn from distributions in a Monte Carlo process, and multiple imputation for measurement error imputes the true value from a model fitted in a validation sample. ECD9 suggests varying exposure definitions, such as days' supply and grace periods, for exposure misclassification, and adjusting for known performance metrics or running a quantitative bias analysis for outcome misclassification. It states that such analyses can be propagated through to cost-effectiveness analyses.

Worked example with a claims-based hospitalisation outcome

The numbers in this example are illustrative. A claims study compares a new drug with standard care, with hospitalisation for heart failure within one year as the outcome. True risks are 10% with standard care and 6% with the new drug. The outcome algorithm has sensitivity 0.80 in both groups. With specificity 0.98 the recorded risks are 0.80 × 0.10 + 0.02 × 0.90 = 0.098 and 0.80 × 0.06 + 0.02 × 0.94 = 0.0668. With perfect specificity they are 0.080 and 0.048. The model takes absolute risks directly from the study.

$$\Delta C = c_D - ARR \times c_H, \qquad \Delta E = ARR \times q_H, \qquad ICER = \frac{\Delta C}{\Delta E}$$

where $\Delta C$ is the incremental cost in GBP, $\Delta E$ is the incremental QALYs, $c_D$ is the extra drug cost (GBP 300), $ARR$ is the absolute risk reduction, $c_H$ is the cost of a hospitalisation (GBP 5,000) and $q_H$ is the QALY loss per hospitalisation (0.10).

At an illustrative cost-effectiveness threshold of GBP 30,000 per QALY, the true ICER indicates good value and both misclassified versions reject the drug. The reduction of 0.0312 is 0.78 × 0.04, the true difference scaled by sensitivity plus specificity minus one. Perfect specificity keeps the relative risk exactly right but still shrinks the reduction to 0.80 × 0.04 = 0.032, so the error persists whenever absolute risks come from the same coded data. The correction formula recovers the truth, since (0.098 minus 0.02) divided by 0.78 gives 0.10 and (0.0668 minus 0.02) divided by 0.78 gives 0.06.

Boundaries with neighbouring biases

Measurement error also covers continuous and count variables, whose accuracy ECD9 describes with measures such as mean absolute error. Detection bias, unequal finding or judging of outcomes between groups, is one route to differential outcome misclassification but also covers continuous outcomes. Recall bias is differential exposure misclassification from memory. Misclassified confounders leave residual confounding. Missing data are a separate problem.

Common misreadings of misclassification

A common misreading is that non-differential error is always conservative. Funk and Landi conclude that comparative effectiveness research cannot rely on it to bias estimates towards the null, and van Smeden and colleagues show that even univariate non-differential exposure error can bias away from the null. A high positive predictive value is often read as proof that an algorithm is fit for use, yet predictive values depend on prevalence, and a high positive predictive value is compatible with low sensitivity, which governs how many events and costs go uncounted.

Sources

• Adams JL, Mehrotra A, Thomas JW, McGlynn EA. Physician cost profiling: reliability and risk of misclassification. New England Journal of Medicine. 2010;362(11):1014-1021. https://doi.org/10.1056/NEJMsa0906323
• Chubak J, Pocobelli G, Weiss NS. Tradeoffs between accuracy measures for electronic health care data algorithms. Journal of Clinical Epidemiology. 2012;65(3):343-349. https://doi.org/10.1016/j.jclinepi.2011.09.002
• Copeland KT, Checkoway H, McMichael AJ, Holbrook RH. Bias due to misclassification in the estimation of relative risk. American Journal of Epidemiology. 1977;105(5):488-495. https://doi.org/10.1093/oxfordjournals.aje.a112408
• Funk MJ, Landi SN. Misclassification in administrative claims data: quantifying the impact on treatment effect estimates. Current Epidemiology Reports. 2014;1(4):175-185. https://doi.org/10.1007/s40471-014-0027-z
• Jackson CH. Multi-state modelling with R: the msm package, version 1.8.2. MRC Biostatistics Unit; 2024. https://cran.r-project.org/web/packages/msm/vignettes/msm-manual.pdf
• National Institute for Health and Care Excellence. NICE real-world evidence framework (ECD9). London: NICE; 2022, updated September 2025. https://www.nice.org.uk/corporate/ecd9
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36
• Sterne JAC, Hernán MA, McAleenan A, Reeves BC, Higgins JPT. Chapter 25: Assessing risk of bias in a non-randomized study. In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-25
• van Smeden M, Lash TL, Groenwold RHH. Reflection on modern methods: five myths about measurement error in epidemiological research. International Journal of Epidemiology. 2020;49(1):338-347. https://doi.org/10.1093/ije/dyz251