Information Bias

Information Bias in Health Research and Economic Evaluation

Information bias is systematic error that enters a study through the data recorded about the people already in it, when their treatments, outcomes, covariates or costs are measured, remembered, coded or processed wrongly. The ROBINS-I risk-of-bias tool treats it as one category beside confounding, selection bias and reporting bias. In health economics it matters because trial records, questionnaires and routine data supply the resource use, utility values and event rates in a cost-effectiveness model, so an error that differs between arms moves incremental costs and QALYs in a direction that is hard to predict. Its mechanisms, such as misclassification, measurement error, detection bias and interviewer bias, are the ways in which it arises, while patient recall of resource use is one of the most direct routes by which the bias reaches a cost-effectiveness estimate.

Where information bias sits among the families of bias

In the original ROBINS-I tool for non-randomised studies, as described in chapter 25 of the Cochrane Handbook, two of the seven domains are labelled information bias, namely bias in classification of interventions and bias in measurement of the outcome. The same table files bias due to missing data under selection bias.

Other sources draw the boundary more widely. The Catalogue of Bias entry by Bankhead, Spencer and Nunan includes the handling of missing data, and the NICE real-world evidence framework (ECD9) groups missing data, measurement error and misclassification under its heading on addressing information bias. A report should therefore say which convention it follows for missing values.

Differential and non-differential errors

The direction of an information bias depends less on the type of error than on whether the error is related to the comparison being made. The Cochrane Handbook defines differential measurement errors in a trial as errors related to intervention assignment and non-differential errors as unrelated to it. For an exposure or intervention, the ROBINS-I guidance calls misclassification differential when it is related to the outcome or the risk of the outcome.

$$\hat{\Delta} = (\mu_1 + e_1) - (\mu_0 + e_0) = \Delta + (e_1 - e_0)$$

where $\hat{\Delta}$ is the observed difference in a mean outcome or cost between the intervention arm (1) and the comparator arm (0), $\mu_1$ and $\mu_0$ are the true arm means, $e_1$ and $e_0$ are the mean recording errors in each arm, and $\Delta$ is the true difference $\mu_1 - \mu_0$. An error of the same size in both arms cancels from the difference, an error proportional to the true value scales the difference by the same factor, so proportional under-recording shrinks it, and an error that differs between arms adds a term of either sign.

The ROBINS-I guidance states that non-differential misclassification of intervention status will usually bias the effect towards the null. Jurek and colleagues showed that non-differentiality alone does not guarantee this, and that even when the bias is towards the null a single study's estimate may by chance be an overestimate. ECD9 adds that random measurement error in continuous outcomes reduces precision but leaves comparative effects unbiased, and that differential measurement error generally biases comparative effects in a direction that can be hard to predict. ROBINS-I asks reviewers to judge whether such bias favours the experimental intervention or the comparator, not whether it lies towards the null.

Recall and self-report of resource use

Recall bias, in the Catalogue of Bias definition by Spencer, Brassey and Mahtani, is systematic error due to differences in the accuracy or completeness of recall of past events or experiences. It is the classic differential error of the case-control study, where, as the Cochrane Handbook notes, cases may be more likely than controls to recall events or report exposures they believe caused their disease.

Costing studies depend on the same memory. Ridyard and Hughes reviewed the 100 primary research papers funded by the UK Health Technology Assessment programme to June 2009. Of 95 with an economic analysis, 85 recorded patient-level resource use, with extensive use of medical records and reliance on patient recall. In a systematic review of 42 studies, Bhandari and Wagner found self-reported utilisation to be of variable accuracy, affected by factors including the recall time frame, the type and frequency of use, questionnaire design and memory aids. Evans and Crawford reported that recall of resource use declines over time and that medication use is recalled less accurately than hospitalisation.

For a trial-based cost-effectiveness analysis, the larger danger is recall that differs between arms. An intervention that adds a patient diary or regular nurse contact can change how completely its patients remember other care, so the incremental cost absorbs a reporting difference unrelated to any effect of treatment on resource use.

Outcomes, utility values and routine data

Bias in measurement of the outcome is the RoB 2 domain closest to information bias in randomised trials, and the Cochrane Handbook says it relates primarily to differential errors. In its example, a drug that causes headaches leads to more MRI scans and so to more diagnoses of symptomless brain tumours, an apparent effect the drug does not have.

For participant-reported outcomes such as pain and health-related quality of life, the participant is the assessor, and awareness of the intervention received leads to a judgement of at least some concerns. This bears on EQ-5D responses from open-label trials, which become the health state utility values behind a QALY gain. The Handbook notes that blinding of outcome assessors is particularly difficult for patient-reported outcomes.

Routine data carry the same problems in other forms. ECD9 states that missing data, measurement error, misclassification and incorrect reporting of dates can apply to all study variables, including eligibility criteria, outcomes, interventions and covariates, and can create information biases in real-world data studies. It notes that imperfect linkage could exclude patient records or lead to misclassification, a problem treated under record linkage.

Worked example with differential recall of resource use (illustrative)

All numbers in this example are illustrative. A trial compares a self-management programme with usual care, and other NHS care over six months is costed from a patient questionnaire. Patients in the programme keep a diary, so they report more of their GP and hospital contacts. The programme cost of GBP 800 per patient comes from trial records, and the QALY gain of 0.025 is taken as measured without error.

$$\Delta C_{obs} = c_P + r_1 R_1 - r_0 R_0, \qquad ICER_{obs} = \frac{\Delta C_{obs}}{\Delta E}$$

where $\Delta C_{obs}$ is the observed incremental cost per patient in GBP, $c_P$ is the programme cost (GBP 800), $R_1$ and $R_0$ are the true mean costs of other NHS care per patient in the programme and usual care arms (GBP 900 and GBP 1,200), $r_1$ and $r_0$ are the shares of that care reported in each arm, $\Delta E$ is the QALY gain per patient (0.025) and $ICER_{obs}$ is the observed ICER.

Net monetary benefit is 25,000 × 0.025 = GBP 625 minus the incremental cost. Non-differential under-reporting scales the true offset of GBP 300 by 0.75 to GBP 225 and raises the ICER by 15%, leaving the decision at an illustrative threshold of GBP 25,000 per QALY unchanged. Differential recall leaves only GBP 45 of the offset (900 minus 855), so the ICER rises above the threshold and the programme is wrongly rejected. Less complete reporting in the programme arm would instead have flattered it.

A validation subsample comparing questionnaires with GP and hospital records would estimate the reporting shares. Dividing by them recovers the truth, since 900 divided by 0.75 is 1,200 and 855 divided by 0.95 is 900, restoring the ICER of GBP 20,000 per QALY. When the shares are uncertain, they can be given distributions and sampled in probabilistic sensitivity analysis, which widens the uncertainty around the corrected ICER as well as shifting it.

Prevention, validation and quantitative bias analysis

Prevention comes first, since the Catalogue of Bias entry observes that little can be done to address incorrect data. Its preventive steps include blinding to intervention status while outcomes are measured, prospective collection with standardised methods, neutral questions and records that corroborate self-report.

When error remains, ECD9 says that information on its likely structure and magnitude, for example from an internal or external validation study, can be incorporated through calibration or other advanced methods. For outcome misclassification its table of sensitivity analyses suggests adjusting for known performance metrics, such as sensitivity, specificity and positive predictive value, or using quantitative bias analysis. ECD9 adds that such analyses can be propagated through to cost-effectiveness analyses.

Quantitative bias analysis, as ECD9 describes it, either examines how much bias would have to be present to change results or a decision threshold, or estimates the direction, magnitude and uncertainty of the bias. Lash and colleagues add that bias analysis counters overconfidence in research results and can guide where scarce research resources go.

Boundaries with neighbouring concepts

Selection bias concerns who enters and remains in the analysis, while information bias concerns what is recorded about them. Performance bias arises from differences in the care the arms receive, which ROBINS-I files under confounding, although blinding helps to prevent both. Measurement error is a mechanism and information bias its effect on an estimate, so random error in a continuous outcome can cost precision without bias. A larger sample leaves information bias in place, as explained under bias.

Common misreadings of information bias

Most mistakes assume a direction or source of error without tracing how the data were produced. Three recur in economic evaluation.

1. Non-differential error is harmless. Jurek and colleagues showed that it does not guarantee bias towards the null, and proportional under-reporting still shrinks a cost difference.
2. Routine records are free of it. ECD9 lists measurement error, misclassification and wrong dates among the limitations of routine data for every type of variable.
3. Blinded assessment settles it. For participant-reported outcomes the participant is the assessor, and blinding does nothing about coding errors that are the same in both arms.

Sources

• Bankhead CR, Spencer EA, Nunan D. Information bias. In: Catalogue of Bias. Catalogue of Bias Collaboration; 2019. https://catalogofbias.org/biases/information-bias/
• Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Medical Care Research and Review. 2006;63(2):217-235. https://doi.org/10.1177/1077558705285298
• Evans C, Crawford B. Patient self-reports in pharmacoeconomic studies. Their use and impact on study validity. PharmacoEconomics. 1999;15(3):241-256. https://doi.org/10.2165/00019053-199915030-00004
• Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial (last updated October 2019). In: Higgins JPT, Thomas J, Chandler J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08
• Jurek AM, Greenland S, Maldonado G, Church TR. Proper interpretation of non-differential misclassification effects: expectations vs observations. International Journal of Epidemiology. 2005;34(3):680-687. https://doi.org/10.1093/ije/dyi060
• Lash TL, Fox MP, MacLehose RF, Maldonado G, McCandless LC, Greenland S. Good practices for quantitative bias analysis. International Journal of Epidemiology. 2014;43(6):1969-1985. https://doi.org/10.1093/ije/dyu149
• National Institute for Health and Care Excellence. NICE real-world evidence framework (ECD9). Published 23 June 2022, minor changes September 2025. Sections on assessing data suitability and methods for real-world studies of comparative effects. https://www.nice.org.uk/corporate/ecd9
• Ridyard CH, Hughes DA. Methods for the collection of resource use data within clinical trials: a systematic review of studies funded by the UK Health Technology Assessment program. Value in Health. 2010;13(8):867-872. https://doi.org/10.1111/j.1524-4733.2010.00788.x
• Spencer EA, Brassey J, Mahtani K. Recall bias. In: Catalogue of Bias. Catalogue of Bias Collaboration; 2017. https://catalogofbias.org/biases/recall-bias/
• Sterne JAC, Hernán MA, McAleenan A, Reeves BC, Higgins JPT. Chapter 25: Assessing risk of bias in a non-randomized study (last updated October 2019). In: Higgins JPT, Thomas J, Chandler J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-25