Detection Bias

Detection Bias in Trials, Screening and Economic Models

Detection bias arises when the groups in a study are not observed in the same way, so that an outcome is found more often, or judged more favourably, in one group for reasons unrelated to the treatment. Its two usual sources are outcome assessors who know which treatment each person received, and differences in how intensively or by what method each group is examined. It matters in health economics because biased relative effects pass straight into cost-effectiveness models, where misclassifying a few patients can move an ICER across a threshold. The current Cochrane tool for randomised trials, RoB 2, assesses it as bias in measurement of the outcome, and the names observer bias and ascertainment bias are also used for parts of the same mechanism.

Two routes from unequal observation to a biased effect

The Cochrane Handbook calls errors in measuring outcomes measurement error for continuous outcomes, misclassification for categorical ones and under-ascertainment or over-ascertainment for events. These errors are differential when they are related to treatment assignment and non-differential when they are not. Detection bias concerns mainly the differential kind, because errors that differ between arms can create or exaggerate a difference, whereas errors shared equally by both arms usually dilute it.

The first route runs through judgement. When an assessor knows the allocation, expectations about the new treatment can tip borderline cases, such as whether an X-ray shows healing or whether a clinical event meets its definition. The second route runs through intensity of looking. If one arm is scanned or tested more often, more events are found there, including some that would never have caused symptoms. The Handbook chapter on non-randomised studies notes that bias in measurement of the outcome is often referred to as detection bias and names both routes, assessors aware of intervention status and different methods or intensities of observation in different groups.

Detection bias is a form of information bias, the family of biases that come from how data are measured or recorded. It differs from random measurement error, which adds noise to both arms alike and tends to blur rather than manufacture a difference. The 2011 version of the first Cochrane risk-of-bias tool grouped its items under six types of bias, and detection bias was the type addressed by blinding of outcome assessment.

How RoB 2 assesses bias in measurement of the outcome

RoB 2 assesses each trial result across five fixed domains, and bias in measurement of the outcome is the fourth. It asks whether the method of measuring the outcome was inappropriate, whether measurement or ascertainment could have differed between intervention groups, whether outcome assessors knew the intervention received and, if so, whether that knowledge was likely to have influenced the assessment. An algorithm maps the answers to low risk, some concerns or high risk, and reviewers may record the likely direction of bias.

The Handbook ties the judgement to who assesses the outcome and how much judgement it involves.

For non-randomised studies, ROBINS-I contains a domain of the same name with parallel signalling questions. NICE's manual for technology evaluations (PMG36) states that non-randomised studies may suffer informational biases from differential data collection and asks for each study's risk of bias to be assessed with a validated tool such as ROBINS-I.

Blinding of outcome assessors and what it can fix

Blinding the people who assess outcomes is the main safeguard against the judgement route. The Handbook notes that assessor blinding is often possible even when participants and clinicians cannot be blinded, although it is particularly difficult when the participant is the assessor, as with pain after surgery. A trial can therefore be open label yet have blinded adjudication of outcomes.

Hróbjartsson and colleagues reviewed 21 randomised controlled trials that assessed the same binary outcome with both blinded and non-blinded assessors. The pooled ratio of odds ratios was 0.64, an average exaggeration of the non-blinded odds ratio by 36%. Blinded and non-blinded assessors agreed in a median of 78% of assessments, and the exaggeration came from misclassifying a median of 3% of patients per trial. For time-to-event outcomes the same group found, in seven trials comparing experimental treatments with standard care, an average exaggeration of hazard ratios of about 27%, but bias in favour of the control arm in four trials comparing oral with intravenous administration of the same drug.

Assessor blinding does not cure the second route. If one arm is examined more often, blinded readers still find more events there, because the extra events are real findings produced by extra testing.

More testing finds more disease

The Handbook gives an example. A new drug causes more severe headaches than placebo, which leads to more MRI scans in the experimental arm and so to more diagnoses of symptomless brain tumours, although the drug does not increase their incidence. An intervention involving extra visits to a healthcare provider likewise creates extra opportunities for outcome events to be identified, and adverse events collected passively are especially exposed.

The same mechanism, often called surveillance bias, distorts population data and cancer screening evaluations. Tancredi, Cullati and Chiolero describe scrutiny-dependent cancers, such as thyroid and prostate cancer and melanoma, whose recorded incidence rises with the intensity of testing. In the United States, prostate cancer incidence rose sharply in the early 1990s after PSA-based screening was introduced and fell after the 2012 USPSTF recommendation against routine screening, without matching changes in mortality. Groups examined more often can also appear to be at higher risk, and the authors favour indicators less open to this bias, such as mortality or the incidence of advanced cancer.

Quality indicators show the economic side. Chung and colleagues showed, with two hypothetical hospitals, that when few patients are imaged a better and a worse hospital can record the same rate of postoperative venous thromboembolism, because the recorded rate depends on how many patients are scanned. Because incentives were tied to the indicator, payers could reward hospitals whose low rates reflect too little imaging rather than good care.

Consequences for effect estimates in economic models

A cost-effectiveness analysis usually takes its relative effect from trials or a meta-analysis, so detection bias there carries through to costs, QALYs and the ICER. PMG36 lists blinding among the features by which a trial's internal validity is assessed and names the selection and measurement of outcomes as a further important consideration. It asks that, for each source of model inputs, details of how bias was assessed and addressed are presented, and that bias adjustments be considered in a network meta-analysis when included studies raise concerns about methodological quality.

The direction of the error depends on which outcome is affected. Exaggerated benefit on a subjective efficacy outcome usually favours the new technology and lowers its ICER. More intensive ascertainment of harms in the experimental arm, as in the MRI example, adds spurious adverse event costs and disutilities and works against it. Resource use can be biased too, because events that are found trigger treatment that is then costed.

Analysts can prefer trials with blinded adjudication, use objective outcomes where they capture the decision, and run a scenario analysis restricted to studies at low risk of bias in this domain.

Worked example with blinded and unblinded assessment of one trial

The numbers in this example are illustrative. A trial randomises 500 patients to a new treatment and 500 to standard care, and the outcome is treatment failure judged from imaging. A panel blinded to allocation records 100 failures with standard care (20%) and 80 with the new treatment (16%), a relative risk of 0.80 and an absolute risk reduction of 0.04. Unblinded local assessors agree with the panel on standard care but read 15 of the 80 failures with the new treatment (3% of the arm) as successes, giving 65 failures (13%), a relative risk of 0.65 and an absolute risk reduction of 0.07.

A simple model per patient treated uses these quantities.

$$\Delta C = c_{T} - ARR \times c_{F}, \qquad \Delta E = ARR \times q_{F}, \qquad ICER = \frac{\Delta C}{\Delta E}$$

where $\Delta C$ is the incremental cost in GBP, $\Delta E$ is the incremental QALYs, $c_T$ is the extra cost of the new treatment (GBP 600), $ARR$ is the absolute risk reduction in failures, $c_F$ is the cost of managing a failure (GBP 6,000) and $q_F$ is the QALY loss from a failure (0.25).

At an illustrative cost-effectiveness threshold of GBP 20,000 per QALY, the net health benefit per patient is 0.01 minus 360 divided by 20,000, or minus 0.008 QALYs, with the blinded result and 0.0175 minus 180 divided by 20,000, or 0.0085, without blinding. Reclassifying 15 of 1,000 patients turns a technology that displaces more health than it adds into one that appears good value.

Boundaries with neighbouring biases

Performance bias, from differences in the care received, falls in the RoB 2 domain on deviations from intended interventions, so the Handbook keeps assessor blinding separate from blinding of participants and carers. Differential loss of outcome data is attrition, assessed in the missing outcome data domain. Confounding and selection of participants concern who ends up in each group, not how they are observed.

In screening, lead time and length time distort survival comparisons through when and which cancers are found, whereas surveillance bias is defined by unequal intensity of detection between the groups or periods compared. Recall bias is misclassification of past exposure in observational studies and sits on the exposure side of information bias.

Misreadings of assessor blinding and outcome type

An open-label design does not mean high risk in this domain. For all-cause mortality or an automated laboratory value, knowledge of allocation is unlikely to change the result, which is why RoB 2 judges each result and outcome rather than the trial as a whole.

High agreement between assessors is not reassurance. In the review of binary outcomes, a median of 3% of patients misclassified produced the average exaggeration, and the worked example shows an error half that size, 15 of 1,000 patients, reversing a reimbursement decision.

Detection bias does not always favour the experimental treatment, as the oral against intravenous trials showed, so its direction is judged case by case. Objective outcomes are not immune to the intensity route, since incidental tumours and silent clots are found more often where people are scanned more often.

Sources

• Boutron I, Page MJ, Higgins JPT, Altman DG, Lundh A, Hróbjartsson A. Chapter 7: Considering bias and conflicts of interest among the included studies (last updated August 2022). In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-07
• Chung JW, Ju MH, Kinnier CV, Sohn MW, Bilimoria KY. Postoperative venous thromboembolism outcomes measure: analytic exploration of potential misclassification of hospital quality due to surveillance bias. Annals of Surgery. 2015;261(3):443-444. https://doi.org/10.1097/SLA.0000000000000850
• Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. https://doi.org/10.1136/bmj.d5928
• Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial (last updated October 2019). In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08
• Hróbjartsson A, Thomsen AS, Emanuelsson F, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012;344:e1119. https://doi.org/10.1136/bmj.e1119
• Hróbjartsson A, Thomsen AS, Emanuelsson F, et al. Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. International Journal of Epidemiology. 2014;43(3):937-948. https://doi.org/10.1093/ije/dyt270
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36
• Sterne JAC, Hernán MA, McAleenan A, Reeves BC, Higgins JPT. Chapter 25: Assessing risk of bias in a non-randomized study (last updated October 2019). In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-25
• Tancredi S, Cullati S, Chiolero A. Screening and surveillance bias in cancer. Epidemiologia. 2023;4(2):117-120. https://doi.org/10.3390/epidemiologia4020012