Cognitive Bias Concept

Cognitive Bias Concept, Heuristics and Biased Inputs to Health Economic Evaluation

In health economics, a cognitive bias is a predictable gap between a person's judgement and the answer that probability theory or a stated decision rule would give, so that errors run in one direction instead of cancelling out. It matters because diagnoses, treatment choices, elicited utilities and expert estimates of model inputs are human judgements, and a judgement that leans one way makes the cost-effectiveness result lean the same way. In an illustrative screening example below, base-rate neglect reverses a resource decision. Statistical bias in study estimates has its own page, Bias.

Heuristics and the systematic errors they produce

Tversky and Kahneman, in their 1974 paper on judgement under uncertainty, argued that people assess probabilities with a small number of heuristics that turn hard statistical tasks into simpler operations. In their words the heuristics are quite useful, but sometimes lead to severe and systematic errors. The three they described were representativeness, availability, and anchoring with adjustment.

Kahneman's 2002 Nobel lecture placed this work within a two-system view of thinking, with an intuitive mode that is fast and automatic, and a controlled mode that is slower and deliberate. Saposnik and colleagues, reviewing cognitive bias in physicians, report the suggestion that most such biases stem from overuse of the intuitive system or from it overriding deliberate reasoning. Tversky and Kahneman noted that experienced researchers are prone to the same biases when they think intuitively.

Representativeness, base rates and sample size

Under the representativeness heuristic, the probability that a case belongs to a category is judged by how closely it resembles the category's stereotype. Resemblance is unaffected by the prior probability, or base rate, which should weigh heavily in the judgement. In the 1974 experiments, subjects told that a group held 70 engineers and 30 lawyers (or the reverse) used those proportions correctly when given no description. Given a description designed to carry no information, they judged the chance that the person was an engineer to be 0.5 whichever base rate they had been told.

The clinical equivalent is reading a positive test as if it carried the test's sensitivity, whatever the prevalence. Sensitivity is the chance of a positive result given disease, while the positive predictive value is the chance of disease given a positive result, which depends heavily on prevalence.

Representativeness also produces insensitivity to sample size. Tversky and Kahneman asked students which of two hospitals, one with about 45 births a day and one with about 15, would record more days on which over 60% of babies were boys. Of the 95 who answered, 53 said about the same, 21 the larger hospital and 21 the smaller one, although sampling theory implies many more such days in the smaller hospital. The same intuition can lead readers of evidence to trust a striking result from a small study, or from a small subgroup, more than its precision allows.

Availability, anchoring and overconfidence

Availability judges frequency or probability by the ease with which instances come to mind. It is a useful cue, but salience and recency also make events easy to recall. The Bojke report on structured expert elicitation for health-care decision-making gives the clinical form, in which a clinician may focus on particularly memorable treatment successes or failures when judging a probability, and neglect cases that come to mind less readily.

Anchoring is estimation by adjustment from a starting value, with adjustment that is typically insufficient, so that estimates are biased towards the starting value. Its effect on willingness to pay and time trade-off answers is covered on the anchoring bias page, and its effect on expert distributions on anchoring bias in elicitation.

Overconfidence matters directly for decision models because it narrows the uncertainty they carry. Tversky and Kahneman reported that in most calibration studies the true values fell outside judges' 1st to 99th percentile range for about 30% of quantities, against the 2% that good calibration implies, in naive and sophisticated subjects alike, and that proper scoring rules did not remove the effect. The Bojke report links availability and anchoring to this overconfidence, which leads to the systematic underestimation of uncertainty, and notes that overconfidence is prevalent among experts as well as novices.

Framing the same outcome two ways

A framing effect occurs when two descriptions of the same options lead to different choices. Kahneman's Nobel lecture explains that rational choice requires invariance, meaning that preferences should not change with irrelevant features of how options are described, and that framing effects violate it. In the best known demonstration, a programme described as saving 200 of 600 people at risk was preferred by a substantial majority over a risky alternative, while a clear majority of other respondents preferred the risky option when the same certain programme was described as 400 people dying.

Kahneman also describes a health example from McNeil, Pauker, Sox and Tversky, in which choices between surgery and radiation therapy changed when outcomes were stated as survival or as mortality, because 90% short-term survival sounds less threatening than 10% immediate mortality. The survival frame produced a substantially higher preference for surgery, and the effect was no smaller among experienced physicians than among patients. How a decision aid, valuation survey or committee paper presents outcomes is therefore a design choice that can move the answer.

Where cognitive bias enters a health economic evaluation

Cognitive bias reaches an evaluation by three main routes. Each affects a different set of model inputs and has its own evidence base.

Clinical decisions and resource use. Saposnik and colleagues reviewed 20 studies covering 6,810 physicians and identified 19 cognitive biases. Overconfidence, lower tolerance to risk, anchoring, and information and availability biases were associated with diagnostic inaccuracies in 36.5% to 77% of case scenarios, and five of seven studies that looked for it found an association with therapeutic or management errors. Biased clinical judgement can change test ordering, prescribing and admissions, and so the costs and outcomes observed in routine data and the comparator pathway that an evaluation models.

Preference elicitation. Health state valuation asks people to trade health against risk or time. Bleichrodt argued that probability weighting, loss aversion and scale compatibility, all departures from expected utility, bias the utilities produced. In the standard gamble their combined effect, together with utility curvature, is an upward bias, while in the time trade-off they push in both directions. The biased utilities then enter the QALY.

Expert judgement for model inputs. When data are missing, elicited distributions feed probabilistic sensitivity analysis. The Bojke report treats heuristics and biases as a concern across every stage of structured expert elicitation and separates cognitive biases, which come from how information is processed, from motivational biases, which come from a stake in a particular outcome. An overconfident distribution understates parameter uncertainty and so understates decision uncertainty.

Worked example of base-rate neglect in a screening pathway

All numbers are illustrative. A screening test with sensitivity 0.90 and specificity 0.95 is offered to 10,000 people, of whom 1% have the condition. A positive result can lead straight to treatment costing GBP 2,000 per person, or to a confirmatory test costing GBP 300, assumed here to be perfectly accurate, with treatment only for confirmed cases.

1. Count the results. Of 10,000 people, 100 have the condition and 9,900 do not. The test finds 0.90 × 100 = 90 true positives and misses 10. Among those without the condition, 0.05 × 9,900 = 495 test positive. Total positives are 90 + 495 = 585.

2. Compute the positive predictive value.

$$PPV = \frac{Se \times p}{Se \times p + (1 - Sp)(1 - p)} = \frac{0.90 \times 0.01}{0.90 \times 0.01 + 0.05 \times 0.99} = \frac{0.009}{0.0585} \approx 0.154$$

where $PPV$ is the probability of the condition given a positive result, $Se$ is sensitivity, $Sp$ is specificity and $p$ is prevalence. Only about 15% of positives (90 of 585) have the condition.

3. Compare the two pathways per positive result. Treating straight away costs GBP 2,000 per positive. Confirming first costs GBP 300 plus GBP 2,000 for the share confirmed, so it is cheaper whenever the positive predictive value is below 1 minus 300 / 2,000 = 0.85. A judge who neglects the base rate and reads a positive result as 90% certain expects confirmation to cost 300 + 0.90 × 2,000 = GBP 2,100 per positive and chooses to treat directly. With the correct value, confirming first costs 300 + 0.154 × 2,000 = about GBP 608 per positive.

4. Total the cost for 10,000 people screened. Treating every positive costs 585 × 2,000 = GBP 1,170,000, of which 495 × 2,000 = GBP 990,000 is spent on people without the condition. Confirming first costs 585 × 300 = GBP 175,500 for confirmatory tests plus 90 × 2,000 = GBP 180,000 for treatment, a total of GBP 355,500. The biased choice costs GBP 814,500 more and exposes 495 people to treatment they cannot benefit from.

5. Interpret. The test, the costs and the decision rule are identical in both judgements; only the neglect of prevalence differs, and it reverses the decision. If prevalence were 10%, the test would find 900 true and 450 false positives, a positive predictive value of 0.667, and confirmation would still pay. Counting people in natural frequencies, as in step 1, makes the base rate visible.

Reducing bias by changing the task

Evidence on debiasing favours changing the structure of the judgement over warning people about bias. The Bojke report lists the behavioural strategies with consistent empirical support in expert elicitation. Experts can be prompted to consider more information, for example by giving lowest, highest and median values separately instead of a single interval, and the report concludes that confidence intervals should not be elicited in a single stage. Repeated feedback on the accuracy of judgements improves accuracy and reduces overconfidence. Elicitation materials should avoid unnecessary anchors, and eliciting lower and upper bounds before the median may reduce anchoring on it. Where experts with relevant knowledge also have a stake in the result, including a range of viewpoints can dilute motivational bias. By contrast, simply warning experts not to be biased is largely ineffective, although in-depth training on biases has been found more effective.

Evidence in clinical practice is thinner. Saposnik and colleagues found only one study in their review that evaluated an intervention to reduce cognitive bias in physicians. The same logic of changing the task applies to patient-facing design, which the choice architecture intervention page covers, and to presenting results in absolute numbers and natural frequencies alongside relative measures.

Boundaries with neighbouring ideas

Cognitive bias concerns errors in human judgement; statistical bias concerns estimators and study designs. Both are systematic, and both contrast with random error, which scatters around the true value without a consistent direction.

Cognitive bias is also distinct from preference. Saposnik and colleagues separate choice tendencies that are factually wrong from decisions that are only potentially suboptimal where no objectively correct decision exists, such as risk aversion or tolerance to ambiguity. A patient who prefers a certain modest gain to a risky larger one may be expressing a preference, not making an error. Bounded rationality covers limited search and computation, and behavioural theory covers present bias, reference dependence and their welfare consequences.

When the bias label misleads

The commonest misunderstanding treats every heuristic as a fault. Marewski and Gigerenzer note that the heuristics-and-biases framework keeps optimisation, such as maximisation of expected utility, as its normative yardstick, whereas on the ecological view a heuristic is judged by its success against an outside criterion, such as a fast and accurate decision in an emergency room. A judgement can therefore count as biased against a coherence standard and still perform well in practice. Calling a judgement biased requires a stated benchmark.

The clinical evidence base is weaker than its reputation. In the Saposnik review 12 of the 20 studies (60%) were rated low quality, 14 used narrative case vignettes instead of real clinical encounters, and only two looked at patient outcomes. Effects shown in hypothetical tasks may not carry over to real decisions with real stakes. Analysts and committees use the same heuristics as the people they study, so the safeguards apply to model building too.

Sources

• Bleichrodt H. A new explanation for the difference between time trade-off utilities and standard gamble utilities. Health Economics. 2002;11(5):447-456. doi:10.1002/hec.688
• Bojke L, Soares M, Claxton K, Colson A, Fox A, Jackson C, Jankovic D, Morton A, Sharples L, Taylor A. Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study. Health Technology Assessment. 2021;25(37):1-124. doi:10.3310/hta25370
• Kahneman D. Maps of bounded rationality: a perspective on intuitive judgment and choice. Nobel Prize Lecture, 8 December 2002. https://www.nobelprize.org/uploads/2018/06/kahnemann-lecture.pdf
• Marewski JN, Gigerenzer G. Heuristic decision making in medicine. Dialogues in Clinical Neuroscience. 2012;14(1):77-89. doi:10.31887/DCNS.2012.14.1/jmarewski
• Saposnik G, Redelmeier D, Ruff CC, Tobler PN. Cognitive biases associated with medical decisions: a systematic review. BMC Medical Informatics and Decision Making. 2016;16:138. doi:10.1186/s12911-016-0377-1
• Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science. 1974;185(4157):1124-1131. doi:10.1126/science.185.4157.1124