Evidence-Based Decision Making

Evidence-Based Decision Making in Health Policy and HTA

In health care, evidence-based decision making means that a ministry, payer, guideline panel or manager chooses only after the best research on effects, costs and certainty has been found, appraised and set out openly beside the values and constraints that also bear on the choice. It carries the logic of evidence-based medicine from the bedside to the committee room, and many writers prefer the term evidence-informed decision making because research never settles a policy question on its own. For health economists it is the idea behind health technology assessment, since an appraisal committee weighing a cost-effectiveness estimate against uncertainty, equity and budget limits is practising evidence-based decision making in institutional form.

From evidence-based medicine to evidence-informed policy

Oxman, Lavis, Lewin and Fretheim, writing in 2009 for the SUPPORT series on health policymaking, traced the idea to the drive towards evidence-based medicine from the early 1990s, which focused at first on decisions by physicians. It was later extended to other professionals, to consumers and to management and policymaking, where it became known as evidence-based policy.

Both have been criticised for assuming that research largely determines practice or policy. Oxman and colleagues called this largely a misperception, because neither patient nor policy decisions are determined by evidence alone. They treated the terms evidence-based and evidence-informed as interchangeable, but chose evidence-informed because it better describes the role of research in policy.

The usual foil is eminence-based decision making. The phrase comes from a satirical BMJ piece by Isaacs and Fitzgerald in 1999, which described colleagues for whom seniority and experience counted for more than evidence. The satire was about clinical practice, but the same failure would appear in health policy if a benefit were funded because a senior figure or a persuasive advocate wanted it, with no record of what the evidence showed.

What counts as evidence in a policy or coverage decision

Oxman and colleagues adopted a common understanding of evidence as facts, actual or asserted, intended for use in support of a conclusion, so evidence alone does not make decisions. Expert opinion is more than evidence, because it combines facts with interpretation and conclusions, and judgements about the quality and applicability of research are better made systematically and explicitly. All evidence is context-sensitive, so the best evidence from around the world is the starting point for judging the effects of policies, while local evidence on need, values, costs and resources informs most other judgements.

Evidence-informed policymaking is therefore characterised by systematic and transparent access to, and appraisal of, evidence as an input, even though the wider policymaking process is not assumed to be systematic. Oxman and colleagues gave the systematic review as the example, because it reduces the risk of being misled by chance or by biased selection of studies.

Culyer, analysing NICE in 2006, treated its appraisals as a deliberative process that weighs scientific evidence alongside colloquial evidence fitting it to the context.

The GRADE Evidence to Decision frameworks

A widely used structure for moving from evidence to a decision is the set of Evidence to Decision (EtD) frameworks developed by the GRADE Working Group and described by Alonso-Coello and colleagues in 2016. Their purpose is to help panels use evidence in a structured and transparent way for clinical recommendations, coverage decisions, and health system or public health decisions. Each framework has three main sections, covering formulation of the question, assessment of the evidence, and drawing conclusions. The authors reported that more than 100 organisations, including WHO, Cochrane and NICE, use or have adopted the principles of the GRADE system.

All five sets of EtD criteria ask about the same core matters, and the frameworks that take a population perspective add equity. The table lists these criteria and the kind of health economics or HTA evidence that typically informs each.

The perspective a panel takes determines which economic consequences are counted, so panels are asked to state it. A coverage decision cannot be strong or weak like a clinical recommendation, so the options are no coverage, coverage only in research, coverage with price negotiation, restricted coverage, and full coverage.

The authors likened the difference between EtD frameworks and multi-criteria decision analysis, which weights criteria in a mathematical model, to that between a balance sheet and an economic evaluation, and noted that few organisations are likely to have the resources for MCDA.

HTA as institutionalised evidence-based decision making

The international definition of HTA agreed in 2020 by a joint task group led by INAHTA and HTAi describes it as a multidisciplinary process that uses explicit methods to determine the value of a health technology, with the purpose of informing decision making. HTA bodies are, in this sense, standing institutions for evidence-based decision making.

The NICE manual for technology appraisal (PMG36) shows how this works in practice. Its appraisal committee bases recommendations on the evidence presented, including stakeholder statements and the views of experts at the meeting, and considers advice from NICE on the approach to scientific and social value decisions. The manual states that the credibility of NICE guidance depends on the transparency of the committee's process, and that decisions must be explained with reference to all the available evidence, expert contributions and consultation comments.

Cost-effectiveness enters as evidence to be weighed, not as an automatic rule. PMG36 says the committee does not use a precise maximum acceptable ICER, because the relevant benchmark is the opportunity cost of programmes displaced and NICE lacks complete information on all of them. Below a most plausible ICER of GBP 25,000 per QALY gained, the decision normally rests on the cost-effectiveness estimate. Above that level the committee considers the certainty around the ICER, uncaptured benefits and non-health factors, and health inequalities, and above GBP 35,000 per QALY it needs an increasingly stronger case. The cost-effectiveness threshold range thus sets how much weight other evidence must carry.

Fair process, deliberation and accountability for reasonableness

Evidence answers questions about effects and costs, but people reasonably disagree about how to share limited resources. Daniels argued in 2000 that, without agreed principles, pluralist societies need a fair process. The elements he named were transparency about the grounds for decisions, rationales that all can accept as relevant to meeting health needs fairly, and procedures for revising decisions in the light of challenges. Together these make up accountability for reasonableness.

Baltussen, Jansen and Oortwijn turned these conditions into evidence-informed deliberative processes for HTA bodies designing a benefit package, with evidence-informed evaluation as one of four elements of legitimacy.

Worked example with net health benefit at two threshold values

All figures in this example are illustrative. A new medicine adds 0.25 QALYs per patient at an extra cost of GBP 8,000 per patient compared with standard care, and 1,500 patients a year would be eligible. The incremental cost-effectiveness ratio is calculated first.

$$ICER = \frac{\Delta C}{\Delta E} = \frac{8000}{0.25} = 32000$$

where $\Delta C$ is the incremental cost per patient in GBP and $\Delta E$ is the incremental QALYs per patient, giving GBP 32,000 per QALY gained.

PMG36 asks for net health benefit at GBP 25,000 and GBP 35,000 per QALY when decision-making modifiers are applied, and the same two values show here what the evidence implies at each end of the range.

$$NHB = \Delta E - \frac{\Delta C}{k}$$

where $NHB$ is the net health benefit per patient in QALYs, $\Delta E$ and $\Delta C$ are as above, and $k$ is the value placed on a QALY in GBP.

At GBP 25,000 per QALY the medicine would cause a net loss of about 105 QALYs a year once health displaced elsewhere is counted, and at GBP 35,000 it would add about 32. The evidence does not decide the case, so the committee must state its view on the certainty of the 0.25 QALY gain, on uncaptured benefits and on inequalities. If a plausible scenario cut the gain to 0.18 QALYs, the ICER would be 8,000 / 0.18, or about GBP 44,400 per QALY, and the net health benefit at GBP 35,000 would be -0.049 QALYs per patient (0.18 less 0.229). If a commercial arrangement lowered the extra cost to GBP 6,000, the ICER would be 6,000 / 0.25 = GBP 24,000 and the net health benefit at GBP 25,000 would be 0.010 QALYs per patient (0.25 less 0.240), or 15 QALYs a year, which corresponds to the EtD option of coverage with price negotiation.

Evidence-based decision making beside related ideas

Evidence-based medicine concerns care for an individual patient, and shared decision making adds that patient's preferences. Evidence-based decision making in the policy sense concerns choices for a population under a budget, where opportunity cost and equity become explicit criteria. Knowledge translation and knowledge brokering deal with getting research to decision makers and getting it used, while evidence-based decision making concerns how the evidence is weighed once it is on the table.

Within HTA, cost-effectiveness analysis supplies one criterion, while EtD frameworks and MCDA structure the weighing of all of them. When the evidence is too uncertain to support a firm decision, value of information analysis shows whether further research is worth paying for, and a managed entry agreement or a NICE recommendation with managed access lets further evidence be gathered under agreed terms.

Limitations and common misreadings

The most common misreading is that evidence-based means evidence-determined. Oxman and colleagues stressed that judgements, values and other factors always play a part, and the worked example shows one ICER supporting different decisions depending on certainty and factors outside the model.

The second is that a formal process guarantees honest use of evidence. Oxman and colleagues listed misuses such as selective use and creating a spurious impression of uncertainty, and saw systematic, transparent processes as the best protection. Alonso-Coello and colleagues noted that research evidence is often lacking for some EtD criteria, and that a panel which drops the resource use criterion either makes implicit judgements about resources or leaves them to the guideline's users.

Process also has costs. EtD frameworks are more complex than the earlier GRADE tables, and Baltussen and colleagues noted that transparency may be hard to achieve where open decision making has no tradition.

Sources

• Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking? Health Research Policy and Systems 2009;7(Suppl 1):S1. https://doi.org/10.1186/1478-4505-7-S1-S1
• Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 2016;353:i2016. https://doi.org/10.1136/bmj.i2016
• Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine. BMJ 1999;319:1618. https://doi.org/10.1136/bmj.319.7225.1618
• Culyer AJ. NICE's use of cost effectiveness as an exemplar of a deliberative process. Health Economics, Policy and Law 2006;1(3):299-318. https://doi.org/10.1017/S1744133106004026
• O'Rourke B, Oortwijn W, Schuller T, International Joint Task Group. The new definition of health technology assessment: a milestone in international collaboration. International Journal of Technology Assessment in Health Care 2020;36(3):187-190. https://doi.org/10.1017/S0266462320000215
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36), section 6, Committee recommendations. Published 31 January 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36/chapter/committee-recommendations-2
• Daniels N. Accountability for reasonableness: establishing a fair process for priority setting is easier than agreeing on principles. BMJ 2000;321:1300-1301. https://doi.org/10.1136/bmj.321.7272.1300
• Baltussen R, Jansen M, Oortwijn W. Evidence-informed deliberative processes for legitimate health benefit package design, part I: conceptual framework. International Journal of Health Policy and Management 2022;11(10):2319-2326. https://doi.org/10.34172/ijhpm.2021.158