Evidence

Evidence: How Information Becomes the Inputs to a Health Economic Decision

Every economic evaluation rests on estimates of what the options in a decision problem will cost and what they will do to health, and evidence is the material from which those estimates are built. In health economics the term reaches well beyond the main clinical trial, to registry data, unit costs, quality of life values and, when empirical data run out, structured expert judgement. This page explains how evidence is tied to a decision problem, which kinds of evidence feed which model inputs and how evidence becomes a parameter with stated uncertainty. It also separates the decision made on current evidence from the decision to collect more, and notes where evidence-based medicine and decision analysis use the word differently.

Evidence is defined by the decision problem

Whether a piece of information counts as evidence for an evaluation depends on the decision problem it is meant to inform. The decision problem names a population, an intervention, the comparators and the outcomes; in a NICE appraisal these elements, often abbreviated PICO, are set out in the scope. The NICE manual for technology appraisal (PMG36) asks that the evidence considered by the committee be relevant in terms of patient groups, comparators, perspective, outcomes and resource use as defined in the scope. A trial in a different population can still be used, but its relevance has to be argued, adjusted for or carried as uncertainty.

The same manual separates the two quantities that an economic evaluation in health technology assessment needs. Effectiveness requires the effects of the technology and the comparators to be quantified on appropriate outcome measures, and costs require evidence on resource use in physical units, such as days in hospital or visits to a GP, valued using prices and unit costs.

Which evidence feeds which model input

A health economic model draws on several bodies of evidence at once, and the most suitable source differs from one input to the next. The NICE manual states that NICE considers all types of evidence, including unpublished, non-UK, registry and other observational sources, and that the preferred source depends on the use being considered. The table summarises the usual pattern.

Clinical evidence and economic evidence are narrower categories, each covering part of the table, and this page treats evidence as the general class that contains them. Patient evidence, the lived experience of a condition and its treatment reported by patients and patient organisations, sits outside the parameter table and informs the committee's judgement on issues that the NICE manual asks an evaluation to consider beyond effects and costs, such as the impact of having a condition and the experience of specific treatments. An evidence base, or body of evidence, is the collection of evidence relevant to one question, and real-world data become evidence once they have been analysed for that question.

Turning evidence into model parameters

Evidence enters a model as parameter values, and each value carries a description of how uncertain it is. The NICE manual asks for evidence on outcomes to come from a systematic review and describes modelling as a framework for synthesising the available evidence into estimates of clinical and cost effectiveness. When several studies estimate the same quantity, meta-analysis or network meta-analysis combines them into the parameter. For probabilistic sensitivity analysis, the manual asks that the parameter distributions be chosen to represent the available evidence on the parameter, not arbitrarily.

Under a fixed-effect assumption, the pooled estimate and its standard error become the mean and standard deviation of the distribution that the parameter is given in the probabilistic analysis, and each further study of the same effect narrows that distribution. If the studies are judged to estimate different true effects, a random-effects model is used instead, and when heterogeneity is present its interval is wider than the fixed-effect one. Whether narrowing a distribution further is worth paying for is a value of information question, taken up in the next section.

Deciding on current evidence and deciding whether to collect more

Evidence serves two decisions in health economics that are taken at the same time but are conceptually separate. Claxton argued that decisions should be based only on mean net benefits, whether or not differences are statistically significant, and that the distribution of net benefit matters only for deciding whether more information is required. Sculpher and colleagues set out the same two decisions: whether to adopt a technology given existing evidence, and whether more evidence is needed to support that decision in the future.

The first decision uses the expected net benefit of each option on the evidence available now. The second uses value of information methods, such as the expected value of perfect information, which put a value on reducing the uncertainty that the evidence leaves. The NICE manual asks the overall assessment of uncertainty to highlight uncertainties that are unlikely to be reduced by further evidence or expert input, which marks where further research would not help.

Judging the quality and relevance of evidence

Evidence is judged on freedom from bias and on applicability to the decision, which the NICE manual frames as internal validity, affected by data quality or methodological concerns, and external validity, affected by differences in population and setting. It asks for limitations to be fully described and their impact on bias and uncertainty characterised and, ideally, quantified. For non-randomised studies, the manual points to confounding, selection bias and informational bias as reasons for a higher risk of bias, and asks for each study to be assessed with a validated risk-of-bias tool.

The GRADE approach, described by Balshem and colleagues, rates the quality of evidence in four categories: high, moderate, low and very low. The rating applies to a body of evidence, not to single studies. Randomised trials start as high-quality evidence and observational studies as low quality; evidence can be rated down for risk of bias, imprecision, inconsistency, indirectness or publication bias, and observational evidence can be rated up. GRADE also keeps the rating of evidence separate from the strength of a recommendation.

Where usage of the term differs

In evidence-based medicine, evidence usually means research findings on the effects of care, rated in the way just described. In decision modelling, evidence means anything that informs a parameter, including registry data, unit cost schedules and expert judgement. Sculpher and colleagues call for estimates based on all the available evidence, combined through evidence synthesis so that decision uncertainty is fully represented. On that view, the weakness of a source shows up in the uncertainty around a parameter instead of leading to its exclusion.

The NICE manual sits between the two senses. It strongly prefers randomised trials for relative effects, yet allows expert elicitation to provide evidence when there is no empirical evidence from trials, non-randomised studies or registries, and it treats the opinions of clinical and patient experts as a separate input that may supplement, support or refute observed data.

Common misreadings of evidence in economic evaluation

Several errors recur in the way models handle evidence. The first is to treat a difference that is not statistically significant as no difference and set the effect to zero; on the decision-analytic view the mean is used and the uncertainty is carried forward. The second is to treat the main registration trial as the whole evidence base; Sculpher and colleagues argue that a single trial used as the vehicle for economic analysis is, in most circumstances, an inadequate and partial basis for decision making.

A third error is to source the treatment effect through a systematic review while other inputs are chosen without a stated search. Cooper and colleagues found this pattern in NHS Health Technology Assessment programme models from 1997 to 2003: the main clinical effect usually came from the companion systematic review, but search strategies for adverse events, baseline clinical data, resource use and utilities were rarely made explicit. A fourth is to treat missing long-term evidence as a reason to leave long-term effects out. The NICE manual notes that a lifetime horizon often requires extrapolation beyond the available evidence, and describes scenario analyses that include assuming no further benefit beyond the period of treatment, alongside more optimistic assumptions. A gap in the evidence becomes an assumption that needs to be stated and tested.

Sources

• Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology. 2011;64(4):401-406.
• Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. Journal of Health Economics. 1999;18(3):341-364.
• Cooper N, Coyle D, Abrams K, Mugford M, Sutton A. Use of evidence in decision models: an appraisal of health technology assessments in the UK since 1997. Journal of Health Services Research and Policy. 2005;10(4):245-250.
• Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA. Chapter 10: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. Section 10.10.4.
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, updated March 2026. Chapters 2, 3 and 4.
• Sculpher MJ, Claxton K, Drummond M, McCabe C. Whither trial-based economic evaluation for health care decision making? Health Economics. 2006;15(7):677-687.