Decision Algorithm

Decision Algorithm in HTA Appraisal and Economic Models

A health technology assessment (HTA) body can reach a recommendation by following steps written down in advance, by committee deliberation, or, as most do, by a mixture of the two. The term decision algorithm names the explicit part, meaning the ordered steps, cut-offs and combination rules that place a technology or a trial result into a category without case-by-case weighing at that step. A decision algorithm differs from the decision rule of economic evaluation and from the criteria an appraisal draws on, and NICE and the German benefit assessment formalise their appraisals to different degrees. The term also has a modelling sense, in which a clinical algorithm such as a test sequence is a strategy being compared.

Criteria, rule and algorithm as three layers of an appraisal

Three ideas are often run together in accounts of HTA decisions, and keeping them apart shows where judgement enters. Decision criteria are what the decision-maker takes into account, such as severity, certainty of evidence and cost-effectiveness. The decision rule is the choice criterion of economic evaluation, namely an incremental cost-effectiveness ratio (ICER) or net benefit compared with a threshold, including its frontier form when several options are compared, as that page sets out.

A decision algorithm, in the sense used here, is a pre-set, multi-step classification procedure used in appraisal or in grading the benefit of a technology, or a clinical pathway that an economic model evaluates. The steps, cut-offs and possible output categories are fixed before the evidence is seen, so two analysts given the same inputs reach the same category. An algorithm of this kind often works on clinical outcomes alone and need not involve costs at all.

How far HTA bodies formalise the appraisal

Neither NICE nor the German system hands the whole decision to an algorithm. The useful question is which steps are fixed and which are left to judgement, since that sets what an analyst can predict from the evidence.

NICE's appraisal is largely deliberative, as described under decision making, but its manual (PMG36) contains rule-based steps. The severity modifier follows a fixed rule. The QALY weight is set by absolute or proportional shortfall, whichever implies the greater severity, and a value on a cut-off takes the higher level. A recommendation with managed access can be considered only when three conditions, joined by "and", all hold, the last being that the needed data could feasibly be collected within a reasonable time, up to a maximum of 5 years. A report for the NIHR Health Technology Assessment programme by Claxton and colleagues represented the sequence of assessments needed for NICE to recommend a technology only in research or approval with research as an algorithm, summarised as a 7-point checklist, and concluded that the right category of guidance depends only partly on expected cost-effectiveness.

The German benefit assessment of new medicines formalises more. The Institute for Quality and Efficiency in Health Care (IQWiG) grades the extent of added benefit at outcome level by published statistical thresholds, but its methods state that a strict formalisation of the overall conclusion is not possible, because no sufficient abstraction is known for the value judgements involved. IQWiG's reports serve as a recommendation to the Federal Joint Committee (G-BA), which decides and considers health care aspects beyond the benefit assessment.

Extent of added benefit in German benefit assessment as a published algorithm

IQWiG's General Methods (version 8.0, December 2025) set out a three-step procedure for the extent of added benefit of a medicine over the appropriate comparator therapy. The first two steps are largely rule-based, although step 1 rests on judgements about risk of bias and the certainty of results, and the third is a reasoned judgement.

1. Probability, per outcome. The certainty of the evidence for each outcome is classed as a hint, an indication or proof.

2. Extent, per outcome. For outcomes with at least a hint of an effect, a quantifiable effect is graded major, considerable or minor by comparing the upper limit of the two-sided 95% confidence interval with thresholds. The extent is non-quantifiable when the effect cannot be reliably quantified, for example when only an effect on a surrogate is available.

3. Overall conclusion. All outcomes are considered together and the result is placed in one of six categories set by the Ordinance on the Benefit Assessment of Medicinal Products, namely major, considerable, minor or non-quantifiable added benefit, no added benefit proven, or less benefit than the comparator.

For binary outcomes the relative risk is arranged so that benefit is a value below 1, and the same thresholds apply to hazard ratios. The upper confidence limit must lie below the threshold to reach a category, and the thresholds are stricter for less serious outcomes.

For non-serious outcomes, IQWiG states that effects judged only marginal, including statistically significant ones, lead to the category of no added benefit.

The outcome-level step written as a rule

For a quantifiable effect, the second step can be written as a function of one number, which shows why it counts as an algorithm. The same function applies to every outcome, and only the thresholds change with the outcome category.

$$\text{Extent}_k(u) = \begin{cases} \text{major} & u < M_k \\ \text{considerable} & M_k \le u < C_k \\ \text{minor} & C_k \le u < N_k \\ \text{no added benefit for this outcome} & u \ge N_k \end{cases}$$

where $u$ is the upper limit of the two-sided 95% confidence interval for the relative risk or hazard ratio, oriented so that benefit lies below 1; $k$ is the outcome category; and $M_k$, $C_k$ and $N_k$ are the thresholds for major, considerable and minor extent in that category. Non-serious outcomes have no major threshold, and for serious outcomes the major class also needs an event risk of at least 5% in one group.

Worked example with one confidence limit and three outcome categories

The figures in this example are illustrative and do not describe a real medicine. A new medicine is compared with the appropriate comparator therapy, step 1 is assumed to have found at least a hint of an effect on three outcomes, and each result has an upper confidence limit of 0.92.

1. Mortality. The upper limit of 0.92 is not below 0.85, so the effect is not major, but it is below 0.95, so the extent is considerable.

2. Serious adverse events. The upper limit of 0.92 is not below 0.90 but is below 1.00, so the extent is minor. The 14% risk meets the 5% condition for major, but the interval does not reach 0.75.

3. Non-serious symptom. The upper limit of 0.92 is not below 0.90, so the effect does not reach minor extent. The result is statistically significant, since the interval excludes 1, yet this outcome contributes no added benefit.

4. Sensitivity at the cut-off. Precision moves results between grades, and the distance to the next cut-off sets how much evidence that takes. On the usual approximation that the variance of a log ratio falls in proportion to the number of events, about 46% more serious adverse events, with the standard error of the log relative risk falling from about 0.085 to 0.070, would give 0.78 (0.68 to 0.89) and a considerable extent. Moving the mortality result to major, with an interval such as 0.80 (0.76 to 0.84), would need roughly seven times the information, as the standard error of the log hazard ratio would have to fall from about 0.070 to 0.026.

5. Overall conclusion. Combining a considerable effect on mortality, a minor effect on serious adverse events and nothing on the non-serious symptom into one of the six categories is a judgement, for which IQWiG gives a justified proposal and the G-BA decides. Any outcome showing harm would also be weighed here.

Scoring in multi-criteria decision analysis as an explicit combination step

Multi-criteria decision analysis (MCDA) can make the combination of criteria explicit. In its simplest form, options are scored on each criterion, the criteria are weighted, and the weighted scores are added to give an overall value for each option.

Whether MCDA acts as a decision algorithm depends on how the total is used. When the ranking by overall value settles the choice, the weighted sum is the algorithm and the value judgements sit in scores and weights fixed in advance. When the scores are one input to committee discussion, the final step is again judgement. Baltussen and Niessen note that informal reading of a performance matrix is prone to well-documented distortions of human judgement, which is the case for writing the combination step down. MCDA in HTA covers its use by assessment bodies.

Clinical algorithms as the strategies a model compares

In economic modelling, the term also refers to a clinical decision algorithm, such as a diagnostic test sequence or a rule for stopping treatment. Here the algorithm is the object being evaluated, so each candidate algorithm is one strategy, and a model estimates its expected costs and health effects.

NICE's manual (PMG36, section 4.6.13) notes that a new diagnostic technology can replace or be added to an existing test or test sequence, or be combined with parts of the existing process to create a new sequence, and that most outcomes arise downstream from the treatment that follows testing. For treatment continuation rules, which continue treatment only for people who achieve a specified response within a given time, section 4.9.13 asks for the rule to be analysed as an additional treatment strategy, not as a separate subgroup, so that costs such as extra monitoring are captured.

In a decision tree, each clinical algorithm becomes a branch leaving the decision node, with chance nodes for test results and responses, and a treatment sequence model does the same for ordered lines of therapy. The appraisal algorithm of an HTA body turns evidence into a recommendation; the clinical algorithm inside a model is one of the options that recommendation is about.

Choices that change what an algorithm returns

An algorithm is only as neutral as the choices built into it, which are value judgements made once, at design time. The choices below change the category returned.

• Placement of cut-offs. Moving a threshold moves results between categories, as the worked example shows.
• The statistic used. Using the upper confidence limit rather than the point estimate makes precision part of the classification.
• Order and gating. In the German procedure, extent is graded only for outcomes where step 1 found at least a hint of an effect, so an outcome that fails step 1 is never graded, whatever its point estimate.
• The combination step. A weighted sum and a committee judgement can rank the same outcome-level results differently.

Where algorithmic appraisal is misread

Several misreadings recur when HTA processes are described as algorithms. Each is listed with the reason it fails.

• An algorithm means an automatic decision. IQWiG states that its overall conclusion cannot be strictly formalised, and the G-BA considers aspects beyond the benefit assessment.
• A decision algorithm is the same as a decision rule. The rule selects the efficient option at a threshold; an algorithm grades evidence through pre-set steps.
• A statistically significant result always counts as a benefit. Under IQWiG's thresholds a significant but marginal effect on a non-serious outcome gives no added benefit for that outcome.
• Explicit steps remove value judgements. They move value judgements into the cut-offs and weights, where they are easier to inspect but still need justification.

Sources

• Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effectiveness and Resource Allocation. 2006;4:14. doi:10.1186/1478-7547-4-14
• Claxton K, Palmer S, Longworth L, Bojke L, Griffin S, McKenna C, Soares M, Spackman E, Youn J. Informing a decision framework for when NICE should recommend the use of health technologies only in the context of an appropriately designed programme of evidence development. Health Technology Assessment. 2012;16(46). doi:10.3310/hta16460
• Institute for Quality and Efficiency in Health Care (IQWiG). General Methods, version 8.0. Cologne: IQWiG; 19 December 2025. Sections 1.3.2, 3.1.4 and 3.3.3, and Appendix A. doi:10.60584/General-Methods_V8.0
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; published 31 January 2022, last updated 13 October 2026. Sections 4.6.13, 4.9.13, 6.2.18 and 6.4.6.