MCDA in HTA

MCDA in HTA, with Criteria and Weights Beside Cost-Effectiveness Analysis

Health technology assessment (HTA) agencies acknowledge factors beyond cost and health benefit, such as equity and burden of disease, but they are not always measured precisely or valued formally. Multi-criteria decision analysis (MCDA) makes them explicit. It lists the criteria, records how each technology performs on them and, in its quantitative form, attaches weights so that a committee's trade-offs can be inspected and reused. Health economists have criticised it for double counting, weak handling of opportunity cost and an additive model that can reward a technology with no health gain. In HTA it is usually set beside cost-effectiveness analysis, and the general method is described under multi-criteria decision analysis.

Why appraisal committees turn to explicit criteria

The first report of the ISPOR MCDA Emerging Good Practices Task Force contrasts two definitions. Keeney and Raiffa require criteria to be combined into one overall appraisal, while Belton and Stewart allow MCDA to support deliberation without aggregation; the task force took the broader view, so a table of evidence against agreed criteria already counts as partial MCDA.

HTA is a repeated decision, and the report suggests that the same principles and criteria should arguably be used across reimbursement decisions for consistency and accountability. Weights could come from committee members, patients or the general public, a normative choice that can change the result. Value measurement models, which build a numerical total for each alternative, are by far the most used in health care, and the task force stresses that MCDA helps decision makers reach "their decision, not the tool's decision".

Three ways a committee can use the performance matrix

Baltussen and colleagues, in a consensus statement for HTA agencies, reviewed 36 MCDA studies in HTA from 1990 to 2018. They classified them by how the committee interprets the performance matrix, the table of each technology's performance on each criterion.

The authors advise agencies always to include deliberation and, at a minimum, to undertake qualitative MCDA. They class the Dutch and UK agencies' approaches as MCDA with decision rules.

The ISPOR steps in a technology appraisal

The first task force report lists eight steps for a value measurement MCDA, and the second attaches a good practice checklist to each. The steps are iterative, not strictly sequential.

1. Define the decision problem, covering objectives, alternatives, stakeholders and output; an HTA committee may decide in the interests of the general population.
2. Select and structure criteria that are complete, nonredundant, nonoverlapping (no double counting) and preference independent.
3. Measure performance, drawing on sources from systematic reviews to expert elicitation, summarised in the performance matrix.
4. Score alternatives, where value functions put performance on a common scale, often 0 to 100; criteria reused across decisions, as in HTA, need "global" scales covering the best and worst realistic performance.
5. Weight criteria, setting trade-offs between criteria, which the report likens to exchange rates.
6. Aggregate, usually with an additive (weighted-sum) model.
7. Handle uncertainty, using probabilistic sensitivity analysis for parameters, scenarios for the choice of criteria, and weights from different stakeholder groups for heterogeneity.
8. Report and examine findings with the committee before it decides.

Weighting methods including swing weighting, DCE and AHP

The weighting method changes what a weight means. The second report explains that weights are more likely to be true scaling constants, showing how far a change in one criterion compensates for a change in another, when elicitation uses performance ranges and asks for trade-offs, conditions best met by swing weighting and decompositional methods.

EVIDEM, an MCDA framework built for HTA

The "Evidence and Value: Impact on DEcisionMaking" (EVIDEM) framework, published in 2008, has seven modules. Its centre is an MCDA Value Matrix of 15 quantifiable components, in which each evaluator weights every component on a 1 to 5 scale and scores the technology from 0 to 3 against defined anchors, combined by a simple linear model. From 2011, Lombardy Region in Italy used EVIDEM criteria within its VTS framework, with 15 quantitative and 6 qualitative criteria, to decide on introducing and delisting technologies. Campillo-Artero, Puig-Junoy and Culyer argue that the 2017 (10th) edition double counts by including expenditures alongside criteria for opportunity costs and financial feasibility, and that its benefit criteria overlap.

What HTA bodies have done with MCDA

The cases below come from the first task force report and the consensus statement. IQWiG's current methods follow the table.

IQWiG's General Methods (version 8.0, 2025) allow outcomes to be aggregated into a single measure or weighted in the conjoint evaluation of benefit and harm. In a health economic evaluation with several outcomes, the decision maker can weight the results or use weights from patient preference elicitation, although the document calls their applicability a subject of scientific debate and says IQWiG does not run such surveys itself under section 35b of the Social Code Book V. It also cites the Institute's own patient preference studies, including its AHP and conjoint analysis pilots.

How MCDA relates to cost-effectiveness analysis

The second task force report notes that cost-utility analysis on welfarist foundations has been shown to be a special case of multicriteria methods. The practical question is how cost enters. In a review cited by the report, 10 of 23 reimbursement MCDAs included cost as a criterion, which the report equates to asking stakeholders for their willingness to pay; it has been argued that stakeholders lack the knowledge of forgone benefits needed to capture opportunity cost.

Marsh and colleagues set out the recommendation that MCDA estimate benefits only, set against incremental cost to form a new incremental cost-effectiveness ratio judged against an opportunity cost threshold, with the QALY kept as one criterion; estimating that threshold for an MCDA scale is harder than for the QALY. The second report adds that IQWiG had suggested combining MCDA benefit with cost in an efficiency frontier, and that a cost-benefit ratio needs benefits on a ratio scale and costs and benefits on a comparable, for example per-patient, basis. Campillo-Artero and colleagues argue that cost-effectiveness is necessary but not sufficient, and that MCDA's procedural strengths can already be adopted within cost-effectiveness analysis and HTA.

Worked example weighting three technologies (illustrative)

The numbers are hypothetical. A committee compares technologies A, B and C with a weighted-sum model and swing weights.

$$V(a)=\sum_{j=1}^{n} w_j\,v_j(x_{aj}),\qquad w_j=\frac{p_j}{\sum_{k=1}^{n}p_k}$$

where $V(a)$ is the overall value of technology $a$ (0 to 100), $x_{aj}$ is its performance on criterion $j$, $v_j$ is the value function giving a 0 to 100 score, $p_j$ is the swing points for criterion $j$ and $w_j$ is the normalised weight.

Step 1 (weights). Health gain runs from 0 to 0.8 QALYs per patient with a linear value function; severity scores 0, 50 or 100 for mild, moderate or severe; strength of evidence scores 0, 50 or 100 for low, moderate or high. Swing points are 100, 60 and 40, a total of 200, so the weights are 0.50, 0.30 and 0.20.

Step 2 (scores). A gains 0.6 QALYs, a score of 0.6 / 0.8 × 100 = 75, for a moderate condition (50) with high-strength evidence (100). B gains 0.4 QALYs (50) for a severe condition (100) with moderate evidence (50). C gains nothing (0) for a severe condition (100) with high-strength evidence (100).

Step 3 (weighted sums).

For A, 37.5 + 15 + 20 = 72.5; for B, 25 + 30 + 10 = 65; for C, 0 + 30 + 20 = 50. A ranks first.

Step 4 (stability). Let $w$ be the health gain weight, with the rest shared 60:40 between severity and evidence, which gives the following.

$$V(A)=75w+70(1-w)=70+5w,\qquad V(B)=50w+80(1-w)=80-30w,\qquad V(C)=100(1-w)$$

where 70 is 0.6 × 50 + 0.4 × 100, 80 is 0.6 × 100 + 0.4 × 50 and 100 is 0.6 × 100 + 0.4 × 100. All three tie at $w = 2/7$, about 0.29, scoring about 71.4. Below that weight C, with no health gain, ranks first. At 0.25, C scores 75, B 72.5 and A 71.25.

Step 5 (cost). If A costs GBP 18,000 per patient and B GBP 8,000, the cost-per-value rule gives about GBP 248 per point for A (18,000 divided by 72.5) and about GBP 123 for B (8,000 divided by 65).

Interpretation. C scores 50 with no health gain, and ranks first whenever the health gain weight is below 2/7, because the additive model lets severity and evidence earn value on their own, the preference independence failure described in the second task force report. Zero on this scale marks the bottom of the chosen ranges (a mild condition, low-strength evidence), not the absence of value, so the totals are not a ratio scale and the step 5 figures cannot show which technology is more efficient. The totals are a structured input to deliberation, in which the committee checks the weight at which the ranking changes and judges value for money against a defined opportunity cost.

Common errors in MCDA for HTA

Baltussen and colleagues attribute most limitations of quantitative MCDA to implementation rather than fundamental problems. The four below are from the studies they reviewed.

1. Double counting. 25 of 35 studies counted one or more criteria twice; the second report's example is cost-effectiveness alongside cost or effectiveness.
2. Preference dependence. 25 of 35 used an additive model. In a South Korean study of cancer drugs, "clinical benefit" carried a weight of 22%, so an ineffective drug strong on other criteria could still score highly.
3. Lost opportunity cost. 29 of 36 included cost as a criterion, assuming respondents know the budget constraint and alternative uses of resources; the second report states that MCDA does not solve the problem of estimating opportunity cost.
4. No deliberation. Only 10 of 35 reported a deliberative step.

Sources

• Baltussen R, Marsh K, Thokala P, Diaby V, Castro H, Cleemput I, et al. Multicriteria decision analysis to support health technology assessment agencies: benefits, limitations, and the way forward. Value in Health. 2019;22(11):1283-1288. https://doi.org/10.1016/j.jval.2019.06.014
• Campillo-Artero C, Puig-Junoy J, Culyer AJ. Does MCDA trump CEA? Applied Health Economics and Health Policy. 2018;16(2):147-151. https://doi.org/10.1007/s40258-018-0373-y
• Danner M, Hummel JM, Volz F, van Manen JG, Wiegard B, Dintsios CM, et al. Integrating patients' views into health technology assessment: analytic hierarchy process (AHP) as a method to elicit patient preferences. International Journal of Technology Assessment in Health Care. 2011;27(4):369-375. https://doi.org/10.1017/S0266462311000523
• Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and Value: Impact on DEcisionMaking, the EVIDEM framework and potential applications. BMC Health Services Research. 2008;8:270. https://doi.org/10.1186/1472-6963-8-270
• Institute for Quality and Efficiency in Health Care (IQWiG). General Methods. Version 8.0 of 19 December 2025. Cologne: IQWiG; 2025. https://doi.org/10.60584/General-Methods_V8.0
• Marsh K, IJzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. Multiple criteria decision analysis for health care decision making: emerging good practices. Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health. 2016;19(2):125-137. https://doi.org/10.1016/j.jval.2015.12.016
• Marsh K, Thokala P, Youngkong S, Chalkidou K. Incorporating MCDA into HTA: challenges and potential solutions, with a focus on lower income settings. Cost Effectiveness and Resource Allocation. 2018;16(Suppl 1):43. https://doi.org/10.1186/s12962-018-0125-8
• Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kaló Z, et al. Multiple criteria decision analysis for health care decision making: an introduction. Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health. 2016;19(1):1-13. https://doi.org/10.1016/j.jval.2015.12.003