Swing-weighted additive value of a technology on three MCDA criteria

Signature

w_1 = p_1 / (p_1 + p_2 + p_3); w_2 = p_2 / (p_1 + p_2 + p_3); w_3 = p_3 / (p_1 + p_2 + p_3); V = w_1 * v_1 + w_2 * v_2 + w_3 * v_3

Function

Value measurement function for MCDA in HTA from criterion performance, ranges and swing points

Computational function

• Computational function for MCDA totals, ranking and a health gain weight scan for three technologies

  Takes the health gain of three technologies with the fixed range of that criterion, their scores on two further criteria and the swing points of the three criteria, and returns each technology's health gain score by HE-FM-MCDH-001, the weights and overall values by HE-FM-MCDH-002, and the highest overall value. Its inputs are raw performance on the first criterion rather than the partial values of HE-FM-MCDH-002. The R and Python code also rank the technologies, scan the health gain weight on a grid of 101 values, 0, 0.01 and so on up to 1, with the other two weights kept in the ratio of their points, and return, for each pair of technologies, the weight at which their overall values are equal. Each total is linear in that weight, so this weight takes the form of the switching weight w_s of HE-FM-WE-004, which that record computes only for the QALYs of two groups, here with the other-criteria blend in place of group A QALYs and the health gain score less that blend in place of group B QALYs (derived). It marks a change of ranking only when it is at least 0 and at most 1; a value below 0 or above 1 means the pair keeps the same order at every feasible weight, and a pair with equal slopes either never ties or, with equal totals, ties at every weight (derived). Equal overall values are reported as ties, with no rule to break them.

  Inputs and outputs: x_A, x_B and x_C are the health gains of technologies A, B and C in QALYs per patient; x_worst and x_best are the ends of the fixed health gain range; s_A, s_B and s_C are the severity scores and e_A, e_B and e_C the strength of evidence scores, each on a 0 to 100 scale; p_1, p_2 and p_3 are the swing points of health gain, severity and evidence; v_A, v_B and v_C are the health gain scores; w_1, w_2 and w_3 are the weights; V_A, V_B and V_C are the overall values on a 0 to 100 scale and V_top is the highest of them.

  Assumption: The assumptions of HE-FM-MCDH-001 and HE-FM-MCDH-002 hold, with a linear partial value function on health gain and the scores on the other two criteria already on their 0 to 100 scales. The weight scan keeps the ratio of the severity and evidence points fixed, as the article's stability check does.

  Worked example (Article technologies at swing points of 100, 60 and 40): With the article's illustrative inputs the health gain scores are 75, 50 and 0, the weights 0.5, 0.3 and 0.2 and the overall values 72.5, 65 and 50, so A ranks first. The scan finds that every pair ties at a health gain weight of 2/7, about 0.2857, with an overall value of about 71.43, and that C ranks first at every grid weight up to 0.28 and A from 0.29 (derived). x_A = 0.6; x_B = 0.4; x_C = 0; x_worst = 0; x_best = 0.8; s_A = 50; s_B = 100; s_C = 100; e_A = 100; e_B = 50; e_C = 100; p_1 = 100; p_2 = 60; p_3 = 40; v_A = 75; v_B = 50; v_C = 0; w_1 = 0.5; w_2 = 0.3; w_3 = 0.2; V_A = 72.5; V_B = 65; V_C = 50; V_top = 72.5

  Worked example (Health gain weight of 0.25): Points of 25, 45 and 30 keep severity and evidence in the ratio 60 to 40 and give C 75, B 72.5 and A 71.25, so C, with no health gain, ranks first. x_A = 0.6; x_B = 0.4; x_C = 0; x_worst = 0; x_best = 0.8; s_A = 50; s_B = 100; s_C = 100; e_A = 100; e_B = 50; e_C = 100; p_1 = 25; p_2 = 45; p_3 = 30; v_A = 75; v_B = 50; v_C = 0; w_1 = 0.25; w_2 = 0.45; w_3 = 0.3; V_A = 71.25; V_B = 72.5; V_C = 75; V_top = 75

  Excel: With health gains in cells named HealthGainTechA, HealthGainTechB and HealthGainTechC, the range ends in HealthGainWorst and HealthGainBest, and points in SwingPointsHealth, SwingPointsSeverity and SwingPointsEvidence (a row named SwingPointsRow), =100*(HealthGainTechA-HealthGainWorst)/(HealthGainBest-HealthGainWorst) returns HealthScoreTechA. With that score beside SeverityScoreTechA and EvidenceScoreTechA in a row named ScoresTechA, =SUMPRODUCT(SwingPointsRow,ScoresTechA)/SUM(SwingPointsRow) returns CFValueTechA, and B and C follow in the same way; =MAX(CFValueTechA,CFValueTechB,CFValueTechC) returns TopValue. =(SwingPointsSeverity*SeverityScoreTechA+SwingPointsEvidence*EvidenceScoreTechA)/(SwingPointsSeverity+SwingPointsEvidence) returns OtherBlendTechA, and =(OtherBlendTechC-OtherBlendTechA)/((HealthScoreTechA-OtherBlendTechA)-(HealthScoreTechC-OtherBlendTechC)) returns SwitchWeightAC, the health gain weight at which A and C tie, or #DIV/0! when their slopes are equal. For the scan, a helper column named WeightGrid holds the weights 0, 0.01 and so on up to 1, and beside it =WeightGrid*HealthScoreTechA+(1-WeightGrid)*OtherBlendTechA filled down gives technology A's overall value at each weight, with one such column per technology.

  R: mcda3 <- function(x, s, e, lo, hi, p) { v <- 100*(x-lo)/(hi-lo); w <- p/sum(p); V <- w[1]*v+w[2]*s+w[3]*e; o <- (p[2]*s+p[3]*e)/(p[2]+p[3]); g <- seq(0, 1, by = 0.01); top <- sapply(g, function(a) which.max(a*v+(1-a)*o)); tie <- outer(1:3, 1:3, function(i, j) (o[j]-o[i])/((v[i]-o[i])-(v[j]-o[j]))); list(v = v, w = w, V = V, rank = rank(-V), top = top, tie = tie) } Called as mcda3(c(0.6, 0.4, 0), c(50, 100, 100), c(100, 50, 100), 0, 0.8, c(100, 60, 40)) it returns v = 75, 50, 0, V = 72.5, 65, 50, ranks 1, 2 and 3, tie weights of 0.2857 for every pair (NaN on the diagonal, and Inf or NaN for a pair with equal slopes) and technology 3 (C) first at the grid weights up to 0.28 and technology 1 (A) from 0.29; on an exact tie which.max returns the first technology listed.

  Python: def mcda3(x, s, e, lo, hi, p): v = [100*(xi-lo)/(hi-lo) for xi in x]; w = [pj/sum(p) for pj in p]; V = [w[0]*v[i]+w[1]*s[i]+w[2]*e[i] for i in range(3)]; o = [(p[1]*s[i]+p[2]*e[i])/(p[1]+p[2]) for i in range(3)]; tie = {(i, j): ((o[j]-o[i])/((v[i]-o[i])-(v[j]-o[j])) if (v[i]-o[i]) != (v[j]-o[j]) else None) for i in range(3) for j in range(i+1, 3)}; top = [max(range(3), key=lambda i: g/100*v[i]+(1-g/100)*o[i]) for g in range(101)]; return v, w, V, tie, top Called as mcda3([0.6, 0.4, 0], [50, 100, 100], [100, 50, 100], 0, 0.8, [100, 60, 40]) it returns v = [75.0, 50.0, 0.0], V = [72.5, 65.0, 50.0], tie weights of 0.2857 for the pairs (0, 1), (0, 2) and (1, 2) (None for a pair with equal slopes), and index 2 (C) first at grid steps 0 to 28 and index 0 (A) from step 29.

  Test (Article technologies): With the article's inputs, CFValueTechA equals 72.5. Expected result TRUE. A health gain score computed on the local range 0 to 0.6 makes the cell 85 and gives FALSE, raw points used without division make it 14,500 and give FALSE, and doubling or halving the cell also gives FALSE. Excel check =ABS(CFValueTechA-72.5)<1E-6

  Test (Tie weight of A and C): With the article's inputs, SwitchWeightAC equals 2/7. Expected result TRUE. A blend that averages severity and evidence equally, ignoring the 60 to 40 ratio, makes A's blend 75 against C's 100 and the cell 0.25, which gives FALSE, and doubling or halving the cell also gives FALSE. Excel check =ABS(SwitchWeightAC-2/7)<1E-6

  Common error (Ranking at the elicited weights only): Reporting only the ranking at the elicited health gain weight of 0.50 hides that C, with no health gain, ranks first at every health gain weight below 2/7, about 0.29 (derived from the article's example). The second ISPOR report states that, at a minimum, deterministic analysis should be undertaken.

  Source: Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making: an introduction. Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health. 2016;19(1):1-13. Appendix, MCDA Modeling Example, Scoring, Weighting and Aggregation (Table A2). Marsh K, IJzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. Multiple criteria decision analysis for health care decision making: emerging good practices. Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health. 2016;19(2):125-137. Step 7b, Report and justify the uncertainty analysis, which states that at a minimum deterministic analysis should be undertaken.

  v_A = 100 * (x_A - x_worst) / (x_best - x_worst); v_B = 100 * (x_B - x_worst) / (x_best - x_worst); v_C = 100 * (x_C - x_worst) / (x_best - x_worst); w_1 = p_1 / (p_1 + p_2 + p_3); w_2 = p_2 / (p_1 + p_2 + p_3); w_3 = p_3 / (p_1 + p_2 + p_3); V_A = w_1 * v_A + w_2 * s_A + w_3 * e_A; V_B = w_1 * v_B + w_2 * s_B + w_3 * e_B; V_C = w_1 * v_C + w_2 * s_C + w_3 * e_C; V_top = max(max(V_A, V_B), V_C)

Try this function

Implementations

• Excel

  Excel swing weights and additive overall value for three MCDA criteria

  Swing points go in cells named SwingPointsFirst, SwingPointsSecond and SwingPointsThird and the partial value scores in ValueFirstCriterion, ValueSecondCriterion and ValueThirdCriterion. The first three formulas return the weights in cells named WeightFirstCriterion, WeightSecondCriterion and WeightThirdCriterion and the last returns the overall value in a cell named OverallValue; for the article's technology A they return 0.5, 0.3, 0.2 and 72.5. With many criteria, =SUMPRODUCT(SwingPointsRow,PartialValueRow)/SUM(SwingPointsRow) gives the same total from a row of points and a row of scores of equal length.

  =SwingPointsFirst/(SwingPointsFirst+SwingPointsSecond+SwingPointsThird); =SwingPointsSecond/(SwingPointsFirst+SwingPointsSecond+SwingPointsThird); =SwingPointsThird/(SwingPointsFirst+SwingPointsSecond+SwingPointsThird); =WeightFirstCriterion*ValueFirstCriterion+WeightSecondCriterion*ValueSecondCriterion+WeightThirdCriterion*ValueThirdCriterion

Assumptions

• Preference independence of criteria in the additive MCDA value

  How much stakeholders care about performance on one criterion does not depend on performance on the others, the preference independence that the second ISPOR report requires of an additive model. It names separate criteria for health gain and severity as a likely failure, since the preference for a gain in health may depend on baseline health, and warns that an additive model can then give a positive value to an alternative that generates no health gain.

• Swing points judged over the ranges of the partial value scores

  Each criterion's points value the swing over the same range on which its partial value scores run, so the weights are scaling constants showing the rate at which changes in one criterion compensate changes in another. The second ISPOR report finds this condition best met by swing weighting and decompositional methods, and notes that direct rating tends to give flatter weights.

• Positive total of swing points in the MCDA weights

  The points are zero or above with a positive total. Only their ratios matter, because multiplying every point by the same constant leaves each weight unchanged (derived), so the 100-point anchor of the elicitation need not appear among the inputs.

• Non-overlapping MCDA criteria without double counting

  No two criteria measure the same consequence. The second ISPOR report gives cost-effectiveness included as a criterion alongside cost or effectiveness criteria as an example of double counting, which would enter one consequence twice in V.

Worked examples

• Overall value of technology A in the MCDA in HTA example

  The article's illustrative swing points are 100 for health gain, 60 for severity and 40 for strength of evidence, a total of 200, so the weights are 0.50, 0.30 and 0.20. Technology A scores 75 on health gain, 50 for a moderate condition and 100 for high-strength evidence, an overall value of 37.5 plus 15 plus 20, or 72.5. With the same weights technology B (50, 100 and 50) gives 65 and technology C (0, 100 and 100) gives 50.

  p_1 = 100; p_2 = 60; p_3 = 40; w_1 = 0.5; w_2 = 0.3; w_3 = 0.2; v_1 = 75; v_2 = 50; v_3 = 100; V = 72.5

• Technology C first at a health gain weight of 0.25 in the MCDA example

  In the article's stability check the health gain weight falls to 0.25 while severity and evidence share the rest in the ratio 60 to 40, weights of 0.25, 0.45 and 0.30, entered here as points of 25, 45 and 30. Technology C, with no health gain, then scores 75, ahead of B at 72.5 and A at 71.25 computed in the same way.

  p_1 = 25; p_2 = 45; p_3 = 30; w_1 = 0.25; w_2 = 0.45; w_3 = 0.3; v_1 = 0; v_2 = 100; v_3 = 100; V = 75

• Three-way tie of the MCDA example at a health gain weight of two sevenths

  Points of 2, 3 and 2 give weights of 2/7, 3/7 and 2/7, a health gain weight of about 0.29 with severity and evidence still in the ratio 60 to 40. Technology A then scores 500/7, about 71.43, and B (50, 100 and 50) and C (0, 100 and 100) score the same, the tie that the article reports at w = 2/7 (checked here with exact fractions). Below this weight C ranks first.

  p_1 = 2; p_2 = 3; p_3 = 2; w_1 = 0.2857; w_2 = 0.4286; w_3 = 0.2857; v_1 = 75; v_2 = 50; v_3 = 100; V = 71.4286

• ISPOR MCDA appendix alternative 1 with unrounded swing weights

  The first ISPOR task force appendix gives 60, 100 and 80 points to its criteria A, B and C, and its alternative 1 scores 80, 40 and 65. Unrounded weights of 0.25, 0.4167 and 0.3333 give an overall value of 58.33; the appendix rounds the weights to 0.25, 0.42 and 0.33 and prints 58.25. Its alternative 2 (32, 70 and 55) gives 55.5 against the printed 55.55, so alternative 1 ranks first either way.

  p_1 = 60; p_2 = 100; p_3 = 80; w_1 = 0.25; w_2 = 0.4167; w_3 = 0.3333; v_1 = 80; v_2 = 40; v_3 = 65; V = 58.3333

Common errors

• Additive MCDA value for a technology with no health gain

  Technology C in the article has no health gain and still scores 50, because severity and strength of evidence earn value on their own, and it ranks first whenever the health gain weight is below 2/7, scoring 75 against A's 71.25 at a weight of 0.25. The second ISPOR report describes this counterintuitive result as the effect of an additive model when health gain and severity interact.

• Changing an MCDA criterion range without new swing points

  Widening the health gain range to 0 to 1.6 QALYs halves the health gain scores to 37.5 for A and 25 for B. Kept at 0.50, the old weight then gives A 53.75, B 52.5 and C 50 in place of 72.5, 65 and 50 (computed here for illustration), although a swing of 1.6 QALYs would be judged worth more points than a swing of 0.8. The second ISPOR report ties weights that act as scaling constants to elicitation over the range of performance, so a new range needs new points.

• Cost per MCDA value point read as relative efficiency

  Dividing cost by overall value, GBP 18,000 by 72.5 for A and GBP 8,000 by 65 for B, gives about GBP 248 and GBP 123 per point, the ratio form of HE-FM-ACER-001. The totals are on an interval scale. Adding 20 to every partial value, as if the bottom of each range scored 20, adds 20 to every total because the weights sum to one (derived), and gives GBP 194.59 and GBP 94.12 per point, so the ratio of the two figures becomes 2.07 in place of 2.02 with no change in either technology (computed here for illustration). The second ISPOR report states that a cost-benefit ratio needs benefits measured on a ratio scale.

Sources

• ISPOR MCDA task force report 2 on the additive function and its conditions

  Marsh K, IJzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. Multiple criteria decision analysis for health care decision making: emerging good practices. Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health. 2016;19(2):125-137. Step 6a, Report and justify the aggregation function used, names the additive model as the most commonly applied aggregation formula in health care MCDAs and gives the unnumbered additive function V_j = sum_(i=1)^n s_ij w_i, with V_j the overall value of intervention j, s_ij its score on criterion i and w_i the weight of criterion i, noting that additive functions are also applied in health-related quality of life instruments such as the EQ-5D and SF-6D. Step 2a lists preference independence among the properties of criteria, with health gain and severity as an example of its failure, and nonoverlap, with cost-effectiveness alongside cost or effectiveness as double counting. Step 5a ties scaling constants to elicitation that takes account of the range of performance, best met by swing weighting and decompositional approaches, with direct rating giving flatter weights. Dealing with Budget Constraints states that a ratio scale is needed to measure benefits in a cost-benefit ratio.

  View source →

• ISPOR MCDA task force report 1 appendix on swing weighting and aggregation

  Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making: an introduction. Report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health. 2016;19(1):1-13. MCDA Modeling Approaches states that value measurement approaches are by far the most widely used in health care, and Calculating Aggregate Scores that they often use the additive model. Appendix, MCDA Modeling Example, Weighting, assigns 100 points to the criterion whose swing (range of performance) matters most, rates each other criterion against it, and normalises the points by dividing each criterion's points by the sum of points so that the weights add up to 1 (60, 100 and 80 points give 0.25, 0.42 and 0.33); Aggregation multiplies each score by its criterion's weight and sums across criteria, with totals of 58.25 and 55.55 in Table A2.

  View source →

• Belton and Stewart textbook chapter on value function methods for MCDA

  Belton V, Stewart TJ. Multiple criteria decision analysis: an integrated approach. Kluwer Academic Publishers; 2002. Chapter 5, Value Function Methods: Practical Basics (pages 119 to 161), the standard textbook account of multiattribute value theory (MAVT) used in practice for multi-criteria decision support.

  View source →

Canonical Identity