Maxdiff model probability of a best and worst pair in one choice set

Signature

P_max = exp(V_b - V_w) / (S_pos * S_neg - J)

Function

Best-worst scaling design, scoring and choice probability function

Computational function

• Computational function: maxdiff and sequential probabilities for every best and worst pair in a choice set

  Takes the utilities of the items in one choice set and returns the probability of every ordered best and worst pair under the maxdiff model (HE-FM-BWS-003) and under the sequential model (HE-FM-BWS-004), as two J by J matrices with zeros on the diagonal. The input is the whole vector of utilities rather than the summary sums the formulas take, and the function evaluates all J(J minus 1) pairs, which shows how far the two models disagree on the same utilities.

  Inputs and outputs: V: Utilities of the items in the set; required, vector of length J of at least 2. Unit: utility.; P_max: Matrix of maxdiff probabilities, best item in rows and worst in columns. Unit: probability.; P_seq: Matrix of sequential probabilities, same layout. Unit: probability.

  Assumption: Logit choice with a common error scale for the set; best and worst come from the same set; under the sequential model the worst choice is made from the items left after the best is removed.

  Worked example (Article's set 1 with utilities 1.0, 0.5, 0 and minus 0.5): The most likely pair under both models is A best and G worst, with probabilities of about 0.2540 under maxdiff and 0.2305 under the sequential model; the next is A best and E worst, 0.1541 and 0.1398; each matrix sums to 1. V = [1, 0.5, 0, -0.5]; P_max_AG = 0.2540; P_seq_AG = 0.2305; P_max_AE = 0.1541; P_seq_AE = 0.1398

  Worked example (Equal utilities in a set of three): Every ordered pair has probability 1 / 6 under both models (computed here for illustration). V = [0, 0, 0]; P_max_bw = 0.1667; P_seq_bw = 0.1667

  Excel: With the set's utilities in a column range named SetUtils, =EXP(SetUtils-TRANSPOSE(SetUtils))*(1-MUNIT(ROWS(SetUtils))) spills the maxdiff numerators, held in a range named MaxDiffNum, and =MaxDiffNum/SUM(MaxDiffNum) the maxdiff matrix; =EXP(SetUtils)/SUM(EXP(SetUtils))*EXP(-TRANSPOSE(SetUtils))/(SUM(EXP(-SetUtils))-EXP(-SetUtils))*(1-MUNIT(ROWS(SetUtils))) spills the sequential matrix.

  R: bw_probs <- function(V) { J <- length(V); N <- exp(outer(V, V, "-")); diag(N) <- 0; P_seq <- outer(1:J, 1:J, function(b, w) exp(V[b])/sum(exp(V))*exp(-V[w])/(sum(exp(-V))-exp(-V[b]))); diag(P_seq) <- 0; list(P_max = N/sum(N), P_seq = P_seq) } Returns 0.254022 and 0.230476 in row 1, column 4 for the first example.

  Python: def bw_probs(V): import math; J = len(V); D = sum(math.exp(V[j]-V[k]) for j in range(J) for k in range(J) if j != k); Sp = sum(math.exp(v) for v in V); Sn = sum(math.exp(-v) for v in V); return {"P_max": [[math.exp(V[b]-V[w])/D if b != w else 0 for w in range(J)] for b in range(J)], "P_seq": [[math.exp(V[b])/Sp*math.exp(-V[w])/(Sn-math.exp(-V[b])) if b != w else 0 for w in range(J)] for b in range(J)]} Returns the same matrices as the R function.

  Test (Each model's probabilities sum to 1 over all ordered pairs): Expected result: TRUE for both matrices. Excel check, with the matrices named MaxDiffMatrix and SeqMatrix: =AND(ABS(SUM(MaxDiffMatrix)-1)<1E-12,ABS(SUM(SeqMatrix)-1)<1E-12)

  Test (Equal utilities make the two models agree): With equal utilities both matrices hold 1 / (J(J minus 1)) off the diagonal. Expected result: TRUE. Excel check: =IF(MAX(SetUtils)=MIN(SetUtils),SUMPRODUCT(ABS(MaxDiffMatrix-SeqMatrix))<1E-12,TRUE)

  Common error (Treating best and worst as independent choices from the full set): Multiplying exp(V_b) / S_pos by exp(minus V_w) / S_neg, without removing the best item or the self-pairs, gives about 0.2071 for A best and G worst under either reading, and the probabilities of the 12 possible pairs sum to about 0.815 instead of 1 (computed here for illustration).

  Source: Mühlbacher AC, Zweifel P, Kaczynski A, Johnson FR. Experimental measurement of preferences in health care using best-worst scaling (BWS): theoretical and statistical issues. Health Economics Review. 2016;6:5. Equations 18 to 20.

  N_bw = exp(V_b - V_w) * (b != w); P_max_bw = N_bw / sum_(b) sum_(w) [N_bw]; P_seq_bw = exp(V_b) / sum_(j) [exp(V_j)] * exp(-V_w) / (sum_(k) [exp(-V_k)] - exp(-V_b)) * (b != w)

Try this function

Implementations

• Excel

  Maxdiff pair probability from a range of set utilities

  With the set's utilities in a column range named SetUtils and the chosen items' utilities in UtilBest and UtilWorst, the formula returns the probability, held in MaxDiffProb.

  =EXP(UtilBest-UtilWorst)/(SUMPRODUCT(EXP(SetUtils))*SUMPRODUCT(EXP(-SetUtils))-COUNT(SetUtils))

Assumptions

• Random utility with independent extreme value errors for the pair

  The utility difference of a pair is V_b minus V_w plus random error terms, and the errors give the logit form over ordered pairs. The utilities are on one scale for the set, with the error scale held constant.

• Best and worst chosen jointly from the same set

  The respondent chooses the pair in one step from all ordered pairs of distinct items; the sequential model (HE-FM-BWS-004) instead chooses the worst item after removing the best.

Worked examples

• Article's set 1 with A best and G worst

  With utilities of 1.0, 0.5, 0 and minus 0.5 for A, C, E and G, S_pos is about 5.9735 and S_neg about 3.6231, so the denominator is about 17.643 and the probability of A best and G worst about 0.254, as in the article.

  V_b = 1; V_w = -0.5; S_pos = 5.973534; S_neg = 3.623131; J = 4; P_max = 0.254

• Equal utilities in a set of four

  When all four utilities are equal every ordered pair is equally likely, 1 / 12 (computed here for illustration).

  V_b = 0; V_w = 0; S_pos = 4; S_neg = 4; J = 4; P_max = 0.0833

Common errors

• Leaving pairs of an item with itself in the maxdiff denominator

  Reading the double sum over all j and k literally, as some statements of the model print it, adds the J pairs of an item with itself, each equal to 1, to the denominator: 21.643 instead of 17.643 in the article's set, and a probability of about 0.2071 instead of 0.2540 (computed here for illustration). One item cannot be both best and worst in a task, so the probabilities of the J(J minus 1) possible pairs then sum to less than 1.

• Labelling any best-worst analysis as MaxDiff

  Hollin and colleagues call the MaxDiff label a potential misnomer, because it implies the maxdiff assumption, and found that most object case studies did not state their choice model. The model behind the estimates should be named.

Sources

• Maxdiff model over ordered best and worst pairs

  Mühlbacher AC, Zweifel P, Kaczynski A, Johnson FR. Experimental measurement of preferences in health care using best-worst scaling (BWS): theoretical and statistical issues. Health Economics Review. 2016;6:5. Equations 19 and 20: the maxdiff model assumes that respondents choose the best-worst pair out of all possible ordered pairs with the greatest utility difference, Delta U_bw = V_b minus V_w plus error terms, giving the joint probability as exp(V_b minus V_w) over a sum of exp(V_j minus V_k); in the discussion of the profile case, the model applies to pairs with x not equal to y. Equation 20 as printed sums over all j and k; the terms with j equal to k are excluded here so that the probabilities of the J(J minus 1) possible choices sum to 1.

  View source →

• MaxDiff label and unstated choice models in object case studies

  Hollin IL, Paskett J, Schuster ALR, Crossnohere NL, Bridges JFP. Best-worst scaling and the prioritization of objects in health: a systematic review. PharmacoEconomics. 2022;40(9):883-899. Discussion: MaxDiff is a potential misnomer because it implies an assumption about how respondents choose; most studies did not state their choice model.

  View source →

Canonical Identity