Choice Model Estimate

Choice Model Estimate in Discrete Choice Experiments and Health Valuation

A fitted choice model returns a table of coefficients, but the numbers that reach a cost-benefit analysis or an EQ-5D value set are usually something else. They are ratios of coefficients, predicted choice probabilities or utility decrements rescaled onto the QALY scale, each computed from the coefficients and each carrying its own sampling uncertainty. Whether such a number means anything outside the study depends on a property that is easy to forget once it has been copied into a report. A logit coefficient is a preference weight divided by the unknown noise in respondents' choices, so it has no fixed unit of its own. The meaning covered here is the estimate produced from stated or revealed choices, mainly in discrete choice experiments, and not estimation in general.

Coefficients measured in units of noise

Train shows that the logit model fixes the scale of utility by fixing the variance of its unobserved part, so each estimated coefficient equals the underlying preference weight divided by a scale factor $\sigma$, as set out under the multinomial logit model. The scale factor drops out of any ratio of two coefficients from the same model.

$$\frac{\hat{\beta}_a}{\hat{\beta}_c} = \frac{\beta_a^{*}/\sigma}{\beta_c^{*}/\sigma} = \frac{\beta_a^{*}}{\beta_c^{*}}$$

where $\hat{\beta}_a$ and $\hat{\beta}_c$ are the estimated coefficients on attributes $a$ and $c$, $\beta_a^{*}$ and $\beta_c^{*}$ are the underlying preference weights, and $\sigma$ is the scale factor common to both. Ratios can therefore be compared across samples and studies in a way that raw coefficients cannot.

Kinds of estimate a choice model yields

Each output of a choice model answers a different question and behaves differently under rescaling. Whatever the model, the estimates fall into a small number of families.

The marginal rate of substitution and marginal willingness to pay are the ratio estimates. Of the derived outputs, only the predicted probability keeps the scale of the fitted data, which is why a coefficient set transferred to a new population with different choice consistency would forecast different shares even with identical trade-offs.

Uncertainty around a ratio of coefficients

A ratio of two normally distributed estimates is not itself normally distributed, and its spread grows sharply as the denominator approaches zero. The delta method approximates the variance of the ratio from the variances and covariance of the two coefficients. The Krinsky and Robb approach simulates many coefficient vectors from their estimated sampling distribution and computes the ratio for each draw, while the nonparametric bootstrap re-estimates the model on resampled respondents. Fieller's method is a fourth option that builds the interval for the ratio directly.

$$\text{Var}(\widehat{WTP}_a) \approx \frac{\text{Var}(\hat{\beta}_a)}{\hat{\beta}_c^{2}} + \frac{\hat{\beta}_a^{2}\,\text{Var}(\hat{\beta}_c)}{\hat{\beta}_c^{4}}-\frac{2\hat{\beta}_a\,\text{Cov}(\hat{\beta}_a,\hat{\beta}_c)}{\hat{\beta}_c^{3}}$$

where $\widehat{WTP}_a$ is the estimated marginal willingness to pay for attribute $a$, equal to $-\hat{\beta}_a/\hat{\beta}_c$, $\text{Var}(\hat{\beta}_a)$ and $\text{Var}(\hat{\beta}_c)$ are the sampling variances of the attribute and cost coefficients, and $\text{Cov}(\hat{\beta}_a,\hat{\beta}_c)$ is their covariance from the same fitted model.

In Hole's 2007 simulation comparison the four methods gave similar intervals in most scenarios. The covariance term matters in practice, because coefficients from the same model are usually correlated.

Predicted uptake and the aggregation problem

Choice models are also used to forecast how many patients would take up a new service, screening programme or treatment, an input to budget impact analysis and to implementation planning. The forecast is a predicted choice probability, and its aggregation needs care. Train explains that discrete choice models are not linear in the explanatory variables, so inserting average values of those variables into the model does not give an unbiased estimate of the average probability.

Train describes sample enumeration as the most common consistent method. Each sampled person's probability is computed from his or her own attribute values and characteristics, and the probabilities are then summed or averaged over a sample drawn from the population whose uptake is being forecast.

De Bekker-Grob and colleagues reported in 2019 that a DCE predicted real choices about influenza vaccination and colorectal cancer screening correctly at the aggregate level, provided the model accounted for scale and preference heterogeneity. They also found that the best-fitting models were not always the best predictors and warned about overfitting.

Anchoring estimates to the QALY scale for value sets

DCE coefficients sit on a latent utility scale with no natural zero, while a QALY weight must equal 1 for full health and 0 for dead. A choice model estimate becomes usable in a value set, such as an EQ-5D-5L value set, only after it has been anchored to that scale. Two broad routes exist.

The first builds survival time into the choice task, so that the anchor is estimated alongside the preferences. Bansback and colleagues set out a DCE of this kind for EQ-5D health states in 2012. In the Australian EORTC QLU-C10D study, King and colleagues modelled the utility of a life year with one coefficient and the utility weights of each dimension level per life year with a vector of coefficients, and computed utility decrements by dividing the level coefficients by the life-year coefficient.

$$V = \alpha T + (\beta' x)\,T \qquad u(x) = 1 + \frac{\beta' x}{\alpha}$$

where $V$ is the observed utility of living in a health state for duration $T$, $\alpha$ is the coefficient on a year in full health, $\beta$ is the vector of coefficients on the level dummies, $x$ is the vector of dummies describing the state, and $u(x)$ is the anchored utility of that state. Because $\beta/\alpha$ is a ratio, the anchored decrements share the scale-free property of willingness to pay.

The second route estimates latent values first and then rescales them. Rowen, Brazier and Van Hout compared four methods using postal DCE data from 263 respondents and interview time trade-off data from 307 respondents valuing the AQL-5D asthma measure. Methods that mapped DCE values onto time trade-off values, or combined both data types in a hybrid model, predicted mean observed time trade-off values better (mean absolute differences 0.052 to 0.084) than methods anchored on the DCE value for dead or on the time trade-off value for the worst state (0.075 to 0.093).

Illustrative worked example of a ratio, its interval and a forecast

All numbers are illustrative. A DCE on a community physiotherapy service includes waiting time in weeks and a one-off out-of-pocket cost in GBP. A conditional logit gives a waiting-time coefficient of minus 0.05 per week (standard error 0.010) and a cost coefficient of minus 0.002 per GBP (standard error 0.0004). For simplicity the covariance between the two estimates is taken as zero.

Step 1 (willingness to pay). Marginal willingness to pay for one extra week of waiting is $-(-0.05)/(-0.002) = -25$, so respondents would pay GBP 25 to wait one week less.

Step 2 (delta method variance). The first term is $\text{Var}(\hat{\beta}_a)/\hat{\beta}_c^{2} = 0.0001/0.000004 = 25$, using a variance of 0.0001 for the waiting coefficient. The second term is $\hat{\beta}_a^{2}\,\text{Var}(\hat{\beta}_c)/\hat{\beta}_c^{4} = 0.0025 \times 0.00000016 / 0.000000000016 = 25$, using a squared waiting coefficient of 0.0025 and a cost variance of 0.00000016. The variance of the willingness-to-pay estimate is therefore 25 + 25 = 50, and its standard error is the square root of 50, about 7.07.

Step 3 (interval). A 95 per cent interval is GBP 25 plus or minus 1.96 times 7.07, about GBP 11.14 to GBP 38.86.

Step 4 (probability at the average). A new clinic would cut waiting by 4 weeks and cost GBP 50 more than the current one. The utility difference is $\Delta V = 0.05 \times 4-0.002 \times 50 = 0.10$, and the binary logit probability of choosing the new clinic is $e^{0.10}/(1+e^{0.10})$, about 1.1052 divided by 2.1052, or 0.525.

Step 5 (sample enumeration). Suppose half the eligible population has a utility difference of 2.0 and half minus 1.8, which also average 0.10. Their probabilities are about 0.881 and 0.142, so the population share choosing the new clinic is about $(0.881 + 0.142)/2 = 0.5115$, close to 51 per cent and not the 52.5 per cent implied by the average person.

The willingness-to-pay interval is wide even with fairly precise coefficients, because both relative standard errors are 20 per cent and they compound in the ratio. A clinic costing GBP 50 more for a 4-week reduction offers a benefit worth about GBP 100 at the point estimate, but only about GBP 44.56 at the lower end of the interval, below the extra cost. The uptake forecast changes by more than a percentage point depending only on how the probabilities are aggregated, before any question of whether stated choices match real behaviour.

Boundaries with nearby quantities

A choice model estimate is defined by how it is obtained, which separates it from estimates that share a name. Willingness to pay from contingent valuation is asked for directly, while willingness to pay from a choice model is inferred from trade-offs and inherits the model's assumptions. A health state utility value may come from a choice model, but only after anchoring, and a latent DCE value used without anchoring is not a QALY weight.

Preference elicitation is the process that generates the choice data, and choice experiment design fixes which estimates the data can support with useful precision. When a choice model estimate enters a decision model as a parameter, its sampling distribution, including the correlation between coefficients, is what a probabilistic analysis should draw from.

Common misreadings of choice model estimates

Several errors recur in reports that take estimates out of their original study. Most of them come from treating a coefficient as if it had a fixed unit.

1. A model has measured scale heterogeneity as a separate finding. Hess and Train argue that this is not identified apart from other heterogeneity.
2. The size of a coefficient shows the importance of an attribute. A coefficient per unit depends on the units chosen, and importance depends on the range of levels offered as well.
3. Predicted uptake at the average respondent is the population uptake. The nonlinearity of the model makes the two differ, and the enumerated average is the consistent estimate.
4. Any DCE value can be used as a utility. Values need anchoring to full health and dead before they can weight life years.

Sources

• Train KE. Discrete Choice Methods with Simulation. 2nd ed. Cambridge: Cambridge University Press; 2009. Chapters 2 and 3. https://eml.berkeley.edu/books/train1201.pdf
• Hess S, Train K. Correlation and scale in mixed logit models. Journal of Choice Modelling. 2017;23:1-8. https://doi.org/10.1016/j.jocm.2017.03.001
• Hole AR. A comparison of approaches to estimating confidence intervals for willingness to pay measures. Health Economics. 2007;16(8):827-840. https://doi.org/10.1002/hec.1197
• Krinsky I, Robb AL. On approximating the statistical properties of elasticities. Review of Economics and Statistics. 1986;68(4):715-719. https://ideas.repec.org/a/tpr/restat/v68y1986i4p715-19.html
• de Bekker-Grob EW, Swait JD, Kassahun HT, et al. Are healthcare choices predictable? The impact of discrete choice experiment designs and models. Value in Health. 2019;22(9):1050-1062. https://www.valueinhealthjournal.com/article/S1098-3015(19)32147-3/fulltext
• Bansback N, Brazier J, Tsuchiya A, Anis A. Using a discrete choice experiment to estimate health state utility values. Journal of Health Economics. 2012;31:306-318. https://doi.org/10.1016/j.jhealeco.2011.11.004
• King MT, Viney R, Simon Pickard A, et al. Australian utility weights for the EORTC QLU-C10D, a multi-attribute utility instrument derived from the cancer-specific quality of life questionnaire, EORTC QLQ-C30. PharmacoEconomics. 2018;36:225-238. https://pmc.ncbi.nlm.nih.gov/articles/PMC5805814/
• Rowen D, Brazier J, Van Hout B. A comparison of methods for converting DCE values onto the full health-dead QALY scale. Medical Decision Making. 2015;35(3):328-340. https://doi.org/10.1177/0272989X14559542