Preference Elicitation

Preference Elicitation in Economic Evaluation and Regulatory Decisions

Most of the numbers in an economic evaluation that express value, as distinct from cost or clinical effect, come from people's answers to questions about what they would choose or give up. The utility weights in a value set, a willingness to pay figure in a cost-benefit analysis and a maximum acceptable risk in a regulator's benefit-risk review are all products of a preference elicitation study. The method chosen decides whose preferences count and what kind of question respondents answer, and different choices can give different values for the same outcome. NICE builds one such choice into the reference case of its technology appraisal manual (PMG36), asking for changes in health-related quality of life to be valued with public preferences from a representative sample of the UK population, using a choice-based method.

Why economic evaluation and regulation need elicited preferences

Health care outcomes are rarely bought and sold in markets that reveal their value, so analysts have to obtain values in other ways. In cost-utility analysis elicited values weight time in each health state to give QALYs, and health state valuation is the branch of preference elicitation that values health states on a scale where full health is 1 and dead is 0. In cost-benefit analysis elicited values put a money figure on outcomes, and multi-criteria decision analysis uses elicited weights to combine several criteria. Regulators also use patients' preferences to judge which balance of benefit and harm patients would accept. Preference elicitation is the general activity behind all of these uses, including attributes of care that are not health states, such as waiting time.

Whose preferences are elicited

The choice of respondents depends on the decision the values will inform. For the reference case of a cost-utility analysis, NICE asks for the preferences of a representative sample of the UK population, who are both the taxpayers funding the NHS and its potential patients.

Patients are the usual respondents when the question is which balance of benefit and risk is acceptable for people with a particular condition. The US Food and Drug Administration (FDA) defines patient preference information as qualitative or quantitative assessments of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among alternative health interventions. The FDA separates such information from patient-reported outcomes, which are measured in clinical trials and describe how patients feel and function, while preference information describes what patients value.

Stated and revealed preferences

Revealed preference methods infer value from choices people have actually made, but health care offers few settings where they work well, because insurance and public provision separate patients from prices and clinicians often choose on patients' behalf. Stated preference methods instead ask hypothetical questions, which can describe technologies that do not yet exist but may differ from what people would do with real money or real risks. In health economics almost all elicitation for QALY weights and most elicitation of willingness to pay is stated preference.

Choice tasks and matching tasks

Another axis of method choice is the kind of response asked for. A choice task presents two or more options and asks the respondent to pick one, or to rank them, or to pick the best and the worst. A matching task asks the respondent to state the quantity of one thing that makes two options equally attractive.

Choice tasks are easier to answer, but each answer carries less information, so answers from many respondents are modelled together to recover relative values on a latent scale, usually with the multinomial logit model or related models built on a choice experiment design. Matching tasks give a value for each respondent and each state directly, but they ask for a harder mental operation, and answers tend to cluster at round numbers.

The phrase choice-based is used more broadly in health state valuation, where it often groups the standard gamble and time trade-off with discrete choice tasks because all of them require the respondent to give something up, unlike a rating scale such as the visual analogue scale. NICE's 2013 methods guide (PMG9) used the term in that broad sense, describing the time trade-off behind the UK EQ-5D values as a choice-based method of valuation. PMG36 keeps the phrase choice-based method in section 4.3.4.

Matching the method to the decision

The decision context drives the choice of method. A value set for QALYs needs values for many health states on a scale anchored at full health and dead, which is why valuation studies for instruments such as the EQ-5D usually rest on matching tasks such as the time trade-off, sometimes combined with choice tasks. A regulator's question about acceptable risk needs patients' trade-offs between named benefits and harms, which a discrete choice experiment answers directly. A cost-benefit analysis needs money values, so a cost attribute or a willingness-to-pay question has to be part of the design.

ISPOR good practice reports and FDA guidance

Guidance on preference elicitation in health comes mainly from ISPOR task forces and, for medical devices, from the FDA. The ISPOR conjoint analysis checklist, reported by Bridges and colleagues in 2011, has ten items, from the research question, attributes and levels, construction of tasks and experimental design to preference elicitation, instrument design, the data collection plan, statistical analyses, results and conclusions, and study presentation. The authors say the checklist should not be read as endorsing any one methodological approach. Johnson and colleagues later reported on experimental design, and Hauber and colleagues, in 2016, on statistical analysis, working through a teaching example analysed with conditional logit before describing alternative methods and their strengths and weaknesses. A 2023 task force report, again led by Bridges, set out a roadmap for making patient-preference studies more useful to decision makers, built around five elements (context, purpose, population, method and impact) with questions that funders, reviewers and readers can use to judge a study.

The FDA issued guidance on 24 August 2016 on patient preference information submitted voluntarily in premarket approval applications, humanitarian device exemption applications and De Novo requests, and on its inclusion in decision summaries and device labelling. That guidance was superseded by the final guidance Incorporating Voluntary Patient Preference Information over the Total Product Life Cycle, announced in the Federal Register on 30 March 2026. The 2026 guidance extends the use of preference information across the product life cycle, including investigational device exemptions, Breakthrough Device designation requests and 510(k) submissions. The FDA states that methods for generating preference information may be qualitative, quantitative or mixed.

Illustrative worked example of a choice task and a matching task

All numbers in this example are illustrative and do not come from any published study. Patients with a chronic condition answer a discrete choice experiment about two treatments described by the chance of remission, the risk of a serious side effect and the monthly cost. A conditional logit model gives a coefficient of 0.45 for each 10 percentage points of remission, minus 0.30 for each percentage point of side-effect risk and minus 0.09 for each GBP 10 of monthly cost.

1. Utility difference between two options. Treatment A offers 20 percentage points more remission than treatment B, 2 percentage points more side-effect risk and GBP 20 more monthly cost. The difference in systematic utility is 0.90 minus 0.60 minus 0.18, which is 0.12, because 0.45 × 2 = 0.90, 0.30 × 2 = 0.60 and 0.09 × 2 = 0.18.

2. Predicted choice share.

$$P_A = \frac{e^{V_A}}{e^{V_A} + e^{V_B}} = \frac{1}{1 + e^{-(V_A-V_B)}}$$

where $P_A$ is the probability that a respondent chooses treatment A, $V_A$ and $V_B$ are the systematic utilities of the two treatments and $e$ is the base of natural logarithms. With a utility difference of 0.12 the predicted share is 1 ÷ (1 + 0.8869) = 0.530, so about 53% of respondents would be expected to choose treatment A.

3. Trade-offs between attributes.

$$\text{MRS}_{E,R} = -\frac{\beta_E}{\beta_R}$$

where $\text{MRS}_{E,R}$ is the marginal rate of substitution of side-effect risk for remission, $\beta_E$ is the coefficient for 10 percentage points of remission and $\beta_R$ is the coefficient for 1 percentage point of side-effect risk. Here 0.45 ÷ 0.30 = 1.5, so respondents would accept up to 1.5 percentage points of extra side-effect risk for each 10 percentage points of remission. Using the cost coefficient instead, 0.45 ÷ 0.09 = 5, and 5 × 10 = 50, a willingness to pay of GBP 50 a month for each 10 percentage points of remission.

4. A matching task for comparison. In a time trade-off, a member of the public judges 7 years in full health equal to 10 years in the condition, so the state's value is 7 ÷ 10 = 0.7 on the QALY scale.

The discrete choice coefficients show relative importance, and turning them into money or risk needs a numeraire attribute such as cost or side-effect risk. They do not give a QALY weight without an anchor to full health and dead. The time trade-off answer gives that weight directly, but from a single, harder question, and in this example from a different group of respondents.

Preference elicitation and expert elicitation

Expert elicitation uses similar interviews and questionnaires for a different purpose. It asks people with subject knowledge what they believe the value of an uncertain quantity to be, such as the five-year survival on a new treatment, and turns those beliefs into a probability distribution for a decision model. An ISPOR task force reported by Soares and colleagues in 2024 describes structured expert elicitation protocols as ways to limit the heuristics and biases that affect expert judgement, and reviews five commonly used protocols, among them the Sheffield elicitation framework and the MRC reference protocol.

What separates the two is the question asked, since the same person can answer either. A clinician asked for the probability of remission at five years is giving an expert judgement about a parameter. The same clinician asked how much side-effect risk is worth accepting for that remission is giving a preference.

Limitations and common misreadings

Elicited values depend on how the question is asked. Framing, the order of tasks and the help given by an interviewer can all move answers, and interviewer bias has been documented in time trade-off valuation studies.

Patient and public values are not interchangeable, because they answer different questions. Stated preferences are also not the same as behaviour, which is why revealed preference evidence is still sought where it exists.

Sources

• Brazier J, Ratcliffe J, Salomon JA, Tsuchiya A. Measuring and Valuing Health Benefits for Economic Evaluation. 2nd edition. Oxford: Oxford University Press; 2017. ISBN 9780198725923.
• Bridges JFP, Hauber AB, Marshall D, et al. Conjoint analysis applications in health, a checklist. A report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value in Health. 2011;14(4):403-413. doi:10.1016/j.jval.2010.11.013
• Bridges JFP, de Bekker-Grob EW, Hauber B, Heidenreich S, et al. A roadmap for increasing the usefulness and impact of patient-preference studies in decision making in health. A good practices report of an ISPOR task force. Value in Health. 2023;26(2):153-162. doi:10.1016/j.jval.2022.12.004
• Hauber AB, González JM, Groothuis-Oudshoorn CGM, Prior T, et al. Statistical methods for the analysis of discrete choice experiments. A report of the ISPOR Conjoint Analysis Good Research Practices Task Force. Value in Health. 2016;19(4):300-315. doi:10.1016/j.jval.2016.04.004
• Johnson FR, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments. Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value in Health. 2013;16(1):3-13. doi:10.1016/j.jval.2012.08.2223
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Section 4.2 (table 4.1) and section 4.3.4. https://www.nice.org.uk/process/pmg36/chapter/economic-evaluation-2
• National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013 (PMG9). 2013; replaced by PMG36. Section 5.3. https://www.nice.org.uk/process/pmg9/chapter/the-reference-case
• Soares M, Colson A, Bojke L, et al. Recommendations on the use of structured expert elicitation protocols for healthcare decision making. A good practices report of an ISPOR task force. Value in Health. 2024;27(11):1469-1478. doi:10.1016/j.jval.2024.07.027
• US Food and Drug Administration. Patient Preference Information, Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling. Guidance for industry, FDA staff and other stakeholders. 24 August 2016; superseded March 2026.
• US Food and Drug Administration. Incorporating Voluntary Patient Preference Information over the Total Product Life Cycle. Final guidance, March 2026; Federal Register notice 30 March 2026, 91 FR 15622, docket FDA-2015-D-1580. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/incorporating-voluntary-patient-preference-information-over-total-product-life-cycle