D-Efficient Design

D-Efficient Design for Discrete Choice Experiments

A D-efficient design is the set of choice tasks that, for a stated choice model and a stated guess about respondents' preferences, makes the joint uncertainty of the estimated preference weights as small as the search allows. Among health studies published from 2013 to 2017 it had become the most common way to construct the choice sets for a discrete choice experiment, because well chosen tasks give tighter estimates of trade-offs and willingness to pay from the same number of patients or clinicians. The criterion is the determinant of the variance-covariance matrix of the model's parameters, and in a choice model that matrix depends on the preference values assumed before any data exist. A design that looks best under one set of assumed values can waste half its tasks under another, so the prior values matter as much as the search software.

Why a choice study needs an efficiency criterion

A discrete choice experiment describes each option by attributes such as effectiveness, side effects, waiting time and cost, and asks respondents to pick one option from each choice set. The full set of attribute combinations is usually far too large to show, so the analyst picks a subset. That subset fixes how much each answer reveals about the preference weights, and so the precision of every marginal rate of substitution and willingness-to-pay figure the study reports.

Soekhai and colleagues reviewed 301 health discrete choice experiments published from 2013 to 2017. D-efficient designs had become the most popular method of creating choice sets, used by 105 studies (35%), with 23 (8%) using a Bayesian D-efficient design, and Ngene was the most used design software (62 studies, 21%). The ISPOR task force report on constructing experimental designs, led by Johnson, does not endorse any single approach; it sets out the requirements for designs that identify choice model preference parameters and compares the available construction methods.

The D-error and what it measures

In a linear model the variances of the estimated coefficients are proportional to $(\mathbf X^{\mathsf T}\mathbf X)^{-1}$, where $\mathbf X$ is the coded design matrix, so they can be computed before data are collected. The NIST engineering statistics handbook describes D-optimal designs as maximising the determinant of the information matrix $\mathbf X^{\mathsf T}\mathbf X$, which minimises the generalised variance of the estimates for a pre-specified model. Kuhfeld scales the linear measure to run from 0 to 100.

$$\text{D-efficiency} = 100 \times \frac{1}{N_D \left|(\mathbf X^{\mathsf T}\mathbf X)^{-1}\right|^{1/p}}$$

where $N_D$ is the number of runs, $p$ is the number of parameters and $|\cdot|$ denotes the determinant, with $\mathbf X$ in standardised orthogonal contrast coding. A value of 100 means the design is balanced and orthogonal and 0 means at least one parameter cannot be estimated. Kuhfeld warns that the benchmark is a hypothetical orthogonal design that might not exist, so the values are best used to compare designs for the same problem.

A choice model is nonlinear. In the multinomial logit model the choice probabilities depend on the attributes and on the unknown preference vector $\boldsymbol\beta$, so the variance-covariance matrix depends on $\boldsymbol\beta$ too. Kuhfeld notes that the matrix cannot be made small without knowing $\boldsymbol\beta$, and if $\boldsymbol\beta$ were known the experiment would not be needed. The choice-model criterion therefore carries the parameters inside it.

$$\text{D-error}(\mathbf X,\boldsymbol\beta) = \left[\det \Omega(\mathbf X,\boldsymbol\beta)\right]^{1/K}, \qquad \Omega(\mathbf X,\boldsymbol\beta) = M(\mathbf X,\boldsymbol\beta)^{-1}$$

where $\mathbf X$ is the design (all attribute levels in all choice sets), $\boldsymbol\beta$ is the vector of assumed preference parameters, $\Omega$ is the asymptotic variance-covariance matrix of the maximum likelihood estimates, $M$ is the Fisher information matrix and $K$ is the number of parameters. A lower D-error means more precise joint estimation, and a D-efficient design is one with a low D-error among the feasible designs searched.

For tasks with two alternatives the information matrix takes a simple form, the two-alternative case of the multinomial logit variance formula given by Kuhfeld.

$$M(\mathbf X,\boldsymbol\beta) = \sum_{s=1}^{S} p_s (1-p_s)\, \mathbf d_s \mathbf d_s^{\mathsf T}, \qquad p_s = \frac{1}{1+\exp(-\boldsymbol\beta^{\mathsf T}\mathbf d_s)}$$

where $S$ is the number of choice sets, $\mathbf d_s$ is the difference between the coded attribute vectors of the first and second alternatives in set $s$, and $p_s$ is the predicted probability that the first alternative is chosen. The weight $p_s(1-p_s)$ is largest at $p_s = 0.5$ and falls towards zero as one option becomes a near-certain choice, so a choice set with a predictable answer adds almost nothing to the information matrix.

Prior values, local designs and Bayesian designs

Because the D-error needs a value of $\boldsymbol\beta$, every D-efficient design rests on an assumption about preferences. Setting $\boldsymbol\beta = \mathbf 0$ gives equal choice probabilities in every set. Kuhfeld describes designs built under this null hypothesis, and Alamri and colleagues review the work of Burgess and Street, who identified D-optimal choice designs under the assumption of equal choice probabilities.

A locally D-efficient design instead uses a point value for each parameter, for instance from earlier studies or a pilot. Talebi notes that even minimal prior information, such as the sign of each parameter, can improve design efficiency, while misestimated priors can leave a design less efficient than one built with zero priors. A Bayesian D-efficient design replaces the point value with a prior distribution and minimises the expected D-error.

$$D_B\text{-error}(\mathbf X) = \int \left[\det \Omega(\mathbf X,\boldsymbol\beta)\right]^{1/K} \pi(\boldsymbol\beta)\, d\boldsymbol\beta$$

where $\pi(\boldsymbol\beta)$ is the prior density and the other terms are as defined above. The systematic survey of simulation studies by Vanniyasingam and colleagues found that larger prior variances raised the $D_B$-error, that misspecified priors gave higher D-errors than correctly specified ones, and that underspecifying a heterogeneity prior lost more efficiency than overspecifying it.

Worked example with two attributes and four choice tasks

The numbers are illustrative and not taken from any survey. Patients choose between two treatment options described by chance of symptom relief and out-of-pocket cost, each high or low and effects coded as $+1$ for high and $-1$ for low, so $K = 2$. Each respondent answers four paired choice sets, and $\mathbf d_s$ lists the relief difference first and the cost difference second.

Design A uses two sets pairing high relief at high cost against low relief at low cost, $\mathbf d = (2, 2)$, and two sets pairing high relief at low cost against low relief at high cost, $\mathbf d = (2, -2)$. Design B uses one set in which only cost differs, $\mathbf d = (0, 2)$, one in which only relief differs, $\mathbf d = (2, 0)$, and two trade-off sets with $\mathbf d = (2, 2)$.

With zero priors every weight is 0.25. Design A's information matrix has 4 in each diagonal cell and 0 elsewhere, so $\det(M_A) = 16$ and its D-error is $\det(M_A)^{-1/2} = 0.25$. Design B's matrix has 3 on the diagonal and 2 off it, so $\det(M_B) = 3 \times 3 - 2 \times 2 = 5$ and its D-error is $\det(M_B)^{-1/2} \approx 0.447$. Of all ways to build four paired tasks from these four profiles, Design A has the largest determinant under zero priors.

Now suppose a pilot suggests illustrative priors of $+1$ for relief and $-1$ for cost. In Design A the trade-off sets have utility difference $\boldsymbol\beta^{\mathsf T}\mathbf d = 2 - 2 = 0$ and keep weight 0.25. The other two sets offer a dominant option, high relief at low cost, with utility difference $\boldsymbol\beta^{\mathsf T}\mathbf d = 2 + 2 = 4$, so it is chosen with probability $p = 1/(1 + e^{-4}) \approx 0.982$ and the weight falls to about 0.0177. Then $\det(M_A) = 64 \times 0.0177 \approx 1.130$ and the D-error rises to about 0.941.

In Design B the single-attribute sets have utility differences of $-2$ and $+2$, so the first option is chosen with probability about 0.119 in one and 0.881 in the other, both weights are about 0.105, and the trade-off sets keep weight 0.25. Then $\det(M_B) = 16 \times 0.105 \times 1.105 \approx 1.856$ and the D-error is about 0.734, the lowest any four-task arrangement of these profiles achieves under this prior.

The ranking has reversed. The relative D-efficiency of A against B is about 0.734 divided by 0.941, or 0.78. Because information grows in proportion to the number of respondents, the D-error falls in proportion to it, so Design A would need about 1.28 times as many respondents, roughly 28% more, to match Design B. Half of Design A's tasks ask a question whose answer the prior already predicts.

How the size of the task changes efficiency

Vanniyasingam and colleagues simulated 3,204 health discrete choice experiment designs with 2 to 20 choice tasks, 2 to 5 alternatives, 2 to 20 attributes and 2 to 5 levels. Relative D-efficiency fell as attributes and levels were added and rose with more alternatives, while adding choice tasks made it converge, though not always to 100%. Designs with binary attributes kept relative D-efficiencies above 80% except with 18 or more attributes. The NIST handbook adds that D-optimal methods suit cases where some factor combinations are infeasible, such as implausible treatment profiles.

Using a D-efficient design in a health preference study

The design is chosen after the attributes, levels and model are fixed, because the D-error is defined for one model. A design built for main effects may not support interactions added later, and one built for a multinomial logit is not automatically efficient for a mixed logit or latent class model. The precision it promises then feeds the sample size calculation.

For decision makers the gain is narrower confidence intervals around preference weights, which feed willingness-to-pay estimates for cost-benefit analysis and patient preference evidence for benefit-risk assessment. Soekhai and colleagues found that many studies presented sophisticated methods with insufficient detail, and 100 of the 301 studies (33%) did not clearly report how their choice sets were created. A transparent report states the model, coding, priors and their source, software, final D-error and constraints.

Boundaries with orthogonal, optimal and other efficient designs

An orthogonal design keeps attribute levels uncorrelated, and Kuhfeld notes that a balanced and orthogonal design, where one exists, has optimum efficiency for a linear model. A D-efficient choice design drops that requirement whenever another arrangement gives a lower D-error at the assumed priors, and the NIST handbook notes that D-optimal designs are usually not orthogonal and their estimates are correlated.

A D-optimal design is the best possible design for the criterion. The NIST handbook explains that computer algorithms build such designs by exchanging runs from a candidate set and warns that there is no guarantee the result is actually D-optimal, which is why applied designs are better described as D-efficient. A D-efficient design is one member of the wider family of efficient designs. Kuhfeld describes A-efficiency, based on the arithmetic mean of the variances, and notes that D-efficiency is easier to optimise and that the ratio of two D-efficiencies does not change with the coding scheme.

What a low D-error does not guarantee

A low D-error describes precision only under the assumed model and priors. If the priors are wrong, or a different model is fitted later, the realised precision can be much worse than the design report suggests. The criterion also says nothing about whether respondents understand the tasks or whether the attributes cover what matters to patients.

Two errors recur in practice. D-efficiency scores from different software are compared directly, although the scaling, coding and number of respondents behind them may differ. Designs are pushed so hard towards near-indifferent choice sets that the options become hard to tell apart, and the statistical gain then rests on answers of doubtful quality.

Sources

• Alamri AS, Georgiou S, Stylianou S. Discrete choice experiments: an overview on constructing D-optimal and near-optimal choice sets. Heliyon. 2023;9(7):e18256. doi:10.1016/j.heliyon.2023.e18256
• Johnson FR, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value in Health. 2013;16(1):3-13. doi:10.1016/j.jval.2012.08.2223
• Kuhfeld WF. Experimental design: efficiency, coding, and choice designs. In: Marketing Research Methods in SAS. Technical note MR-2010C. SAS Institute; 2010. https://support.sas.com/techsup/technote/mr2010c.pdf
• NIST/SEMATECH. e-Handbook of Statistical Methods, section 5.5.2.1 D-optimal designs. National Institute of Standards and Technology. Accessed 11 October 2026. https://www.itl.nist.gov/div898/handbook/pri/section5/pri521.htm
• Soekhai V, de Bekker-Grob EW, Ellis AR, Vass CM. Discrete choice experiments in health economics: past, present and future. PharmacoEconomics. 2019;37(2):201-226. doi:10.1007/s40273-018-0734-2
• Talebi A. Simulation in discrete choice models evaluation: SDCM, a simulation tool for performance evaluation of DCMs. arXiv. 2024 (version 3, 2026). arXiv:2407.17014
• Vanniyasingam T, Cunningham CE, Foster G, Thabane L. Simulation study to determine the impact of different design features on design efficiency in discrete choice experiments. BMJ Open. 2016;6(7):e011985. doi:10.1136/bmjopen-2016-011985
• Vanniyasingam T, Daly C, Jin X, et al. Investigating the impact of design characteristics on statistical efficiency within discrete choice experiments: a systematic survey. Contemporary Clinical Trials Communications. 2018;10:17-28. doi:10.1016/j.conctc.2018.01.002