Measurement Scale Type

Measurement Scale Type in Health Outcome and Cost Data

Every number in an economic evaluation belongs to a level of measurement, and that level fixes which calculations give the same answer whatever arbitrary coding choices were made along the way. Stevens, then director of the Psycho-Acoustic Laboratory at Harvard University, set out four types in a 1946 paper in Science, nominal, ordinal, interval and ratio, and classed each by the transformations of its numbers that leave its information intact. A single trial dataset used for a cost-utility analysis usually holds all four, with treatment arm recorded as a category, EQ-5D responses recorded as ordered levels, health state utilities treated as interval values anchored at dead and full health, and costs and survival time recorded as ratio quantities. The level matters because QALY arithmetic multiplies a utility by a duration and then compares sums across treatments, which is only defensible if the numbers carry the properties those operations assume.

Stevens's four types and their permissible transformations

Stevens defined each type by its group of permissible transformations, meaning the changes to the numbers that preserve everything the scale records. A statement about the data is meaningful at a given level when its truth does not change under any permissible transformation. The types are nested, so each one keeps the properties of the weaker types and adds one more.

The interval row is the one that matters most for health utilities. Its permissible transformations take the form

$$x' = a + b x, \quad b > 0$$

where $x$ is the original value, $x'$ is the transformed value, $a$ is any constant that moves the origin and $b$ is a positive constant that changes the unit. A difference between two interval values keeps its sign and its relative size under this transformation, so comparing mean utility gains is meaningful, while a statement that one value is twice another is not, because adding $a$ changes the ratio. A ratio scale allows only the second part of the transformation, so $a$ must be zero and ratios are preserved. At the other end, a nominal scale allows any relabelling at all, and ordinal scaling allows any change that keeps the ranks, which is why questionnaire responses support medians but not means. Health state values are usually claimed to reach at least interval scaling, and whether they also reach the ratio level is argued in the QALY literature.

The four types inside one cost-utility analysis

A trial-based economic evaluation shows the four types side by side. Arm allocation is nominal, since swapping the codes for intervention and control changes nothing. A patient's answer on an EQ-5D dimension is ordinal, because the levels are ordered by severity but the numbers 1, 2 and 3 are labels for those levels rather than measured amounts. Devlin, Parkin and Janssen describe the conventional level labels as a categorical description, not a number.

A value set is the device that turns those ordinal profiles into numbers intended to carry interval information. Health state valuation methods such as the time trade-off and the standard gamble elicit preferences from a sample of the public, and the fitted model assigns a utility to every profile on a scale where full health is 1 and dead is 0. Torrance's review of utility measurement for economic appraisal covered the rating scale, standard gamble and time trade-off methods for states better and worse than death, so values below zero are part of the scale rather than errors.

Costs and survival time sit at the ratio level, since a cost of zero means no resource was used and a duration of zero means no time lived. The QALY multiplies a utility by a duration, and Prieto and Sacristán and later Roudijk, Donders and Stalmeier both argue that this product needs ratio properties on the utility scale as well as on the duration scale. They differ on whether conventional utilities have them. Prieto and Sacristán describe utility as an interval scale on which 0 is an arbitrary value for death and treat that as a defect of the QALY, while Roudijk and colleagues conclude that the health-utility scale has ratio properties with dead assigned 0. Where dead sits changes the QALY gain of any treatment that extends life, which makes the utility zero a ratio scale question.

Summing ordinal responses into a level sum score

Adding the codes of an EQ-5D profile into a level sum score treats ordinal response levels as if they were interval measurements. Devlin, Parkin and Janssen describe the score as treating each level's conventional label as if it were a number rather than simply a categorical description. They show the consequence with three very different EQ-5D-3L profiles, 22222, 33211 and 11233, all of which give a level sum score of 10. Adding the codes assumes that a step from level 1 to level 2 on one dimension is worth the same as the same step on any other, and that every step within a dimension is the same size. A value set does not make these assumptions, because it estimates a separate decrement for each step on each dimension from stated preferences.

Summed scores of this kind are common outside the EQ-5D too. Many disease-specific instruments add ordered item responses, often Likert-type items, into a total and then report mean totals, mean changes and standardised effect sizes, all of which are interval operations on ordinal inputs. Whether that is acceptable depends on evidence that the total behaves like an interval measure, not on the fact that the total looks like an ordinary number.

Worked example with an illustrative ordinal recoding

The numbers in this example are illustrative. A small trial records one EQ-5D-5L dimension for five patients in each arm, with level 1 meaning no problems and level 5 the most severe level. Arm A reports levels 1, 1, 1, 5 and 5, and arm B reports level 2 for all five patients.

Step 1 (conventional codes). With codes 1 to 5, the total for arm A is 1 + 1 + 1 + 5 + 5 = 13 and its mean is 13 / 5 = 2.6. The total for arm B is 2 + 2 + 2 + 2 + 2 = 10 and its mean is 10 / 5 = 2.0. Arm A looks worse.

Step 2 (another order-preserving code). The five levels are now coded 1, 4, 5, 6 and 7. Every level keeps its rank, so the recoding is a permissible ordinal transformation. The total for arm A is now 1 + 1 + 1 + 7 + 7 = 17 and its mean is 17 / 5 = 3.4. Every patient in arm B now scores 4, so its mean is 20 / 5 = 4.0. Arm B looks worse.

Step 3 (a statistic that respects the order). The median level is 1 for arm A and 2 for arm B under both codings, and under any other order-preserving coding.

The comparison of means reverses because the mean of ordinal codes depends on the spacing chosen for the levels, which the data do not supply. The medians do not move, which is what Stevens's framework predicts for an ordinal scale. Neither summary says which arm is better for health, since that depends on how much people value a step to level 5 compared with a step to level 2, and supplying that information is the job of a value set and the preference data behind it.

Rating scales and the interval assumption

The question of level also runs through the choice among valuation methods. A visual analogue scale and category rating ask respondents to place states on a line or in ordered categories between two anchors, and analysts who use the resulting values in QALYs treat the marks as interval measures of preference. Parkin and Devlin critically reviewed the theoretical and empirical propositions about visual analogue scale valuations and argued that there is a case for using them in cost-utility analysis. Brazier and McCabe published a response subtitled "yes there is a case, but what does it add to ordinal data?".

The exchange shows that the level of a set of utility values is a claim about the elicitation method, not a property of the numbers printed in a value set. Two sets of values can both run from 0 to 1 while one rests on choices involving risk or time and the other on positions on a line. Cardinal scaling is the requirement that differences between values reflect strength of preference, and each valuation method meets it with a different set of assumptions.

Boundaries with scaling methods and data types

Measurement scale type describes what the numbers on a scale record. Scaling methods are procedures for producing those numbers, so ratio scaling, in which respondents state one magnitude as a multiple of another, aims at a ratio scale without guaranteeing one. Ordinal, interval and cardinal scaling, in the health utility literature, name the properties that health state values are claimed to have.

Statistical data types cut across the levels in a different way. Categorical data cover both nominal and ordinal variables. Continuous data can be interval or ratio, and counts such as the number of admissions are discrete but sit on a ratio scale, since zero admissions is a true zero and four admissions are twice two. Classifying a variable as continuous therefore settles which distributions suit it, not which ratios are meaningful.

Limitations and common misreadings

Stevens's types are a classification of what measurement procedures justify, and assigning a level to a health economic variable involves judgement. The exchange between Parkin and Devlin and Brazier and McCabe on rating scales shows that experienced analysts can place the same values at different levels. Prieto and Sacristán and Roudijk and colleagues agree that QALYs need ratio-scale utilities and still reach different conclusions on whether conventional utilities have that property.

1. A value with decimals is interval data. A mean of ordinal codes has decimals but inherits none of the interval properties, as the worked example shows.
2. A value set makes the EQ-5D descriptive system cardinal. The value set makes the index values claim interval properties, while the profile itself stays ordinal, and summaries of the profile still have to respect that.
3. Negative utilities break the scale. Values below zero follow from the anchoring at dead and full health and from states that respondents judge worse than dead.
4. Level sum scores are a cheap substitute for utilities. A level sum score carries no preference weights, so it cannot enter a QALY calculation.

Sources

• Stevens SS. On the theory of scales of measurement. Science. 1946;103(2684):677-680. https://doi.org/10.1126/science.103.2684.677
• Devlin N, Parkin D, Janssen B. Methods for Analysing and Reporting EQ-5D Data. Cham: Springer; 2020. Chapter 2, Analysis of EQ-5D profiles. https://doi.org/10.1007/978-3-030-47622-9
• Torrance GW. Measurement of health state utilities for economic appraisal. Journal of Health Economics. 1986;5(1):1-30. https://doi.org/10.1016/0167-6296(86)90020-2
• Prieto L, Sacristán JA. Problems and solutions in calculating quality-adjusted life years (QALYs). Health and Quality of Life Outcomes. 2003;1:80. https://doi.org/10.1186/1477-7525-1-80
• Roudijk B, Donders ART, Stalmeier PFM. Setting dead at zero: applying scale properties to the QALY model. Medical Decision Making. 2018;38(6):627-634. https://doi.org/10.1177/0272989X18765184
• Parkin D, Devlin N. Is there a case for using visual analogue scale valuations in cost-utility analysis? Health Economics. 2006;15(7):653-664. https://doi.org/10.1002/hec.1086
• Brazier J, McCabe C. 'Is there a case for using visual analogue scale valuations in CUA' by Parkin and Devlin a response: 'yes there is a case, but what does it add to ordinal data?'. Health Economics. 2007;16(6):645-647. https://doi.org/10.1002/hec.1246