Convergent Validity

Convergent Validity of Health Measures and Utility Instruments

A health measure gains support for its intended meaning when its scores rise and fall with instruments that tap the same aspect of health. In health economics this matters most for preference-based instruments such as the EQ-5D, SF-6D and HUI3, whose utility values become the quality weights in a QALY, and weak convergence is one of the grounds for arguing that an instrument does not suit a patient group. Convergent validity is one part of construct validity. Its hypotheses are written and judged under the COSMIN methodology, and the evidence is used when choosing a utility instrument and when mapping to EQ-5D. An illustrative example tests six hypotheses for EQ-5D in a hearing-loss sample and then shows why two instruments can correlate closely yet give different QALY gains.

Convergent evidence within the COSMIN taxonomy

The COSMIN consensus on measurement properties for patient-reported outcomes, published by Mokkink and colleagues in 2010, frames where convergent evidence sits. In that taxonomy, construct validity contains structural validity, hypotheses testing and cross-cultural validity, and it is defined as the degree to which scores are consistent with hypotheses about, for instance, relationships with other instruments or differences between relevant groups. Convergent validity is not a separate measurement property in this scheme. The COSMIN user manual for systematic reviews splits its hypotheses-testing checklist into part 9a, comparison with other instruments, labelled convergent validity, and part 9b, comparison between subgroups, labelled discriminative or known-groups validity.

The line between convergent and criterion validity depends on whether a gold standard exists. The manual states that, in principle, no gold standard exists for a patient-reported outcome measure, the exception being the long version of an instrument when its short form is tested. Correlating an EQ-5D index with an SF-6D index is therefore convergent evidence, since neither is a gold standard.

The multitrait-multimethod logic

Testing convergence and divergence together comes from psychology. In a 1959 paper, Donald Campbell and Donald Fiske proposed a matrix of intercorrelations among tests of at least two traits, each measured by at least two methods, in which measures of the same trait should correlate more highly with each other than with measures of different traits taken by separate methods.

The authors later recalled that, in the projects behind the paper, different traits measured by the same method regularly correlated more highly than the same trait measured by different methods, a pattern they called method variance. Two self-completed questionnaires can therefore correlate partly because they share a format and a respondent. The SF-6D is derived from the SF-36, so a correlation between the SF-6D and another SF-36 score contains overlap by construction.

Writing convergent hypotheses before the data are seen

A convergent correlation means little unless its expected direction and size were stated in advance. The COSMIN manual asks for both, because otherwise it is hard to decide afterwards whether a result agrees with the hypothesis. As a starting point it reproduces generic hypotheses from de Vet and colleagues.

These are prompts, not universal pass marks. The COSMIN standards for part 9a also ask whether it is clear what the comparator measures and whether its measurement properties are sufficient in a similar population. The manual states that p-values should not be used, because the question is whether the direction and size of a correlation match the constructs, not whether it differs from zero.

When results across studies are summarised in a review, the manual rates hypotheses testing as sufficient when at least 75% of results agree with the hypotheses and insufficient when at least 75% do not. It adds that correlations with instruments measuring similar constructs can be treated as stronger evidence, so a review team may weight them more heavily when results are inconsistent.

Measuring the association

Most convergent evidence is a correlation coefficient between two scores from the same people at the same time. The choice of coefficient follows the distribution of the scores. The Pearson correlation measures how far two approximately continuous scores move together linearly, while the rank-based Spearman correlation is the usual alternative for ordinal or heavily skewed scores.

The COSMIN manual also accepts factor-analytic or structural equation models across several scales. Sign conventions need care, because a utility index rises with better health while many condition-specific scores rise with worse health, so a negative correlation can be exactly what the hypothesis predicts.

Convergent evidence alongside discriminant and known-groups tests

Convergent evidence is one of several construct-validity tests that are usually run together. The NICE Decision Support Unit's Technical Support Document 8 names two common tests for utility instruments, namely whether the measure differentiates between groups thought to differ in health and how far it correlates with another measure of health.

Convergent and discriminant evidence are most informative as a pair, since a correlation of 0.55 with a similar instrument says more when correlations with measures of distinct constructs are clearly lower. Known-groups evidence asks a different question, whether scores separate groups defined independently as differing in health.

Convergent validity in choosing a utility instrument for HTA

In health technology assessment the choice of utility instrument changes the QALYs, so its validity in the patient group is a decision question. NICE's manual (PMG36, section 4.3) states that different methods produce different utility values, prefers the EQ-5D for adults for consistency across evaluations, and sets the test for arguing that the EQ-5D is inappropriate. That test requires qualitative evidence that it lacks content validity, supported by evidence that it performs poorly on tests of construct validity and responsiveness in the population, drawn from a synthesis of peer-reviewed literature.

Technical Support Document 8 asks whether the EQ-5D and its relevant dimensions fail to converge with other measures of the same concept, and states that convergence can be assessed against another generic preference-based measure such as the HUI3 or SF-6D. It notes that validation is best done at dimension level, that clinical comparators may relate only weakly to health-related quality of life, and that convergence with directly elicited time trade-off or standard gamble values is limited because those reflect patient rather than general population values.

The evidence differs by condition. A 2014 review of measures used in NICE decision-making by Longworth and colleagues found the EQ-5D valid and responsive for skin conditions and most cancers, variable by cause in visual disorders and poor for hearing impairments, while the HUI3 performed well for hearing and vision disorders.

Convergence and mapping to EQ-5D

When EQ-5D data are missing, a mapping function can predict EQ-5D values from another measure, which PMG36 treats as a departure from the reference case; the manual asks for data sets containing both measures, a justified choice of function, evidence of fit and sensitivity analyses across algorithms. The Decision Support Unit's Technical Support Document 22, which replaces TSD 10, states that mapping needs a plausible relationship between the source measure and EQ-5D, called conceptual overlap.

Where overlap is not obvious, as with visual acuity, TSD 22 suggests adding empirical evidence on the psychometric properties of the EQ-5D, and it states that mapping is not appropriate if the EQ-5D is itself inappropriate for the condition. Convergent evidence supports the case for mapping, but a strong correlation does not by itself validate a mapping function. Judging the function is a question of fit, and TSD 22 notes that summary fit statistics such as mean absolute error and root mean squared error are relatively insensitive and can conceal systematic prediction errors, so it recommends plots of predicted against observed means across the range of severity.

Worked example with EQ-5D in an illustrative hearing-loss sample

The figures below are illustrative and do not come from any study. An analyst is testing whether the EQ-5D-3L index behaves as expected in 250 adults with hearing loss and writes six hypotheses before analysis. The utility indices rise with better health, while the hearing questionnaire and fatigue scale rise with worse health.

Testing the hypotheses

Each observed correlation is compared with its prespecified range and recorded as confirmed or not confirmed. No p-values are used.

1. Count the results. Four of six hypotheses are confirmed, 4 / 6 = 0.667, or 67%. By analogy with COSMIN's review-level rule, neither the 75% threshold for sufficient nor the 75% threshold for insufficient is met, so the result would be rated inconsistent (±). The H6 gap is 0.58 − 0.36 = 0.22. H3 could instead be classed as a related but dissimilar comparison (0.30 to 0.50), and the observed $r = -0.24$ fails either way.

2. Look at which hypotheses failed. Both failures are convergent, involving the comparators closest to hearing, namely a hearing-specific questionnaire and an instrument the 2014 review found performed well in hearing disorders. Weighting these more heavily, as COSMIN suggests, points to weak convergence where it matters most.

3. Interpret for a submission. One study cannot meet the PMG36 test, which needs content-validity evidence and a literature synthesis, but it feeds that synthesis and weakens the case for mapping a hearing-specific measure to EQ-5D.

Correlation is not agreement

A second check shows why convergence alone cannot settle which instrument to use. Five illustrative patients complete the EQ-5D (X), scoring 0.20, 0.45, 0.60, 0.80 and 0.95, and the SF-6D (Y), scoring 0.52, 0.60, 0.65, 0.73 and 0.85. The means are 3.00 / 5 = 0.60 and 3.35 / 5 = 0.67.

1. Deviations from the means. For X they are $-0.40, -0.15, +0.00, +0.20, +0.35$ and for Y they are $-0.15, -0.07, -0.02, +0.06, +0.18$.

2. Sums. The cross-products sum to 0.06 + 0.0105 + 0 + 0.012 + 0.063 = 0.1455. The squared deviations sum to 0.16 + 0.0225 + 0 + 0.04 + 0.1225 = 0.345 for X and 0.0225 + 0.0049 + 0.0004 + 0.0036 + 0.0324 = 0.0638 for Y.

3. The correlation.

$$r_{XY} = \frac{0.1455}{\sqrt{0.345 \times 0.0638}} = \frac{0.1455}{0.1484} = 0.98$$

where the numerator is the sum of cross-products and the denominator is the square root of the product of the two sums of squares. This is very strong convergence.

4. The consequence for QALYs. A treatment moves a patient from the second patient's profile to the fourth patient's profile for one year at an extra cost of £5,000. The QALY gain is 0.80 − 0.45 = 0.35 on the EQ-5D but 0.73 − 0.60 = 0.13 on the SF-6D, so the incremental cost-effectiveness ratios are as follows.

$$ICER_{EQ} = \frac{5000}{0.35} = 14286 \qquad ICER_{SF} = \frac{5000}{0.13} = 38462$$

where $ICER_{EQ}$ and $ICER_{SF}$ are the costs in pounds per QALY gained using EQ-5D and SF-6D values. At an illustrative threshold of £25,000 per QALY the decision depends on the instrument, although the indices correlate at 0.98, because the correlation is unaffected by differences in scale or level between the instruments. Published data show the same pattern. Comparing the two instruments across seven patient and population groups in 2004, Brazier and colleagues reported an SF-6D mean 0.045 above the EQ-5D mean, an intraclass correlation of 0.51, a smaller range and lower variance for the SF-6D, floor effects in the SF-6D and ceiling effects in the EQ-5D. Agreement statistics such as the intraclass correlation are needed when one instrument might replace another.

Pitfalls in reading convergent correlations

Most misreadings come from treating one coefficient as a verdict on the instrument instead of as one tested prediction. Four errors recur in health economic work.

A high correlation can be inflated by shared method or overlapping content. A low one is not automatically poor validity, since measurement error, a narrow spread of health in the sample or an ill-chosen comparator can all depress it. Validity belongs to a population and a use, so convergence in cancer says little about hearing loss, and index-level convergence can hide an unresponsive dimension. Convergence shows that two instruments rank patients similarly, not that their values can be swapped in a cost-utility analysis, where the size of utility differences drives the result.

Sources

• Brazier J, Longworth L. NICE DSU Technical Support Document 8: An introduction to the measurement and valuation of health for NICE submissions. Sheffield: Decision Support Unit, ScHARR, University of Sheffield; 2011. Sections 3 and 3.3.2. https://doi.org/10.15131/shef.data.33738301
• Brazier J, Roberts J, Tsuchiya A, Busschbach J. A comparison of the EQ-5D and SF-6D across seven patient groups. Health Economics. 2004;13(9):873-884. https://doi.org/10.1002/hec.866
• Fiske DW, Campbell DT. This Week's Citation Classic: Campbell DT and Fiske DW, Convergent and discriminant validation by the multitrait-multimethod matrix, Psychological Bulletin 56:81-105, 1959. Current Contents. 1987. https://garfield.library.upenn.edu/classics1987/A1987G546400001.pdf
• Longworth L, Yang Y, Young T, Mulhern B, Hernández Alava M, Mukuria C, et al. Use of generic and condition-specific measures of health-related quality of life in NICE decision-making: a systematic review, statistical modelling and survey. Health Technology Assessment. 2014;18(9). https://doi.org/10.3310/hta18090
• Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology. 2010;63(7):737-745. https://doi.org/10.1016/j.jclinepi.2010.02.006. Taxonomy as presented by the COSMIN initiative: https://www.cosmin.nl/tools/cosmin-taxonomy-measurement-properties/
• Mokkink LB, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, de Vet HCW, Terwee CB. COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs): user manual. Version 1.0. Amsterdam: COSMIN; February 2018. Box 9 and Table 8. https://www.cosmin.nl/wp-content/uploads/COSMIN-syst-review-for-PROMs-manual_version-1_feb-2018.pdf
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. Section 4.3. https://www.nice.org.uk/process/pmg36
• Wailoo A, Hernández Alava M, Pudney S. NICE DSU Technical Support Document 22: Mapping to estimate health state utilities (updates and replaces TSD 10). Sheffield: Decision Support Unit, ScHARR, University of Sheffield; June 2023, updated September 2026. Sections 2 to 4. https://doi.org/10.15131/shef.data.33849340