Confirmatory Factor Analysis

Confirmatory Factor Analysis for Testing the Structure of Health Outcome Questionnaires

Confirmatory factor analysis (CFA) starts from a stated hypothesis about a questionnaire, such as which items reflect physical health and which reflect mental health, and tests whether the correlations among the answers are consistent with it. In health economics it checks the dimensional structure of quality-of-life and patient-reported outcome instruments, and whether they measure the same thing across languages, countries or patient groups, before their scores become evidence in a cost-utility analysis. Exploratory factor analysis, factor analysis and the structural equation model have their own pages.

From a hypothesis about an instrument to a testable model

Jöreskog set out the method in 1969. Any number of parameters of the factor model can be held fixed at chosen values, the remaining free parameters are estimated by maximum likelihood, and the fit of the solution under the hypothesis represented by the fixed parameters is tested with a large-sample chi-square test based on the likelihood ratio. Choosing the fixed parameters defines the factors in advance and makes rotation unnecessary.

In a questionnaire model most fixed values are zeros, so a physical health item has its loading on the mental health factor fixed at zero. Exploratory factor analysis instead lets the data suggest the factors and loadings. Within structural equation modelling a CFA is the measurement part, and Kenny notes that a good-fitting measurement model is required before the causal paths of the structural model are interpreted.

The measurement model and the correlations it implies

Each item is written as a function of the latent variable it is hypothesised to reflect, plus a residual. The model then implies a correlation for every pair of items, which the data test. For one factor and standardised items the model takes the form below.

$$x_i = \lambda_i \xi + \delta_i, \qquad \rho_{ij} = \lambda_i \lambda_j, \qquad \operatorname{Var}(\delta_i) = 1 - \lambda_i^2$$

where $x_i$ is the standardised response to item $i$, $\xi$ is the latent factor with its variance fixed at 1, $\lambda_i$ is the standardised factor loading of item $i$, $\delta_i$ is its residual, and $\rho_{ij}$ is the implied correlation between items $i$ and $j$. Residuals are assumed uncorrelated with each other and with the factor, as in Gamst-Klaussen and colleagues' measurement equation. Newsom derives the product rule from Wright's tracing rules and notes that the residual may hold systematic unique variance, not only measurement error. If two items share wording the factor does not capture, their observed correlation exceeds the product of their loadings and the model misfits at that pair.

Identification by fixing the scale and counting degrees of freedom

A latent factor has no unit of its own. Newsom describes imposing one by setting one loading or the factor variance to 1. Kenny's t rule then requires at least as many known values as free parameters, with the degrees of freedom calculated as follows.

$$df = \frac{k(k+1)}{2} - q$$

where $df$ is the model's degrees of freedom, $k$ is the number of observed items, $k(k+1)/2$ is the number of distinct variances and covariances, and $q$ is the number of free parameters. A single factor with three items has 6 known values and 6 free parameters (three loadings, three residual variances), so it is just identified. Newsom shows its loadings solve exactly from the three correlations, and Kenny notes that such a model reproduces the data, leaving no fit to test. A fourth item adds four known values and two parameters, giving 10 − 8 = 2 degrees of freedom.

Estimation and judging fit

Jöreskog's method uses maximum likelihood estimation, but questionnaire items are usually ordinal. For the ordinal SF-12v2 items, Lau and colleagues used weighted least squares with mean and variance adjustment (WLSMV) on the polychoric correlation matrix. Gamst-Klaussen and colleagues kept maximum likelihood for the EQ-5D-5L, with bootstrap standard errors for possible non-normality.

The chi-square test asks whether the model reproduces the observed covariances exactly; Kenny notes that with 400 or more cases it is almost always significant, which is why approximate indices such as those below are used.

$$\text{RMSEA} = \sqrt{\frac{\chi^2_M - df_M}{df_M\,(N-1)}}, \qquad \text{CFI} = \frac{(\chi^2_B - df_B) - (\chi^2_M - df_M)}{\chi^2_B - df_B}$$

where $\chi^2_M$ and $df_M$ belong to the proposed model, $\chi^2_B$ and $df_B$ to the baseline (null) model in which the items are uncorrelated, with $df_B = k(k-1)/2$, and $N$ is the sample size. RMSEA is set to zero when $\chi^2_M$ is below $df_M$, and CFI is bounded between 0 and 1. The standardised root mean square residual (SRMR) summarises the standardised differences between observed and predicted correlations and has no penalty for complexity.

The COSMIN guideline for reviews of patient-reported outcome measures (PROMs) defines structural validity as the degree to which PROM scores adequately reflect the dimensionality of the construct, and rates it sufficient when a CFA gives CFI or TLI above 0.95, or RMSEA below 0.06, or SRMR below 0.08. The "or" matters, because one index can pass while another fails badly. Kenny cites RMSEA values of 0.01, 0.05 and 0.08 for excellent, good and mediocre fit, with 0.10 suggested by others for poor fit, warns that small samples and few degrees of freedom inflate RMSEA, and calls cherry picking an index problematic.

Two applied cases with the EQ-5D-5L and the SF-12v2

Two published analyses apply CFA to instruments used in economic evaluation; both reject a single reflective factor. Gamst-Klaussen, Gudex and Olsen fitted CFA models to EQ-5D-5L responses from 7,933 respondents in six countries. Treating all five dimensions as effect (reflective) indicators of one health-related quality of life factor gave CFI 0.967 and SRMR 0.028 but RMSEA 0.117, which the authors judged a poor fit. Treating self-care and usual activities as effect indicators and pain/discomfort, anxiety/depression and mobility as causal indicators gave CFI 0.992, RMSEA 0.075 and SRMR 0.012, agreeing with their tetrad analysis. On CFI or SRMR alone the rejected model would clear the COSMIN thresholds. The authors argued that this causal pattern supports preference weighting.

Lau and colleagues tested the SF-12v2 in 6,126 respondents to the Singapore Mental Health Study 2016. A single factor fitted poorly (RMSEA 0.109, CFI 0.940, SRMR 0.116); correlated physical and mental health factors did better (RMSEA 0.083, CFI 0.965); and adding five residual correlations used in earlier SF-12v2 studies gave RMSEA 0.044, CFI 0.991 and SRMR 0.044. Because the developer's scoring treats the two components as orthogonal, the authors advised caution with its summary scores. The caution concerns the physical and mental component summary scores; the SF-6D, which generates health state utility values directly from SF-36 and SF-12 data, is a preference-weighted classification with its own value set.

Measurement invariance for comparing groups on the same scale

Putnick and Bornstein describe measurement invariance as the psychometric equivalence of a construct across groups or across time, including pretests and posttests around an intervention, and a prerequisite for comparing group means. Without it, a difference between countries in a multinational trial, or between baseline and follow-up, may reflect items working differently. Multiple-group CFA tests it in the steps shown in the table.

Once the configural, metric and scalar steps are passed, latent means can be compared. Putnick and Bornstein report that the change in chi-square is overly sensitive in large samples, that a CFI change of −0.01 is a common criterion, and that there is no consensus on cutoffs. Lau and colleagues found configural, metric and scalar invariance for the SF-12v2 across English, Mandarin and Malay versions, ethnic groups, and people with and without chronic physical conditions.

Worked example with a four-item physical function scale

All figures are illustrative. A disease-specific questionnaire has four physical function items hypothesised to reflect one factor; in $N = 500$ patients the estimated standardised loadings are 0.8, 0.7, 0.6 and 0.5, with the factor variance fixed at 1.

1. Implied correlations. The product rule gives 0.8 × 0.7 = 0.56 for items 1 and 2, then 0.48, 0.40, 0.42 and 0.35 for the pairs (1, 3), (1, 4), (2, 3) and (2, 4), and 0.6 × 0.5 = 0.30 for items 3 and 4. Residual variances are 1 − 0.64 = 0.36, 1 − 0.49 = 0.51, 1 − 0.36 = 0.64 and 1 − 0.25 = 0.75. Suppose items 3 and 4, both about walking, correlate at 0.45 in the data, 0.15 above the implied value, while the other pairs lie within 0.01 of theirs.

2. Degrees of freedom. With $k = 4$ there are 4 × 5 / 2 = 10 known values and 8 free parameters, so $df_M = 2$; the baseline model has $df_B = 4 × 3 / 2 = 6$.

3. Fit. Suppose $\chi^2_M = 32.0$ and $\chi^2_B = 900$.

$$\text{RMSEA} = \sqrt{\frac{32.0 - 2}{2 \times 499}} = \sqrt{\frac{30}{998}} = 0.173, \qquad \text{CFI} = \frac{894 - 30}{894} = \frac{864}{894} = 0.966$$

where 894 is $\chi^2_B - df_B = 900 - 6$ and 30 is $\chi^2_M - df_M$. CFI clears the COSMIN threshold of 0.95 while RMSEA is far above 0.06, and the misfit sits at the walking pair.

4. A modification. Freeing the residual covariance between items 3 and 4 adds one parameter, so $df_M = 1$, and suppose $\chi^2_M$ falls to 3.1.

$$\text{RMSEA} = \sqrt{\frac{3.1 - 1}{1 \times 499}} = \sqrt{0.0042} = 0.065, \qquad \text{CFI} = \frac{894 - 2.1}{894} = \frac{891.9}{894} = 0.998$$

where 2.1 is $\chi^2_M - df_M$. The chi-square difference is 32.0 − 3.1 = 28.9 on 1 degree of freedom, well above the 5% critical value of 3.84.

5. Interpretation. The modified model clears the COSMIN rule through CFI, while RMSEA stays just above 0.06; the change followed the data, so it stays exploratory until justified by item content and replicated, and with one degree of freedom RMSEA is unstable. In a summed score, walking-specific variance gives walking more weight than the one-factor hypothesis intended.

What CFA tells an economic evaluation

Under the NICE manual (PMG36, section 4.3.2), a health state utility comes from measuring health-related quality of life and then valuing it by preferences; CFA belongs entirely to the measuring step. A factor score carries no preference weights and is not anchored at 1 for full health and 0 for dead, so it cannot enter a QALY calculation without valuation or a justified mapping function.

Where a submission argues that the EQ-5D is inappropriate, the NICE manual (PMG36, section 4.3.10) asks for evidence on content validity together with poor performance on construct validity and responsiveness. Factor structure evidence can add to that case but not replace it.

Limitations and common misreadings

Most errors treat a fit statistic as a verdict on the instrument. Four recur.

1. Fit read as validity. Good fit shows consistency with the proposed structure, not that the factors are the intended constructs; construct validity needs hypotheses tested against other evidence.
2. Respecifying until the model fits. Putnick and Bornstein call post hoc alterations data-driven and exploratory, and each parameter freed in one sample risks overfitting; Lau and colleagues freed only residual correlations used in earlier studies.
3. Cutoffs read as laws. Kenny reports that most analysts caution against strict reliance on cutoffs. Chi-square flags small misfit in large samples, RMSEA is inflated in small low-df models, and the EQ-5D case shows indices disagreeing on one model.
4. Assuming every item is reflective. Where some items cause the construct, as the EQ-5D analysis suggests for pain, anxiety and mobility, a single reflective factor is misspecified however the indices look.

Sources

• Gamst-Klaussen T, Gudex C, Olsen JA. Exploring the causal and effect nature of EQ-5D dimensions: an application of confirmatory tetrad analysis and confirmatory factor analysis. Health and Quality of Life Outcomes. 2018;16:153. https://doi.org/10.1186/s12955-018-0975-y
• Jöreskog KG. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika. 1969;34(2):183-202. https://doi.org/10.1007/BF02289343
• Kenny DA. SEM: terminology and basics. 2011. https://davidakenny.net/cm/basics.htm
• Kenny DA. Measuring model fit. 2024. https://davidakenny.net/cm/fit.htm
• Lau JH, Abdin E, Vaingankar JA, Shafie S, Sambasivam R, Shahwan S, et al. Confirmatory factor analysis and measurement invariance of the English, Mandarin, and Malay versions of the SF-12v2 within a representative sample of the multi-ethnic Singapore population. Health and Quality of Life Outcomes. 2021;19:80. https://doi.org/10.1186/s12955-021-01709-9
• Mukuria C, Rowen D, Mulhern B, McDool E, Kharroubi S, Bjorner JB, Brazier JE. The Short Form 6 Dimensions (SF-6D): development and evolution. Applied Health Economics and Health Policy. 2025;23(1):19-33. https://doi.org/10.1007/s40258-024-00919-8
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. Sections 4.3.2 and 4.3.10.
• Newsom JT. Latent variables. Lecture notes, Psy 523/623 Structural Equation Modeling, Portland State University; 2025. https://web.pdx.edu/~newsomj/semclass/ho_latent.pdf
• Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, Terwee CB. COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research. 2018;27(5):1147-1157. https://doi.org/10.1007/s11136-018-1798-3
• Putnick DL, Bornstein MH. Measurement invariance conventions and reporting: the state of the art and future directions for psychological research. Developmental Review. 2016;41:71-90. https://doi.org/10.1016/j.dr.2016.06.004