Factor Analysis

What factor analysis does

It is widely used to investigate or test the dimensional structure of questionnaires, patient-reported outcomes and other sets of related measures. This page explains the common-factor model, exploratory and confirmatory approaches, extraction, rotation, interpretation, scoring and validation in health research and economic evaluation.

Factor analysis does not discover a latent construct automatically. The meaning of a factor depends on the variables included, the model assumptions, the data and a defensible substantive interpretation.

The common-factor model separates shared and unique variation

For (p) observed variables and (m) latent factors, the common-factor model can be written as:

$$x=\mu+\Lambda f+\varepsilon$$

where (x) is the (p\times1) vector of observed variables, (\mu) is the mean vector, (\Lambda) is the (p\times m) loading matrix, (f) is the latent-factor vector and (\varepsilon) contains variable-specific influences and measurement error. A loading describes the relationship between an observed variable and a factor, conditional on the model and rotation used.

Under the usual assumptions that (E[f]=0), (E[\varepsilon]=0), (\operatorname{Cov}(f,\varepsilon)=0), (\operatorname{Cov}(f)=\Phi) and (\operatorname{Cov}(\varepsilon)=\Psi), the model-implied covariance matrix is:

$$\Sigma=\Lambda\Phi\Lambda^{\mathsf T}+\Psi$$

The matrix (\Psi) is commonly diagonal, meaning that residuals are assumed uncorrelated after the factors are accounted for. Correlated residuals require substantive justification rather than being added only to improve statistical fit.

Loadings, communalities and uniqueness describe each variable

The communality of an observed variable is the portion of its variance represented by the common factors. Uniqueness is the remaining variance attributed to variable-specific influences and measurement error. Their calculation depends on the factor covariance structure.

With orthogonal factors and a standardised observed variable (x_i), its communality is:

$$h_i^2=\sum_{j=1}^{m}\lambda_{ij}^2$$

and its uniqueness is:

$$\psi_i=1-h_i^2$$

For oblique factors, factor correlations in (\Phi) must be included:

$$h_i^2=\lambda_i\Phi\lambda_i^{\mathsf T}$$

where (\lambda_i) is row (i) of (\Lambda). Squaring and summing oblique pattern coefficients as though the factors were independent can give the wrong communality.

A worked one-factor example

Suppose three standardised questionnaire items have loadings of 0.80, 0.70 and 0.60 on one common factor. Because there is only one factor with variance one, the communalities are the squared loadings.

For the first item:

$$h_1^2=0.80^2=0.64$$

Its uniqueness is:

$$\psi_1=1-0.64=0.36$$

The three items have communalities and uniquenesses:

Under this model, the factor accounts for more of Item 1's variance than Item 3's. A high loading does not by itself establish that the item is valid, unbiased or clinically important.

Exploratory factor analysis searches for a plausible structure

Exploratory factor analysis, or EFA, is used when the number of factors or pattern of item–factor relationships is not fully specified in advance. All observed variables are initially allowed to load on all factors, and the analyst examines whether a clear, stable and substantively interpretable pattern emerges. EFA remains model-based and still requires choices about correlations, extraction, factor retention and rotation.

An EFA should report:

• The variables and population included in the analysis.
• The correlation matrix used, including whether Pearson or polychoric correlations were analysed.
• The extraction method and rationale.
• The evidence used to retain the number of factors.
• The rotation method and whether factors were allowed to correlate.
• Pattern coefficients, factor correlations, communalities and notable residuals.
• Sensitivity analyses and any item-removal decisions.

Repeatedly deleting items solely to create a visually simple loading table risks overfitting the sample and weakening content validity.

Confirmatory factor analysis tests a prespecified measurement model

Confirmatory factor analysis, or CFA, specifies in advance which observed variables measure which latent factors and which parameters are fixed, constrained or freely estimated. It tests how well the implied covariance structure reproduces the observed covariance structure. The model should be grounded in theory, prior evidence or an independently developed exploratory analysis.

A CFA can evaluate:

• Whether a proposed one-factor or multidimensional structure fits the data.
• Whether factors are distinguishable from one another.
• Whether particular items have cross-loadings or correlated residuals that require explanation.
• Whether measurement parameters are equivalent across groups or time.
• Whether a higher-order or bifactor model is substantively and statistically defensible.

Modification indices can identify local strain, but data-driven changes should not be made mechanically. Each change alters the measurement model and should be justified, cross-validated and reported.

Exploratory and confirmatory analyses serve different purposes

EFA and CFA use related latent-variable mathematics but answer different questions. EFA develops or investigates a structure with relatively few prior restrictions, while CFA evaluates an explicitly specified structure. Using the same sample to search for and confirm a model exaggerates the independence of the evidence.

The distinction is about analytical purpose and constraints, not simply which software command is used.

Factor analysis is not principal component analysis

Principal component analysis, or PCA, creates weighted combinations of observed variables that explain total observed variance. Common-factor analysis models only the covariance attributed to latent common factors and separates uniqueness. The two methods can sometimes produce superficially similar patterns, but their aims and mathematical models differ.

Calling PCA a factor analysis can mislead readers about measurement error and latent constructs. The chosen method should match the research question.

Correlation choice depends on the measurement scale

Standard factor analysis of continuous variables often uses a Pearson correlation or covariance matrix. Questionnaire items with ordered response categories may be better represented using ordinal latent-response methods and polychoric correlations, especially when categories are few or distributions are strongly skewed. Binary items may require tetrachoric or categorical-data models.

The analysis should examine:

• Whether variables are continuous, ordinal, binary or mixed.
• Whether response categories have adequate counts.
• Whether floor or ceiling effects limit observed variation.
• Whether missingness and imputation preserve the intended relationships.
• Whether variables require reverse scoring before analysis.
• Whether the estimator matches non-normality and categorical measurement.

Treating ordinal categories as continuous is sometimes a pragmatic approximation, but its consequences should be assessed rather than assumed negligible.

Factor extraction estimates the shared structure

Extraction methods estimate loadings and uniquenesses from the observed correlation or covariance matrix. Maximum likelihood supports likelihood-based inference and fit testing under its distributional assumptions. Principal-axis factoring and minimum-residual methods offer alternatives when the purpose and data do not support standard maximum-likelihood assumptions.

The extraction method should be selected before inspecting the preferred-looking solution. Analysts should distinguish non-convergence, improper solutions and unstable estimates from evidence that a substantive factor does not exist.

Warning signs include:

• A negative estimated uniqueness, sometimes called a Heywood case.
• A standardised loading greater than one in absolute value.
• A non-positive-definite correlation or covariance matrix.
• Very large standard errors or implausible factor correlations.
• Failure of the estimation algorithm to converge.
• A solution that changes materially under small data or method perturbations.

These problems may reflect model misspecification, sampling variability, too many factors, redundant variables or data-quality issues.

Factor retention needs more than the eigenvalue-greater-than-one rule

Choosing the number of factors is one of the most consequential decisions in EFA. No single numerical rule should replace substantive knowledge and evaluation of alternative solutions. The commonly used eigenvalue-greater-than-one rule can over- or under-extract factors and should not be used alone.

Evidence may include:

• Parallel analysis comparing observed eigenvalues with those from random data.
• A scree plot identifying where the eigenvalue curve begins to level off.
• Theoretical expectations and the intended use of the instrument.
• Interpretability, factor definition and absence of trivial factors.
• Residual correlations and overall reproduction of the observed matrix.
• Stability across samples, estimators and plausible analytical choices.

Parallel analysis is a strong empirical aid, but its implementation should match the data type and extraction framework where possible.

Rotation makes a factor solution easier to interpret

Factor solutions are not unique without additional constraints: different rotated loading matrices can reproduce the same covariance structure. Rotation seeks a simpler pattern, such as variables loading strongly on a limited number of factors. It changes the representation of the solution, not the fundamental fit of an unrestricted exploratory factor model.

Orthogonal rotation constrains factor correlations to zero. Oblique rotation allows factors to correlate and is often more realistic for related health constructs. Under an oblique solution, two loading-related matrices should be distinguished:

• The pattern matrix contains regression-like coefficients linking variables to factors while accounting for factor correlations.
• The structure matrix contains correlations between observed variables and factors.

The matrices coincide for orthogonal factors but differ when factors correlate. Interpretation should identify which matrix is being reported.

Identification makes estimation possible

A factor model must impose enough constraints to place each factor on a scale and distinguish parameters. Common scaling choices fix one loading per factor to one or fix each factor variance to one. Changing the scaling convention changes numerical parameter values but should not change the substantive model.

Identification also depends on the number and arrangement of indicators, fixed zeros, cross-loadings and residual covariances. A formally identified model can still be weakly identified in practice, producing unstable estimates. Software reporting a solution is not proof that the measurement model is empirically informative.

Model fit requires several kinds of evidence

CFA fit statistics compare the observed and model-implied covariance structures. Fit indices are diagnostic summaries, not pass–fail certificates, and fixed universal cut-offs should not replace examination of sample size, estimator, model complexity, residuals and substantive plausibility.

Common measures include:

• The chi-square test assesses exact fit but is sensitive to sample size and model assumptions.
• The comparative fit index and Tucker–Lewis index compare the proposed model with a baseline model.
• The root mean square error of approximation describes approximate discrepancy per degree of freedom under its assumptions.
• The standardised root mean square residual summarises standardised residual differences.
• Information criteria can compare models fitted to the same data under compatible likelihoods.
• Residual matrices reveal specific relationships the model fails to reproduce.

Good global fit can coexist with biased parameters, poor item coverage or an uninterpretable factor. Poor fit can also arise from dependence, non-normality or omitted cross-loadings rather than a completely wrong construct.

Factor scores estimate positions on latent dimensions

Factor scores assign each person an estimated value on one or more latent factors. Common approaches include regression scores, Bartlett scores and model-based posterior estimates. Because factors are unobserved and scores are estimated with error, different scoring methods can produce different values and rankings.

A linear factor-score estimate may be written as:

$$\hat{f}=B^{\mathsf T}(x-\mu)$$

where (B) is a matrix of scoring coefficients. Factor-score indeterminacy means that the common-factor model does not uniquely determine every individual's true factor values. Scores should not be treated as error-free observed outcomes, especially in subsequent regression or economic modelling.

For an operational questionnaire, a published sum score or preference-weighted value may have a different purpose from a factor score. Factor analysis does not automatically authorise replacing the instrument's validated scoring algorithm.

Measurement invariance supports valid group comparisons

Measurement invariance evaluates whether a construct is measured comparably across groups or time. Without adequate invariance, an observed score difference may reflect different item functioning rather than a true difference in the latent construct. Testing proceeds through increasingly restrictive models, interpreted with parameter changes, fit and substantive evidence.

Common levels include:

Partial invariance may be defensible when a limited set of parameters differs, but the exceptions and consequences for the intended comparison should be explicit.

Sample adequacy depends on the model and data

There is no universally sufficient observations-per-item rule. Required sample size depends on factor loadings, communalities, number of indicators per factor, factor correlations, missing data, estimator, response distributions and model complexity. Strong, well-determined factors may be estimable with fewer observations than weak factors with sparse indicators.

Planning should consider:

• Whether each factor has enough strong and conceptually appropriate indicators.
• The smallest subgroup when invariance or subgroup models are planned.
• Rare response categories and missing-data patterns.
• Expected convergence, standard-error precision and factor-recovery performance.
• Simulation-based sample-size assessment for complex or high-stakes models.

Sampling adequacy statistics can identify problematic correlation patterns, but they do not replace design-based sample-size justification.

Factor analysis contributes to health measurement and economics

Health-economic studies use multi-item measures of symptoms, functioning, capability, satisfaction and quality of life. Factor analysis can evaluate whether items reflect the proposed dimensions before scores or latent variables enter clinical or economic analyses. It can also inform item reduction, although content coverage and decision relevance must be preserved.

Applications include:

• Developing and validating patient-reported outcome measures.
• Evaluating the dimensionality of health-related quality-of-life instruments.
• Testing measurement equivalence across languages, populations or treatment groups.
• Reducing correlated indicators before regression or prediction, when a latent-factor interpretation is justified.
• Building structural equation models that connect latent health constructs to costs, outcomes or behaviour.
• Investigating whether response patterns support a proposed capability or wellbeing framework.

Evidence for factor structure is only one component of instrument validity. Reliability, content validity, responsiveness, interpretability and relevance to the target population also matter.

Common errors weaken conclusions

Factor analysis involves many defensible choices, which also creates opportunities for selective analysis. Transparent reporting should distinguish decisions made in advance from those made after seeing the data.

• Treating PCA as common-factor analysis changes the meaning of the solution.
• Using only the eigenvalue-greater-than-one rule can select the wrong number of factors.
• Forcing orthogonal rotation can hide meaningful correlations among health constructs.
• Interpreting factor labels from one or two weak items can overstate the evidence.
• Deleting cross-loading items automatically can damage content validity.
• Applying Pearson correlations and normal-theory estimation to sparse ordinal data can bias results.
• Reporting only global fit indices can conceal large local residuals or implausible parameters.
• Confirming a structure in the same data used to discover it overstates replication.
• Treating estimated factor scores as perfectly measured variables understates uncertainty.
• Assuming measurement invariance without testing can invalidate group comparisons.

A defensible factor-analysis workflow

A high-quality analysis links the substantive construct, measurement design, statistical model and intended use. The workflow should preserve a record of alternative decisions so readers can judge whether the final structure is stable or selectively chosen.

1. Define the construct: State what each proposed factor represents and how the variables cover its content.
2. Inspect the data: Check coding, reverse-scored items, missingness, response distributions, category counts and dependence.
3. Choose correlations: Match Pearson, polychoric, tetrachoric or another association measure to the variables and model.
4. Specify the purpose: Distinguish exploratory structure development from confirmatory model testing.
5. Select the estimator: Justify extraction or estimation against distribution, scale and sample characteristics.
6. Determine factor number: Combine empirical evidence, theoretical expectations and interpretability rather than relying on one rule.
7. Choose rotation: Allow correlated factors when theory and evidence make independence implausible.
8. Evaluate the solution: Examine loadings, communalities, factor correlations, residuals, fit, improper estimates and stability.
9. Validate independently: Replicate or cross-validate the structure and test measurement invariance when comparisons require it.
10. Report the consequences: Explain scoring, uncertainty, item decisions, limitations and implications for downstream health-economic use.

Factor analysis is most credible when it clarifies the measurement structure without allowing statistical simplicity to override the meaning and coverage of the health construct.