Eigenvalue

A mathematical quantity indicating the amount of variance explained by its corresponding direction, used in factor analysis to decide how many factors to keep.

Source: Belsley, Kuh & Welsch 1980

In factor analysis an eigenvalue represents the amount of variance among the measured variables that a particular factor accounts for. A large eigenvalue marks a factor that explains a substantial share of the shared variation, while a small one marks a factor that captures little. Comparing eigenvalues, often by keeping those above one or by reading a scree plot, guides how many factors are worth retaining. Measuring the variance a factor explains is what it does. Kline (2015) describes this.

Source: Kline 2015

Eigenvalues are used in factor analysis to decide how many factors to retain, since each eigenvalue indicates the amount of variance a factor explains, and factors with larger eigenvalues account for more of the variation. Common rules retain factors with eigenvalues above one or use a scree plot of the eigenvalues to find where they level off. So eigenvalues are used to guide the number of factors kept, by indicating each factor's contribution to explaining the variance, which helps balance capturing the important structure against retaining too many factors, and they underpin common criteria for deciding the dimensionality of a factor solution.

Source: Belsley, Kuh & Welsch 1980

An eigenvalue represents the amount of variance in the data explained by its associated factor or component, so a larger eigenvalue means that direction captures more of the total variability. In factor analysis, the eigenvalues from the correlation matrix indicate how much variance each potential factor accounts for. So an eigenvalue represents the variance captured by a factor or component, providing a measure of its importance in summarising the data, which is why the pattern of eigenvalues, from largest to smallest, is examined to judge how many factors meaningfully contribute and how much of the total variance the retained factors together explain.

Source: Belsley, Kuh & Welsch 1980

Eigenvalues help decide the number of factors through criteria based on their sizes: the Kaiser rule retains factors with eigenvalues greater than one, on the reasoning that a factor should explain at least as much variance as a single variable, and the scree plot displays the eigenvalues in order to find the point where they level off, retaining factors before it. So eigenvalues guide the choice of the number of factors by indicating each factor's contribution, with these criteria offering rules of thumb, though they are applied with judgement, since mechanical use can retain too many or too few factors, and interpretability is also considered alongside the eigenvalues.

Source: Belsley, Kuh & Welsch 1980

Eigenvalues relate to collinearity diagnostics in regression because small eigenvalues of the predictor matrix indicate near-linear dependencies among the predictors, that is collinearity, and the condition indices used to detect collinearity are derived from the eigenvalues. A very small eigenvalue signals a direction along which the predictors barely vary, reflecting strong collinearity. So eigenvalues underlie collinearity diagnostics, with small eigenvalues revealing the near-dependencies that define collinearity and giving rise to large condition indices, which is why examining the eigenvalue structure of the predictors is part of diagnosing collinearity in regression as well as determining dimensions in factor analysis.

Source: Belsley, Kuh & Welsch 1980