Principal Component

In principal component analysis, a derived variable combining original observed variables to capture the maximum possible variance, uncorrelated with prior components.

Source: Pearson 1901

A principal component is a new variable formed by combining the original measured variables so as to capture as much of their total variation as possible. The first component captures the most, the next captures the most of what remains while staying uncorrelated with the first, and so on, so a few components can summarise many variables. This makes principal component analysis a way to reduce dimensionality, condensing correlated measurements into a handful of informative summaries. Packing maximum variance into a new variable is what it does. Kline (2015) describes this.

Source: Kline 2015

Principal components are derived from the covariance or correlation matrix of the variables by finding the directions, given by the eigenvectors, along which the data vary most, with the associated eigenvalues giving the variance captured. The first component is the direction of greatest variance, and each subsequent component the greatest remaining variance while orthogonal to the previous ones. So principal components are derived as the eigenvectors of the variables' covariance or correlation matrix, ordered by the variance they explain, which produces uncorrelated components that successively capture the maximum remaining variance, providing a mathematically optimal summary of the data's variation.

Source: Pearson 1901

Principal component analysis is used for data reduction, summarising many correlated variables by a few components that capture most of the variance; for visualisation, by plotting the leading components; and for addressing collinearity, since the components are uncorrelated. So principal component analysis is used to simplify high-dimensional data, retaining most of the information in fewer dimensions, which is valuable for exploration, visualisation, and as a preprocessing step for other analyses, and because the components are uncorrelated it can also handle collinearity among predictors, making principal components a widely used tool for reducing and understanding multivariate data.

Source: Pearson 1901

A principal component is a combination of the observed variables constructed to capture maximum variance, being a summary of the data, while a factor in factor analysis is a latent variable hypothesised to cause the correlations among the observed variables, representing an underlying construct. Principal component analysis is descriptive data reduction; factor analysis models latent structure. So a principal component and a factor differ in interpretation and purpose, with the component a variance-maximising combination of the variables and the factor a hypothesised underlying cause, which is why principal component analysis is used for summarising and reducing data and factor analysis for identifying latent constructs, though the two are related and sometimes confused.

Source: Pearson 1901

The number of principal components to retain is chosen to capture a sufficient proportion of the total variance while reducing dimensionality, using criteria such as the cumulative variance explained, the eigenvalues, or a scree plot showing where the explained variance levels off. So the number of principal components is selected by balancing how much variance is retained against the reduction in dimensions, keeping enough components to summarise the data adequately without retaining many that add little, which is why the pattern of eigenvalues and the cumulative variance are examined, along with the purpose of the analysis, to decide how many components meaningfully represent the data.

Source: Pearson 1901