4 Principal Component Analysis
Section 3 measured the scatter of a multivariate sample and found that the total variability can be summarised by the trace or the determinant of the covariance matrix. Neither summary says where the variability lies. With \(p\) variables there are \(p\) variances and \(\binom {p}{2}\) covariances, and beyond three or four variables no one can hold that structure in mind, still less plot it.
Principal component analysis answers the question directly. It asks for the direction in which the data vary most, then the direction of greatest remaining variation perpendicular to the first, and so on. The answer turns out to require no new machinery at all: the directions are the eigenvectors of \(\Sigma \) and the variances along them are its eigenvalues, so the whole method is the spectral decomposition of Section 1.9 applied to a covariance matrix.
4.2 How Much Variation Each Component Explains
4.3 Components from the Correlation Matrix
4.4 Sample Principal Components
4.5 How Many Components to Keep
4.6 Practice Problems
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.