4.2 How Much Variation Each Component Explains
Theorem 4.4 (Total variation is preserved). \[\sum ^{p}_{i=1}\operatorname {var}(X_i) = \operatorname {tr}(\Sigma ) = \sum ^{p}_{i=1}\lambda _i = \sum ^{p}_{i=1}\operatorname {var}(Y_i).\]
Proof. \(\operatorname {tr}(\Sigma ) = \operatorname {tr}(P\Lambda P') = \operatorname {tr}(\Lambda P'P) = \operatorname {tr}(\Lambda ) = \sum \lambda _i\), using the cyclic property of the trace from Section 1.4 and \(P'P=I\). □
Definition 4.5 (Proportion explained). The proportion of total variation explained by the \(i\)th component is \[\frac {\lambda _i}{\lambda _1+\cdots +\lambda _p},\] and by the first \(k\) components, \(\left (\lambda _1+\cdots +\lambda _k\right )/\left (\lambda _1+\cdots +\lambda _p\right )\).
Note 4.6. Theorem 4.4 is what licenses the word “explained”. The total variability is a fixed quantity, \(\operatorname {tr}(\Sigma )\), and the eigenvalues partition it exactly. If the first two components account for \(0.9\) of that total, then a two-dimensional plot of \(Y_1\) against \(Y_2\) reproduces ninety percent of the variation in \(p\) variables — and, crucially, one knows what has been lost.
Theorem 4.7 (Correlation between a component and a variable). \[\operatorname {corr}\left (Y_i, X_k\right ) = \frac {e_{ik}\sqrt {\lambda _i}}{\sqrt {\sigma _{kk}}}.\]
These correlations, often called loadings, are how a component is interpreted: a component correlating strongly with several variables is read as whatever those variables have in common.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.