4.4 Sample Principal Components
In practice \(\Sigma \) is unknown and is replaced by the sample covariance matrix \(S\) of Section 3.3. Writing \((\widehat {\lambda }_i, \widehat {\underline {e}}_i)\) for the eigenvalue–eigenvector pairs of \(S\), the \(i\)th sample principal component of an observation \(\underline {x}_j\) is \[y_{ji} = \widehat {\underline {e}}_i'\left (\underline {x}_j-\overline {\underline {x}}\right ),\] the centring being conventional so that the components have mean zero. Every statement of the previous sections holds with \(\Sigma ,\lambda _i,\underline {e}_i\) replaced by \(S,\widehat {\lambda }_i,\widehat {\underline {e}}_i\).
Note 4.9. Two practical points. The eigenvector is determined only up to sign: if \(\widehat {\underline {e}}_i\) maximises the variance then so does \(-\widehat {\underline {e}}_i\), so software may return either, and a component that appears to have reversed between two analyses may not have changed at all. And if \(n\leq p\) then \(S\) is singular, so at most \(n-1\) eigenvalues are non-zero and no more than that many components exist — a real constraint when variables outnumber observations.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.