5.3 The Wishart Distribution
The chi-square distribution describes the sampling behaviour of a sample variance. Its multivariate counterpart describes the sampling behaviour of a whole covariance matrix, and it is the distribution behind every result in the remainder of this section.
Definition 5.15 (Wishart distribution). Let \(\underline {Z}_1,\dots ,\underline {Z}_m\) be independent \(N_p(\underline {0},\Sigma )\) vectors. The distribution of the random matrix \[W = \sum ^{m}_{j=1}\underline {Z}_j\underline {Z}_j'\] is the Wishart distribution with \(m\) degrees of freedom and scale matrix \(\Sigma \), written \(W\sim W_p(m,\Sigma )\).
Note 5.16. Taking \(p=1\) and \(\Sigma =\sigma ^{2}\) gives \(W = \sum Z_j^{2}\) with the \(Z_j\) independent \(N(0,\sigma ^{2})\), so \(W/\sigma ^{2}\sim \chi ^{2}_{m}\): the Wishart with \(p=1\) is the chi-square, scaled. Everything in this subsection reduces to a familiar univariate statement when \(p=1\), and checking that it does is the quickest way to remember any of it.
Theorem 5.17 (Properties). Let \(W\sim W_p(m,\Sigma )\) and \(W_1, W_2\) be independent Wisharts with the same scale matrix. Then
- (i)
- \(E(W) = m\Sigma \);
- (ii)
- \(W_1 + W_2 \sim W_p\left (m_1+m_2,\Sigma \right )\);
- (iii)
- \(AWA' \sim W_q\left (m, A\Sigma A'\right )\) for \(A_{q\times p}\) of constants;
- (iv)
- \(W\) is positive definite with probability one if and only if \(m\geq p\).
Note 5.18. Part (iv) is a practical constraint rather than a technicality. Since \((n-1)S\sim W_p(n-1,\Sigma )\), the sample covariance matrix is singular whenever \(n-1<p\) — fewer observations than variables — and then \(S^{-1}\) does not exist. Every procedure in the rest of this section requires \(S^{-1}\), so all of them fail in that regime, which is the formal reason a multivariate analysis needs more observations than variables. It is the same fact met in Section 4.4, where at most \(n-1\) principal components can be extracted.
Note 5.19. Part (i) explains the divisor in \(S\): since \(E\left [(n-1)S\right ] = (n-1)\Sigma \) we have \(E(S)=\Sigma \), so dividing the sum of squares and cross-products by \(n-1\) rather than \(n\) is what makes \(S\) unbiased — the same reason as in one dimension, with the same degrees of freedom lost to estimating the mean.
Note 5.20. With this in place, Hotelling’s \(T^{2}\) of the next subsection can be seen for what it is. The univariate \(t\) statistic squares to \[t^{2} = \frac {\left (\overline {X}-\mu \right )^{2}}{s^{2}/n} = n\left (\overline {X}-\mu \right )\left (s^{2}\right )^{-1}\left (\overline {X}-\mu \right ),\] and replacing the scalars by their matrix counterparts gives \[T^{2} = n\left (\overline {\underline {X}}-\underline {\mu }\right )' S^{-1}\left (\overline {\underline {X}}-\underline {\mu }\right ).\] It is a normal vector and an independent Wishart matrix combined exactly as a normal variable and an independent chi-square are combined in the univariate case, which is why its null distribution is a multiple of an \(F\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.