5.2 Sampling from a Multivariate Normal Population
Let \(\underline {X}_1,\dots ,\underline {X}_n\) be a random sample from \(N_p(\underline {\mu },\Sigma )\), with \(\overline {\underline {X}} = \frac {1}{n}\sum \underline {X}_j\) and \(S = \frac {1}{n-1}\sum \left (\underline {X}_j-\overline {\underline {X}}\right ) \left (\underline {X}_j-\overline {\underline {X}}\right )'\).
Theorem 5.11 (Maximum likelihood estimators). The maximum likelihood estimators of \(\underline {\mu }\) and \(\Sigma \) are \[\widehat {\underline {\mu }} = \overline {\underline {X}}, \qquad \widehat {\Sigma } = \frac {1}{n}\sum ^{n}_{j=1} \left (\underline {X}_j-\overline {\underline {X}}\right ) \left (\underline {X}_j-\overline {\underline {X}}\right )' = \frac {n-1}{n}\,S .\]
Note 5.12. As in the univariate case the maximum likelihood estimator of the covariance divides by \(n\) and is biased; \(S\) divides by \(n-1\) and is unbiased. Both are used, and which is meant should always be checked, particularly in software output.
Theorem 5.13 (Sampling distributions). For a random sample from \(N_p(\underline {\mu },\Sigma )\):
- (i)
- \(\overline {\underline {X}}\ \sim \ N_p\!\left (\underline {\mu },\ \tfrac {1}{n}\Sigma \right )\);
- (ii)
- \((n-1)S \sim W_p\left (n-1,\Sigma \right )\), the Wishart distribution defined below;
- (iii)
- \(\overline {\underline {X}}\) and \(S\) are independent.
Note 5.14. These are the exact multivariate analogues of the univariate facts that \(\overline {X}\sim N(\mu ,\sigma ^{2}/n)\), that \((n-1)s^{2}/\sigma ^{2}\sim \chi ^{2}_{n-1}\), and that the two are independent. Part (iii) is the property that makes Hotelling’s \(T^{2}\) possible at all: the statistic is a ratio involving both \(\overline {\underline {X}}\) and \(S\), and its distribution can only be derived because the numerator and denominator are independent.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.