5.2 Random vectors

For a random vector \(X = (X_1, \dots , X_n)'\), the mean is the vector \(\mu = E(X)\) of means, and the covariance matrix is \[ \Var (X) = E\bigl [(X - \mu )(X - \mu )'\bigr ], \] whose \((i, j)\) element is \(\Cov (X_i, X_j)\) and whose diagonal holds the variances.

Theorem 5.2.1. Let \(X\) have mean \(\mu \) and covariance matrix \(\Sigma \), and let \(A\) be a constant \(m \times n\) matrix and \(b\) a constant \(m\)-vector.

(i): \(E(AX + b) = A\mu + b\) and \(\Var (AX + b) = A\Sigma A'\).
(ii): \(\Sigma \) is symmetric and psd.
(iii): For a constant symmetric \(n \times n\) matrix \(B\), \(E(X'BX) = \tr (B\Sigma ) + \mu 'B\mu \).

Proof. (i) Expectation is linear element by element. Then \(AX + b - (A\mu + b) = A(X - \mu )\), so \(\Var (AX + b) = E\bigl [A(X - \mu )(X - \mu )'A'\bigr ] = A\Sigma A'\).

(ii) Symmetry is clear. For any \(c\), \(c'\Sigma c = \Var (c'X) \geq 0\) by (i) with \(A = c'\).

(iii) By Lemma 5.1.2, \(X'BX = \tr (BXX')\), and \(E(XX') = \Sigma + \mu \mu '\). Since the trace is linear, \(E(X'BX) = \tr \bigl (B(\Sigma + \mu \mu ')\bigr ) = \tr (B\Sigma ) + \mu 'B\mu \). □

Part (iii) needs no distributional assumption at all. For example, if \(X\) has independent elements with mean \(\mu \) and variance \(\sigma ^2\), then with \(B = M_0\), \[ E\Bigl [\sum (X_i - \bar X)^2\Bigr ] = \tr (M_0\sigma ^2I) + \mu ^2\iota 'M_0\iota = (n - 1)\sigma ^2 , \] because \(M_0\iota = 0\). That is the familiar reason for the divisor \(n - 1\) in a sample variance.

[next] [prev] [prev-tail] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.