Problem 2.1. Let \(A\) be \(m\times n\) and \(B\) be \(n\times m\). Prove that \(tr\left (AB\right )=tr\left (BA\right )\), and give an example showing that \(AB\) and \(BA\) need not even have the same size.

Problem 2.2. Show that for any matrix \(X\), \(X^tX\) is symmetric and positive semi-definite, and that it is positive definite if and only if \(X\) has full column rank. Where to start: consider \(\underline {a}^tX^tX\underline {a} =\left \|X\underline {a}\right \|^2\).

Problem 2.3. Write the quadratic form \(Q(X)=3x_1^2-2x_1x_2+4x_2^2\) in the form \(X^tAX\) with \(A\) symmetric, and determine whether it is positive definite.

Problem 2.4. Let \(\underline {Y}\) be a random vector with mean \(\underline {\mu }\) and variance-covariance matrix \(\Sigma \), and let \(A\) be a constant matrix. Derive \(E\left (A\underline {Y}\right )\) and \(\text {Var}\left (A\underline {Y}\right )\) from the definitions.

Problem 2.5. Show that a variance-covariance matrix is always symmetric and positive semi-definite, and explain what a zero eigenvalue would mean about the random vector.

Problem 2.6. Using the derivative results of Section 2.3, minimise \(\left (\underline {Y}-XB\right )^t\left (\underline {Y}-XB\right )\) with respect to \(B\) and obtain the normal equations. State where full column rank is used.

Problem 2.7. Let \(B\) be symmetric idempotent. Prove that its eigenvalues are all \(0\) or \(1\), and hence that \(rank(B)=tr(B)\). Explain why this makes the trace a convenient way to read off degrees of freedom.