11.4 Practice Problems
Problem 11.1. Show that when \(p_1=p_2=1\) the single canonical correlation is the ordinary correlation coefficient between the two variables.
Problem 11.2. Show that when \(p_1=1\) the squared canonical correlation equals the coefficient of determination \(R^{2}\) from regressing that variable on the second set. Where to start: Theorem 11.2 reduces to a scalar in that case.
Problem 11.3. Explain why the canonical correlations are invariant under separate non-singular linear transformations of the two sets of variables, and why this is a desirable property that ordinary covariances lack.
Problem 11.4. With \(p_1=6\), \(p_2=8\) and \(n=25\), comment on the reliability of the first canonical correlation. What would you do before interpreting it?
Show solution
Solution. The analysis has fourteen variables and twenty-five observations, so \(\Sigma _{11}\) and \(\Sigma _{22}\) are estimated from very little: each involves \(21\) and \(36\) distinct entries respectively. Maximising a correlation over so many free directions will produce a large value even from unrelated data, and the first canonical correlation is close to guaranteed to look impressive.
Before interpreting anything one would test whether the correlations exceed what chance produces at these dimensions - Bartlett’s test on the remaining correlations - and, better, cross-validate by computing the variates on one part of the data and correlating them on another. If the correlation collapses on the held-out part it was fitted noise.
Problem 11.5. Explain in what sense discriminant analysis is a special case of canonical correlation.
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.