12.4 Practice Problems

Problem 12.1. State two differences between principal component analysis and factor analysis, and give a situation in which each would be the appropriate choice.

Problem 12.2. For a one-factor model with \(p=3\), write \(\Sigma =LL'+\Psi \) in full and count the free parameters against the entries of \(\Sigma \). Is the model testable? Where to start: \(\Sigma \) has \(\tfrac 12 p(p+1)=6\) distinct entries; the model has \(pm+p=6\) parameters.

Show solution

Solution. With \(m=1\), \(L=(\ell _1,\ell _2,\ell _3)'\) and \(\Psi =\operatorname {diag}(\psi _1,\psi _2,\psi _3)\), so \[\Sigma = \begin {pmatrix} \ell _1^{2}+\psi _1 & \ell _1\ell _2 & \ell _1\ell _3\\ \ell _1\ell _2 & \ell _2^{2}+\psi _2 & \ell _2\ell _3\\ \ell _1\ell _3 & \ell _2\ell _3 & \ell _3^{2}+\psi _3 \end {pmatrix}.\] There are \(6\) parameters and \(6\) distinct entries in \(\Sigma \), and with \(m=1\) the rotational constraint count \(\tfrac 12 m(m-1)\) is zero. The degrees of freedom are therefore \(6-6=0\): the model is exactly determined and fits any \(\Sigma \) with the right sign pattern, so there is nothing to test. Testability requires \(\tfrac 12\left [(p-m)^{2}-p-m\right ]>0\), which for \(m=1\) needs \(p\geq 4\).

Problem 12.3. Prove Theorem 10.5, and explain why it means the factors cannot be identified from the covariance matrix alone.

Problem 12.4. Show that when \(p_1=p_2=1\) the single canonical correlation is the ordinary correlation coefficient between the two variables.

Problem 12.5. Explain why single linkage is prone to chaining, and construct a small set of points in the plane on which single and complete linkage give different two-cluster solutions.

Problem 12.6. Data with no group structure at all are given to a \(k\)-means algorithm with \(k=3\). Describe what it returns and how one would detect that the clustering is spurious.

Pre-requisites: Linear Algebra, Mathematical Statistics

Course Contents

: Descriptive Techniques of Multivariate data: Data organisation; summary statistics.
: Random Vectors: Random vectors and random matrices; mean vectors; covariance matrices; partitioning a covariance matrix; the mean vector and covariance matrix for a linear combination.
: Random Sampling: Random samples and the expected values of the sample mean and sample covariance matrix; sample values of linear combinations of variable.
: The Multivariate Normal distribution: Definition and properties; conditional and marginal distributions of multi normal variates; sampling from a multivariate normal distribution; maximum likelihood estimation of \(\mu \) and \(\Sigma \); sampling distribution of the sample mean vector \(\overline {X}\) and the sample covariance \(S\); joint distribution of \(\overline {X}\) and \(S\); Wishart distribution; the Hotelling \(T^2\)- distribution.
: Inferences About a Population Mean Vector: Tests on mean vector \(\mu \); the \(T^2\) statistics; confidence regions and simultaneous comparisons of component means.
: Comparison of Two Multivariate Means: Paired comparisons; comparing mean vectors of two populations.
: One Way MANOVA: Comparison of several multivariate population means; simultaneous confidence intervals for treatment effects.
Two Way Multivariate Analysis of Variance: Multivariate two-way fixed effects model.
: Multivariate Linear Regression Model: Multivariate multiple regression model, estimation of parameters, likelihood ratio tests for regression parameters, concept of linear regression and predictions.

[prev] [prev-tail] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.