8.4 Diagonalisation of Symmetric Matrices
A real matrix may not have real eigenvalues unless it is symmetric \((A^t=A)\).
We denote the set of all real symmetric \(n\times n\) matrices by \(M^{(s)}_n(\mathbb {R})\).
Definition 8.4.1. \(A,B\in M^{(s)}_n(\mathbb {R})\) are called orthogonally similar if there exist an orthogonal matrix \(P\in M_n(\mathbb {R})\) such that \(B=P^tAP\).
Proof. Let \(A\in M_n^{(s)}(\mathbb {R})\) and \(\lambda \) be an eigenvalue of \(A\) with corresponding vector \(X\in V_n(\mathbb {R})\), then \(AX=\lambda X\qquad (1)\).
From (1) \begin {align*} X^{-t}AX &=X^{-t}\lambda X\\ X^{-t}AX&=\lambda X^{-t}X\qquad (2) \end {align*}
Also from (1) \begin {align*} \Big (\overline {A}\hspace {0.1cm}\overline {X}\Big )^t &=\Big (\overline {\lambda }\hspace {0.1cm}\overline {X}\Big )^t\\ \overline {X}^t\hspace {0.1cm}\overline {A}^t &=\overline {\lambda }^t\hspace {0.1cm}\overline {X}^t\\ \overline {X}^tA &=\overline {\lambda }\hspace {0.1cm}\overline {X}^t\\ \overline {X}^tAX &=\overline {\lambda }\hspace {0.1cm}\overline {X}^tX\qquad (3) \end {align*}
From (1) and (2) \begin {align*} \lambda X^{-1}tX &=\overline {\lambda }\hspace {0.1cm}\overline {X}^tX\\ (\lambda -\overline {\lambda })\hspace {0.1cm}\overline {X}^tX &=0\\ \implies \hspace {0.3cm}(\lambda -\overline {\lambda })&=0\qquad \text {since}\qquad \overline {X}^tX\neq 0\\ \implies \hspace {0.3cm} \lambda &=\overline {\lambda }\\ \implies \hspace {0.3cm}\lambda \hspace {0.3cm}\text {is real}.\\ \end {align*} □
Lemma 8.4.3. If the eigenvalues of a real symmetric matrix \(A\) are distinct, then the corresponding eigen vectors are orthogonal to each other.
Proof. Let \(\lambda _1,\lambda _2,\dots ,\lambda _n\) be the distinct eigenvalues of \(A\) with corresponding eigen vectors \(X_1,X_2,\dots ,X_n\) respectively. Then \begin {align*} AX_i &=\lambda _iX_i,\hspace {0.5cm} i=1,2,\dots ,n\\ X_i^tA &=\lambda _iX_i^t\\\\ X^t_iAX_j &=\lambda _iX^t_iX_j\qquad (1) \end {align*}
Also \begin {align*} AX_j &=\lambda _j X_j\\ X^t_iAX_j &=X^t_i\lambda _iX_j=\lambda _jX_i^tX_j\qquad (2) \end {align*}
(1) and (2) gives \[\lambda _jX_i^tX_j=\lambda _iX^t_iX_j\implies (\lambda _j-\lambda _i)X^t_iX_j=0\] \(\implies X^t_iX_j=0\) since \(\lambda _i\neq \lambda _j\). Therefore \((X^t_i,X_j)=0\hspace {0.3cm} \forall i\neq j\). \(\implies \hspace {0.3cm X_i\perp X_j}\). □
Theorem 8.4.4. If \(A\) is a real symmetric matrix with distinct eigenvalues then \(A\) is orthogonally similar to a diagonal matrix.
Proof. Two facts about a real symmetric \(A\) do the work.
Eigenvectors for distinct eigenvalues are orthogonal
Let \(Av=\lambda v\) and \(Aw=\mu w\) with \(\lambda \neq \mu \). Using \(A^t=A\), \[\lambda (v,w)=(Av,w)=(v,A^tw)=(v,Aw)=\mu (v,w),\] so \((\lambda -\mu )(v,w)=0\), and as \(\lambda \neq \mu \) we get \((v,w)=0\).
Assembling the matrix
Since the eigenvalues are real and distinct, there are \(n\) of them, with eigenvectors \(v_1,\dots ,v_n\). Normalise each to unit length, \(e_i=v_i/\|v_i\|\); by the paragraph above the \(e_i\) are pairwise orthogonal, so \(\{e_1,\dots ,e_n\}\) is an orthonormal set of \(n\) vectors and hence an orthonormal basis.
Let \(P\) be the matrix with columns \(e_1,\dots ,e_n\). Its columns being orthonormal means precisely that \(P^tP=I\), that is, \(P\) is orthogonal and \(P^{-1}=P^t\). As in Theorem 8.2.3(i), \(AP=PD\) with \(D=\text {diag}(\lambda _1,\dots ,\lambda _n)\), so \[P^tAP=P^{-1}AP=D.\] Thus \(A\) is orthogonally similar to a diagonal matrix. □
Theorem 8.4.5. If \(A\) is a real symmetric matrix then there exists an orthogonal matrix \(P\) such that \(P^tAP=\text {diag}(\lambda _1,\lambda _2,\dots ,\lambda _n)\), where \(\lambda _1,\dots ,\lambda _n\) are the eigenvalues of \(A\). (Spectral theorem.)
Proof. The previous theorem assumed the eigenvalues were distinct. The point here is that no such assumption is needed; the proof is by induction on \(n\).
For \(n=1\) there is nothing to prove. Suppose the result holds for symmetric matrices of size \(n-1\), and let \(A\) be real symmetric of size \(n\). By Lemma 8.4.2 its eigenvalues are real, so choose one of them, \(\lambda _1\), with a unit eigenvector \(e_1\).
Let \(W=\{x:(x,e_1)=0\}\) be the orthogonal complement of \(e_1\); by Theorem 7.2.8, \(\mathbb {R}^n=\langle e_1\rangle \oplus W\) and \(\dim W=n-1\). The key observation is that \(A\) maps \(W\) into itself: if \((x,e_1)=0\) then, using symmetry, \[(Ax,e_1)=(x,Ae_1)=(x,\lambda _1e_1)=\lambda _1(x,e_1)=0,\] so \(Ax\in W\).
Choose an orthonormal basis of \(W\) and let \(A'\) be the matrix of the restriction of \(A\) to \(W\) in that basis. \(A'\) is symmetric — it is the matrix of a map satisfying \((Ax,y)=(x,Ay)\) with respect to an orthonormal basis — and has size \(n-1\), so by the induction hypothesis \(W\) has an orthonormal basis consisting of eigenvectors of \(A\). Adjoining \(e_1\) gives an orthonormal basis \(e_1,\dots ,e_n\) of \(\mathbb {R}^n\) made of eigenvectors of \(A\).
Let \(P\) have these as its columns. Then \(P^tP=I\), so \(P\) is orthogonal, and \(AP=PD\) gives \(P^tAP=D=\text {diag}(\lambda _1,\dots ,\lambda _n)\). □
Remark. This is the strongest diagonalisation result in the course, and its two extra conclusions matter: the diagonalising matrix can be taken orthogonal, so that \(P^{-1}\) costs nothing to compute, and every real symmetric matrix is diagonalisable, whatever multiplicities its eigenvalues have.
Example 8.4.6. Find an orthogonal matrix \(P\) such that \(P^tAP=\) diagonal matrix for \(A= \begin {pmatrix} 10&-14&-10\\-14&7&-4\\-10&-4&19\\ \end {pmatrix} \).
Solution. \begin {align*} X_A(x) &= \begin {pmatrix} 10-x&-14&-10\\-14&7-x&-4\\-10&-4&19-x\\ \end {pmatrix}\\\\ &=(x-18)(x-27)(x+9)\\\\ \text {So that}\hspace {0.4cm}& \lambda _1=18, \lambda _2=27, \lambda _3=-9\\ \end {align*}
For \(\lambda _1=18\) we have \( V_1= \begin {pmatrix} 1\\-2\\2\\ \end {pmatrix} \).
\(\lambda _2=27,\hspace {0.5cm} V_2= \begin {pmatrix} -2\\1\\2\\ \end {pmatrix} \).
\(\lambda _3=-9,\hspace {0.5cm} V_3= \begin {pmatrix} 2\\2\\1\\ \end {pmatrix} \). Thus \(P=\frac {1}{3} \begin {pmatrix} 1&-2&2\\-2&1&2\\2&2&1\\ \end {pmatrix} \) and \begin {align*} P^tAP &=\text {diag}(\lambda _1,\lambda _2,\lambda _3)\\ &=\text {diag}(18,27,-9)\\\\ &= \begin {pmatrix} 18&0&0\\0&27&0\\0&0&-9\\ \end {pmatrix}\\ \end {align*}
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.