1.8 Vectors and Matrices

1.8.1 Inner Product

Suppose that \(\underline {X} = \begin {pmatrix} X_1, & X_2, & \cdots , & X_n\\ \end {pmatrix}'\) and \(\underline {Y} = \begin {pmatrix} Y_1, & Y_2, & \cdots , & Y_n\\ \end {pmatrix}'\) are two vectors, one useful product of \(\underline {X}\) and \(\underline {Y}\) is called the INNER PRODUCT. It is defined by \begin {align*} \underline {X}'\underline {Y} & = \begin {pmatrix} X_1 & ,, \ldots , & X_n\\ \end {pmatrix} \begin {pmatrix} Y_1\\ \vdots \\ \vdots \\ Y_n\\ \end {pmatrix}\\\\ & = X_1Y_1 + X_2Y_2 + \cdots + X_nY_n\\ & = \sum ^n_{i=1}X_iY_i\quad \cdots \cdot \quad (1)\\ \end {align*}

If \(\underline {Y}\) is replaced by \(\underline {X}\) we obtain the square of \((\underline {X}^2)\) of the length of \(\underline {X}\) from the origin

\[L^2_x = \underline {X}'\underline {X}= \sum ^n_{i=1}X^2_i\]

\begin {equation} \tag {2} L_x = \sqrt {\sum X_i^2} \end {equation}

Similarly for \(\underline {Y}\)

\begin {equation} \tag {3} L_y = \sqrt {\sum Y_i^2} \end {equation}

In \(\mathbb {R}^2\) we have a geometrical concept to the relation (1)

In the figure:

1.

\(\theta = \theta _2 - \theta _1\)

2.

\(\theta _2\) is associated with \(\underline {Y}\)

3.

\(\theta _1\) is associated with \(\underline {X}\)

4.

\(\frac {X_1}{L_x}= \cos \theta _1, \frac {X_2}{L_x}= \sin \theta _1\)

5.

\(\frac {Y_1}{L_y}= \cos \theta _2, \frac {Y_2}{L_y}=\sin \theta _2\)

6.

Now \begin {align*} \cos 3\theta & = \cos ( \theta _2 - \theta _1)\\ & = \cos \theta _2\cos \theta _1 + \sin \theta _2\sin \theta _1\\\\ & = \frac {Y_1}{L_y}\cdot \frac {X_1}{L_x} + \frac {Y_2}{L_y}\cdot \frac {X_2}{L_x}\\ \end {align*}

\begin {align*} \cos 3\theta & = \frac {X_1Y_1 + X_2Y_2}{L_XL_Y}=\frac {\underline {X}'\underline {Y}}{L_xL_y}\\\\ & = \frac {\underline {X}'\underline {Y}}{\sqrt {\underline {X}'\underline {X}}\sqrt {\underline {Y}'\underline {Y}}}\tag {4}\\ \end {align*}

From relationship (4) we have the \(\displaystyle {4.1\qquad \theta = 0 \implies 1 = \frac {\underline {X}'\underline {Y}}{L_xL_y}\qquad \text {or}\quad L_xL_y = \underline {X}'\underline {Y}}\)

suggesting \(\underline {Y} = t\underline {X}\) for some \(t\) \(\displaystyle {4.2\qquad \theta = \frac {\pi }{2}\implies 0 = \frac {\underline {X}'\underline {Y}}{L_xL_y}\implies \underline {X}'\underline {Y} = 0}\)

suggesting \(\underline {X}\) and \(\underline {Y}\) are perpendicular.

1.8.2 Projection

If \(\underline {X}\) and \(\underline {Y}\) are two vectors the projection of \(\underline {X}\) on \(\underline {Y}\) is \(\displaystyle {= \frac {\underline {X}'\underline {Y}}{\underline {Y}'\underline {Y}}\,\underline {Y}}\)

( a vector on \(\underline {Y}\)) \(\qquad (5)\)

\[\frac {\underline {x}'\underline {y}}{L^2_Y}\,\underline {y}\]

Note that \(\displaystyle {\frac {\underline {X}'\underline {Y}}{L_yL_y}\, \underline {Y}= \frac {\underline {X}'\underline {Y}}{L_y}\cdot \frac {\underline {Y}}{L_y}=\frac {\underline {X}'\underline {Y}}{L_y}\cdot \mu _y}\)

\(\mu _y = \frac {\underline {Y}}{L_y}\) ( a unit vector in the direction of \(\underline {Y}\)). So \(\frac {\underline {X}'\underline {Y}}{L_yL_y}\, \underline {Y} = L_x\cos \theta \frac {\underline {Y}}{L_y}\).

The length of the projection is

\begin {align*} \Bigg [\Bigg (\frac {\underline {X}'\underline {Y}}{L_y^2}\,\underline {Y}\Bigg )'\Bigg (\frac {\underline {X}'\underline {Y}}{L_y^2}\,\underline {Y}\Bigg )\Bigg ]^{\frac {1}{2}} & = \frac {L_x L_y \begin {vmatrix} \cos \theta \\ \end {vmatrix} }{L_y} = L_x \begin {vmatrix} \cos \theta \\ \end {vmatrix}\tag {6}\\ \end {align*}

\[\underline {X}'\underline {Y} = 39350.24\]

\[L^2_y = 42600.83\]

\[\frac {\underline {X}'\underline {Y}}{L^2_y}\,\underline {Y}\]

1.8.3 Eigen Values and Eigen Vectors

1.

General result
If \(A_{p\times p}\) is any square matrix then \[q(\lambda ) = \begin {vmatrix} A - \lambda I\\ \end {vmatrix}\] is the \(p^{\text {th}}\) order polynomial in \(\lambda \). The \(P\) roots of \(q(\lambda )\), \(\quad \lambda _1,\ldots , \lambda _p\) possibly complex numbers, are called eigenvalues of matrix \(A\).
Some of the \(\lambda _i\) will be equal if \(q(\lambda )\) has multiple roots. For each \(i=1,2,\ldots , p\),
\(\begin {vmatrix} A - \lambda I\\ \end {vmatrix} = 0\), so \(A - \lambda _i I\) is singular. Hence there exists a non-zero vector \(\underline {e}\) satisfying \begin {equation} \tag {2} A\underline {e} = \lambda \underline {e} \end {equation} Any vector satisfying (2) is called an eigenvector of \(A\) and \(\begin {vmatrix} A - \lambda I\\ \end {vmatrix} = 0\) is called the characteristic equation.

2.

An eigenvector \(\underline {e}\) with real entries is called standardised if \(\underline {e}'\underline {e} = 1\qquad (3)\)

3.

Since the coefficients of \(\lambda ^p\) in \(q(\lambda )\) is \((-1)^p\) we can write \(q(\lambda )\) in terms of its roots as follows \begin {equation} \tag {4} q(\lambda ) = \prod ^p_{i=1} (\lambda _i - \lambda ) \end {equation} and setting \(\lambda = 0\) in (1) and (4) gives \begin {equation} \tag {5} \begin {vmatrix} A\\ \end {vmatrix}= \prod ^p_{i=1} \lambda _i \end {equation} i.e \(\begin {vmatrix} A\\ \end {vmatrix}\) is the product of the eigenvalues of A.

4.

Matching the coefficients of \(\lambda ^{p-1}\) in equation (1) and (4) gives \begin {equation} \tag {6} \sum ^p_{i=1} a_{ij} = \sum ^p_{i=1} \lambda _i = Tr(A) \end {equation} \(Tr(A)\) is the sum of eigenvalues of \(A\).

1.

Determine

(a)

eigenvalues \(\lambda _1, \lambda _2, \lambda _3\)

(b)

eigenvectors

2.

Verify

(a)

\(\begin {vmatrix} A\\ \end {vmatrix} = \prod \lambda _i\)

(b)

\(Tr(A) = \sum \lambda _i\)

\begin {align*} (1-\lambda )(1-\lambda )(2-\lambda )-9(2-\lambda ) & = 0\\ (2-\lambda )\big [(1-\lambda )^2-9\big ] & = 0\\ (2-\lambda )(1-\lambda - 3) (1-\lambda + 3) & = 0\\ (2-\lambda )(-2-\lambda )(4-\lambda ) & = 0\\ \end {align*}

\[\lambda = 2, \lambda = 4, \lambda = -2\]

1.

(a)

\(\lambda _1 = 2, \lambda _2 = 4, \lambda _3 = -2\)

(b)

\(\lambda _1 = 2\) \begin {align*} A\underline {e} & = 2 \underline {e}\qquad (A\underline {e} = \lambda \underline {e})\\\\ \begin {pmatrix} 1 & 3 & 0\\ 3 & 1 & 0\\ 0 & 0 & 2\\ \end {pmatrix}\begin {pmatrix} e_1\\ e_2\\ e_3\\ \end {pmatrix} & = 2\begin {pmatrix} e_1\\ e_2\\ e_3\\ \end {pmatrix}\\ \end {align*}

\begin {align*} e_1 + 3e_2 & = 3e_1 \quad \cdots \quad (i)\\ 3e_1 + e_2 & = 2e_2\quad \cdots \quad (ii)\\ 2e_3 & = 2e_3\quad \cdots \quad (iii)\\ \end {align*}

From \((ii)\) , \(e_2=3e_1\) and using \((i)\) we obtain \[e_1 =0 \implies e_2 = 0\] Choose \(e_3 = 1\) so \(\underline {e}_1 = \begin {pmatrix} 0, & 0, & 1\\ \end {pmatrix}'\)

Alternatively: \(\begin {pmatrix} 1 & 3 & 0\\ 3 & 1 & 0\\ 0 & 0 & 2\\ \end {pmatrix}\underline {e} = 2\underline {e}\)

\begin {align*} \implies \qquad \Bigg [\begin {pmatrix} 1 & 3 & 0\\ 3 & 1 & 0\\ 0 & 0 & 2\\ \end {pmatrix} -2 I_3\Bigg ] \underline {e} & = 0\\\\ \begin {pmatrix} -1 & 3 & 0\\ 3 & -1 & 0\\ 0 & 0 & 0\\ \end {pmatrix}\underline {e} & = 0\\\\ \begin {pmatrix} A^* & \underline {0}\\ \underline {0} & 0\\ \end {pmatrix}\begin {pmatrix} \underline {e}_1\\ \underline {e}_2\\ \end {pmatrix} & = \begin {pmatrix} \underline {0}\\ \underline {0}\\ \end {pmatrix}\quad \cdots \quad *\\ \end {align*}

From \(*\) we obtain \(e_1 = e_2 =0\) since a \( \begin {vmatrix} A^*\\ \end {vmatrix}= \begin {vmatrix} -1 & 3\\ -3 & -1\\ \end {vmatrix}\neq 0\) and \(e_3\) can be chosen to be any value.

For \(\lambda = 4\) \(\qquad \begin {pmatrix} 1 & 3 & 0\\ 3 & 1 & 0\\ 0 & 0 & 2\\ \end {pmatrix}\underline {e} =4\underline {e}\)

\begin {align*} e_1 + 3e_2 & = 4e_1 \quad \cdots \quad (i)\\ 3e_1 + e_2 & = 4e_2 \quad \cdots \quad (ii)\\ 2e_3 & = 4e_3 \quad \cdots \quad (iii)\\ \end {align*}

From \((iii)\), \(2e_3 = 0 \implies e_3 = 0\) in \((i)\) \[3e_2 = 3e_1 \implies e_2 = e_1\] the same as in \((ii)\). Choose \(e_1 = e_2 = 1\) \[\underline {e}^*_2 = \begin {pmatrix} 1, & 1, & 0\\ \end {pmatrix}'\]

Note 1.8. \(\begin {vmatrix} \begin {vmatrix} \underline {e}_2^*\\ \end {vmatrix} \end {vmatrix} = \sqrt {2}\), but we require \(e^*_2\) to be a unit vector, hence we normalise \(\underline {e}^*_2\) to obtain

\begin {align*} \underline {e}_2 & = \begin {pmatrix} \frac {1}{\sqrt {2}}, & \frac {1}{\sqrt {2}}, & 0\\ \end {pmatrix}'\\\\ & = \begin {pmatrix} \frac {\sqrt {2}}{2}, & \frac {\sqrt {2}}{2}, & 0 \\ \end {pmatrix}'\\ \end {align*}

For \(\lambda = -2\qquad \) \(\begin {pmatrix} 1 & 3 & 0\\ 3 & 1 & 0\\ 0 & 0 & 2\\ \end {pmatrix}\underline {e} = -2\underline {e}\)

\begin {align*} e_1 + 3e_2 & = -2e_1 \quad \cdots \quad (i)\\ 3e_1 + e_2 & = -2e_2 \quad \cdots \quad (ii)\\ 2e_3 & = -2e_e \quad \cdots \quad (iii)\\ \end {align*}

From \((iii)\) we get \(4e_3 = 0 \implies e_3 =0\)

\((i)\) and \((ii)\) \(\implies \quad \left .\begin {aligned} 3e_1 + 3e_2 & = 0\\ 3e_1 + 3e_2 & = 0\\ \end {aligned} \right \}\quad \cdots \quad * \)

from \(*\) we get \(e_1 = -e_2\). Choose \(e_1 = 1 \implies e_2 = -1\) . Hence \(\underline {e}^*_3 = \begin {pmatrix} 1, & -1, & 0\\ \end {pmatrix}'\)

\[\text {Thus}\qquad \underline {e}_3 = \begin {pmatrix} \frac {1}{\sqrt {2}}, & -\frac {1}{\sqrt {2}}, & 0\\ \end {pmatrix}'\]

\(\lambda _1 = 2\)		\(\lambda _2 = 4\)		\(\lambda _3 = -2\)
\(\displaystyle { \underline {e}_1 = \begin {pmatrix} 0\\ 0\\ 1\\ \end {pmatrix} }\)		\(\displaystyle {\underline {e}_2 = \begin {pmatrix} 1/\sqrt {2}\\ 1/\sqrt {2}\\ 0\\ \end {pmatrix} }\)		\(\displaystyle {\underline {e}_3= \begin {pmatrix} 1/\sqrt {2}\\ -1/\sqrt {2}\\ 0\\ \end {pmatrix} }\)

Note 1.9. \begin {align*} \underline {e}_1 \cdot \underline {e}_2 & = 0 (1/\sqrt {2}) + 0(1/\sqrt {2}) + 1 (0) = 0\\ \underline {e}_1 \cdot \underline {e}_3 & = 0(1\sqrt {2}) + 0(-1/\sqrt {2}) + 1 (0) = 0\\ \underline {e}_2 \cdot \underline {e}_3 & = \frac {1}{\sqrt {2}}(1/\sqrt {2}) + \frac {1}{\sqrt {2}}(-1\sqrt {2}) + 0(0) = 0\\ \end {align*}

\[\implies \qquad \underline {e}_1 \perp \underline {e}_2, \underline {e}_1 \perp \underline {e}_3, \underline {e}_2 \perp \underline {e}_3\]

2.

(a)

\begin {align*} \begin {vmatrix} A\\ \end {vmatrix} & = \begin {vmatrix} 1 & 3\\ 3 & 1\\ \end {vmatrix}(2) = (1-9)(2)\\\\ & = (-8)(2)\\\\ & = \prod ^3_{i=1}\lambda _i = \lambda _1 \cdot \lambda _2 \cdot \lambda _3\\ & = 2(4)(-2)\\ & = -16\\ \end {align*}

(b)

\begin {align*} Tr(A) & = Tr \begin {pmatrix} 1 & 3 & 0\\ 3 & 1 & 0\\ 0 & 0 & 2\\ \end {pmatrix}\\\\ & = 1 + 1 + 2 = \sum ^3_{i=1} \lambda _i = \\\\ & = 4 = 2 + 4 + (-2)\\ \implies \qquad & 4 = 4\\\\ \end {align*}

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.