2.1 Review of Matrix Algebra

Consider a linear transformation from \(n\) variables \(x_1,x_2,\ldots ,x_n\) to \(m\) variables \(y_1,y_2,\ldots ,y_m\) where \[Y_i=\sum ^n_{j=1}a_{ij}X_i=a_{i1}x_1+a_{i2}+\cdots + a_{in}x_n\]

\[X= \begin {pmatrix} x_1\\ x_2\\ \vdots \\ x_n\\ \end {pmatrix} = \begin {pmatrix} x_1 & x_2 & \cdots x_n \end {pmatrix}^t \]

\[A= \begin {pmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \vdots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn}\\ \end {pmatrix} \]

\[\boxed {Y=AX}\]

\(A\) is known as the matrix of the linear transformation.

Definition 2.1.1. The transpose of an \(m\times n\) matrix \(A\) is denoted as \(A^t\), which is an \(n\times m\) matrix obtained from \(A\) by interchanging the rows and columns.

Lemma 2.1.2. If \(A,B, C\) and \(D\) are matrices and \(\lambda \) is a scalar (constant). Then (whenever the operation is well defined)

i.: \(A\big (B+C\big ) = AB+AC\)
ii.: \(\big (B+C\big )D = BD + CD\)
iii.: \(\big (\lambda A\big )B = A\big (\lambda B\big )=\lambda \big (AB\big )= \lambda AB\).
iv.: \(\big (AB\big )^t=B^tA^t\)
v.: \(\big (AB\big )D=A\big (BD\big ) = ABD\)

Note.

i.: Identity matrix \(I_{m\times m}=\big (\delta _{ij}\big )\) where \(\delta _{ij}= \begin {cases} 1 & \text {if}\quad i=j\\ 0 & \text {if}\quad i\neq j\\ \end {cases} \)
ii.: If \(AB=BA=I\), then \(A\) is the inverse of \(B\) and is written as \(A=B^{-1}\), also \(B\) is the inverse of \(A\) and is written as \(B=A^{-1}\).
iii.: Rank of a matrix \(A\) in the number of linear independent rows or columns.
iv.: \(|A|=\) determinate of \(A\) and only exists if \(A\) is a square matrix.

Lemma 2.1.3.

i.: If \(A\) is an \(n\times n\) matrix then \(|A^t|=|A|\).
ii.: \(\big (A^t\big )^{-1}=\big (A^{-1}\big )^t\)
iii.: \(\big (AB\big )^{-1}=B^{-1}A^{-1}\)
iv.: \(A^{-1}\) exists if \(A\) is square matrix and is non-singular i.e \(|A|\neq 0\).
v.: If \(A\) is an \(m\times n\) matrix, \(P\) is \(m\times m\) and \(Q\) is \(n \times n\), then \(rank\big (PAQ\big )=rank\big (A\big )\) if \(P\) and \(Q\) are non-singular matrices.

Definition 2.1.4. A quadratic form in \(n\) variables \(X=\big (x_1,x_2,\ldots ,x_n\big )^t\) is a function of the form \[Q(X)=\sum ^n_{i=1}\sum ^n_{j=1}a_{ij}x_ix_j\] where \(a_{ij'}s\) are constants.

\(Q(X)=X^tAX\) is matrix form. \(A\) is called the matrix of the quadratic form.

Lemma 2.1.5. Without loss of generality we may assume \(A\) to be symmetric in the \(X^tAX\).

Proof. \(Q(X)\) is a constant \begin {align*} Q^t(X) & = Q(X)\\ \big (X^tAX\big )^t & = X^tAX\\ X^tA^t\big (X^t\big )^t & = X^tAX\\ X^tA^tX & = X^tAX\\ \frac {1}{2}X^tA^tX+\frac {1}{2}X^tAX & = Q(X)\\ X^t\Big (\frac {1}{2}A^t+\frac {1}{2}A\Big )X & =Q(X)\\ X^tA^*X & = Q(X)\, , \quad A^*=\frac {1}{2}\big (A^t+A\big ),\quad A^{*t}=\frac {1}{2}\big (A^{t}+A\big )=A^*\\ \end {align*} □

Example 2.1.6. \[Q(X)=3x^2_1+8x_1x_2+5x^2_1= \begin {pmatrix} x_1& x_2\\ \end {pmatrix} \begin {pmatrix} 3&7\\ 1&5\\ \end {pmatrix} \begin {pmatrix} x_1\\ x_2 \end {pmatrix} \] \[ A= \begin {pmatrix} 3 & 7\\ 1 & 5\\ \end {pmatrix} \quad A^t= \begin {pmatrix} 3 & 1\\ 7 & 5\\ \end {pmatrix} \] \begin {align*} A^* & =\frac {1}{2}\big (A^t+A\big ) = \frac {1}{2} \begin {pmatrix} 6 & 8\\ 8 & 10\\ \end {pmatrix}\\ & = \begin {pmatrix} 3 & 4\\ 4 & 5\\ \end {pmatrix} \end {align*}

Definition 2.1.7.

i.: A matrix \(P\) is called orthogonal matrix if \(P^tP=I\) and the transformation of \(Y=PX\) is called an orthogonal transformation i.e \(P^{-1}=P^t\).
ii.: A square matrix \(D\) is called a diagonal matrix if \(D=\big (d_{ij}\big )_{m\times m}\), where \(d_{ij}=0\) if \(i\neq j\).
iii.: A quadratic form is called diagonal form if there are no cross-products.

Lemma 2.1.8.

i.: The Quadratic for \(Q(X)=X^tAX\) is positive definite iff \(Q(X)>0\) for all \(X\neq 0\).
ii.: For every \(Q(X)=X^tAX\), there exists a non-singular transformation \(Y=PX\) which reduces the \(Q(X)\) to a diagonal form \[Y=PX\implies X=P^{-1}Y\] \[Q(X)=X^tAX=\big (P^{-1}Y\big )^tAP^{-1}Y\]
iii.: Every symmetric matrix \(A\), there exists a non-singular matrix \(P\) such that \(P^tAP\) is diagonal.

[next] [front] [up]

Questions on this section

Stuck on something here? Ask below and it stays attached to this topic.