7.1 Inner Product Spaces
Length and angle are not part of the definition of a vector space, so they must be added. The device that adds them is a single operation taking two vectors to a scalar; everything else in this section — norms, perpendicularity, projection — is derived from it.
We begin with the familiar case and then abstract it.
Definition 7.1.1. For \(x,y\in \mathbb {R}^n\), the dot product of \(x\) and \(y\) is \[x\cdot y=(x,y)=x_1y_1+x_2y_2+\cdots +x_ny_n.\]
Taking \(y=x\) gives \((x,x)=x_1^2+\cdots +x_n^2\), which is never negative, and Pythagoras identifies its square root as the length of \(x\).
So for \(x\in \mathbb {R}^2\) we have \(||x||=\sqrt {x^2_1+x^2_2}\), for \(x\in \mathbb {R}^3\), \(||x||=\sqrt {x^2_1+x^2_2+x^2_3}\), and for \(x\in \mathbb {R}^n\), \[||x||=\sqrt {x^2_1+x^2_2+\cdots +x^2_n}=\sqrt {(x,x)}.\]
The angle between two vectors is recovered from the same operation by \((x,y)=||x||\,||y||\cos \theta \). In particular \((x,y)=0\) exactly when \(\cos \theta =0\), that is, when the vectors are perpendicular — which is the observation the rest of the section is built on.
The general definition
Nothing above used the coordinates in an essential way, only the four properties below. Taking those as the definition allows the same geometry on spaces of polynomials, matrices and functions, where “length” and “angle” have no prior meaning.
Definition 7.1.2. An inner product space is a vector space \(V\) over \(\mathbb {F}\) together with a function \((\cdot ,\cdot ):V\times V\longrightarrow \mathbb {F}\) satisfying, for all \(x,y,z\in V\) and \(\alpha \in \mathbb {F}\),
- 1.
- \((x,x)\geq 0\), with \((x,x)=0\) if and only if \(x=0\);
- 2.
- \((x,y+z)=(x,y)+(x,z)\);
- 3.
- \((\alpha x,y)=\alpha (x,y)\);
- 4.
- \((x,y)=\overline {(y,x)}\).
The function \((\cdot ,\cdot )\) is called an inner product on \(V\).
Remark.
- i.
- A real inner product space is called a Euclidean space.
- ii.
- A complex inner product space is called a unitary space.
- iii.
- Conditions (3) and (4) together give \((x,\alpha y)=\overline {\alpha }(x,y)\): a scalar taken out of the second argument is conjugated. Over \(\mathbb {R}\) conjugation does nothing and (4) reduces to symmetry, \((x,y)=(y,x)\).
- iv.
- Condition (1) is what makes the next definition possible: it guarantees \((x,x)\) is a non-negative real number, so its square root exists.
Definition 7.1.3. The length, or norm, of \(x\in V\) is denoted \(||x||\) and defined by \[||x||=\sqrt {(x,x)}.\] A vector with \(||x||=1\) is called a unit vector. Any non-zero \(x\) can be made into one by dividing by its length: \(\dfrac {x}{||x||}\) is a unit vector pointing in the same direction, a step called normalising \(x\).
Definition 7.1.4. Two vectors \(x,y\in V\) are orthogonal (perpendicular) if \((x,y)=0\).
For example, with \(x=(1,0)\) and \(y=(0,1)\) in \(\mathbb {R}^2\), \[(x,y)=1(0)+0(1)=0,\] so \(x\) and \(y\) are orthogonal — as the picture of the two axes requires.
Note that \((0,y)=0\) for every \(y\), so the zero vector is orthogonal to everything, itself included.
Proposition 7.1.5. If \(x,y\in V\) and \(\alpha \in \mathbb {F}\) then
- i.
- \(||x||\geq 0\) and \(||x||=0\) if and only if \(x=0\)
- ii.
- \(||\alpha x||=|\alpha |||x||\)
- iii.
- \(|(x,y)|\leq ||x||\,||y||\) (Cauchy–Schwarz inequality);
- iv.
- \(||x+y||\leq ||x||+||y||\) (triangle inequality).
Proof. We prove the triangle inequality; the other three follow directly from the definition of an inner product.
For (iv), \begin {align*} ||x+y||^2 &=(x+y,x+y)\\ &=(x,x)+(x,y)+(y,x)+(y,y)\\ &=||x||^2+\text {Re}2(x,y)+||y||^2\\ &\leq ||x||^2+2(x,y)+||y||^2\\ &\leq ||x||^2+2||x||||y||+||y||^2\\ &=(||x||+||y||)^2\\\\ \text {That is},\hspace {0.4cm} ||x+y||^2 &\leq (||x||+||y||)^2\implies ||x+y||\leq ||x||+||y|| \end {align*} □
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.