Lecture 28 - Continuing Inner Product Spaces

Review of Last Time...

Consider the inner product space $V$ , as defined in Chapter 6 - Inner Product Spaces#^a64259, equipped with $⟨ \cdot, \cdot ⟩$ as the inner product. There's an array of properties that we saw in Chapter 6 - Inner Product Spaces#Inner Products, allowing us to define the norm via:

‖ v ‖ = \sqrt{⟨ v, v ⟩}

which is the norm induced by the inner product.

norms in a higher context

Normally norms are defined in the same way in other fields of mathematics, namely things like Real Analysis and Abstract Algebra refer to this definition.

Here, if $⟨ u, v ⟩ = 0$ iff $u, v$ are orthogonal. Review the Pythagorean Theorem:

Pythagorean Theorem

Suppose $u, v$ are orthogonal vectors in $V$ . Then:
$
||u + v||^2 = ||u||^2 + ||v||> $

Which we proved in Chapter 6 - Inner Product Spaces#^4c14b5 below.

We also talked about orthogonal decomposition, which is talked in our notes via Chapter 6 - Inner Product Spaces#^315ed2,

where we had:

u = \underset{orange vector}{\underset{⏟}{\frac{⟨ u, v ⟩}{‖ v ‖^{2}} v}} + \underset{blue vector}{\underset{⏟}{u - \frac{⟨ u, v ⟩}{‖ v ‖^{2}} v}}

Notice that the orange vector is what we've seen in Calc III. We saw:

{proj}_{v} u = \frac{u \cdot v}{‖ v ‖^{2}} v

which here now is more generalized.

Cauchy-Schwartz Inequality

We'll prove the following:

Cauchy-Schwartz Inequality

Given two vectors $u, v \in V$ :

| ⟨ u, v ⟩ | \leq ‖ u ‖ ‖ v ‖

where we have equality when $u, v$ are scalar multiples of each other.

Proof
If either $v = \vec{0}$ is then we get an easy equality, so suppose $v \neq \vec{0}$ . Thus:

u = \frac{⟨ u, v ⟩}{‖ v ‖^{2}} v + w

we know that the left vector in the sum is parallel to $v$ , while $w$ is the other vector which is orthogonal to $v$ . As such, then we can use the Pythagorean Theorem between the left and right vectors:

‖ u ‖^{2} = {‖ \frac{⟨ u, v ⟩}{‖ v ‖^{2}} v ‖}^{2} + ‖ w ‖^{2}

The whole scalar in the left vector can come out:

\begin{aligned} ‖ u ‖^{2} & = {| \frac{⟨ u, v ⟩}{‖ v ‖^{2}} |}^{2} {‖ v ‖}^{2} + ‖ w ‖^{2} \\ = \frac{| ⟨ u, v ⟩ |^{2}}{‖ v ‖^{2}} + ‖ w ‖^{2} \end{aligned}

Notice that the left term in the sum is non-negative, and $‖ w ‖$ is also non-negative. Hence:

‖ u ‖^{2} \geq \frac{| ⟨ u, v ⟩ |^{2}}{‖ v ‖^{2}}

Thus, multiplying over and taking the square root:

‖ u ‖ ‖ v ‖ \geq | ⟨ u, v ⟩ |

Notice that we only get equality when $u, v$ are scalar multiples of each other, since then $w$ is perpendicular to $v$ while pointing in the $u$ direction, so then $‖ w ‖^{2} = 0$ so $w = \vec{0}$ .
☐
Bring this back to our standard dot product, where:

| a \cdot b | = | a | | b | \cos (θ) \Rightarrow | a \cdot b | = | a | | b | | \cos (θ) | \leq | a | | b |

since $0 \leq | \cos (θ) | \leq 1$ .

Triangle Inequality

We do something similar for this new theorem:

Triangle Inequality

$‖ u + v ‖ \leq ‖ u ‖ + ‖ v ‖$

where equality holds when $u, v$ are positive multiples of one another.

Proof
Consider $‖ u + v ‖^{2}$ first:

\begin{aligned} ‖ u + v ‖^{2} & = ⟨ u + v, u + v ⟩ \\ = ⟨ u, u ⟩ + ⟨ v, v ⟩ + ⟨ u, v ⟩ + ⟨ v, u ⟩ \\ = ⟨ u, u ⟩ + ⟨ v, v ⟩ + 2 R (⟨ u, v ⟩) & (Adding two conjugates gives 2 R of z) \\ \leq ‖ u ‖^{2} + ‖ v ‖^{2} + 2 | ⟨ u, v ⟩ | & (Real Part is smaller than magnitude) \\ \leq ‖ u ‖^{2} + ‖ v ‖^{2} + 2 ‖ u ‖ ‖ v ‖ & (Cauchy-Schwartz Inequality) \\ = (‖ u ‖ + ‖ v ‖)^{2} \end{aligned}

Square rooting both sides gives the theorem above.
☐

Intro to 6.B: Unit Norms

Recall that we used unit vectors to do a lot of things, so we define that below:

orthonormal

A list of vectors that are pairwise orthogonal (orthogonal to each and every one of each other) and are all of norm 1 is called orthonormal.

Here it just means that:

all vectors are orthogonal (perpendicular) to each other
all vectors are unit length

For instance, the list $e_{1}, e_{2}, . . ., e_{n}$ ie the standard basis in $R^{n}$ is orthonormal.

Suppose a different list $e_{1} . . ., e_{n}$ is an orthonormal list for an inner product space $V$ . Suppose that we took a linear combination of these:

α_{1} e_{1} + \dots + α_{n} e_{n}

if we wonder what the norm of this vector:

\begin{aligned} ‖ \underset{orthogonal}{\underset{⏟}{α_{1} e_{1} + \dots}} + \underset{orthogonal}{\underset{⏟}{α_{n} e_{n}}} ‖^{2} & = ‖ α_{1} e_{1} + \dots ‖^{2} + ‖ α_{n} e_{n} ‖^{2} \end{aligned}

which we repeat $n - 1$ times to get:

‖ α_{1} e_{1} + \dots + α_{n} e_{n} ‖^{2} = \sum_{i = 1}^{k} ‖ α_{i} e_{i} ‖^{2}

So taking the square roots, we notice that finding norms of vectors in this format allows us to find the norms very easily (just add the sums of the separate normals).