Chapter 6 - Inner Product Spaces

6.A: Inner Products and Norms

Inner Products

Think of vectors in $R^{2}$ or $R^{3}$ . Specifically, think of distances in these spaces. We denote the norm of $x$ , via $| | x | |$ where for $x = (x_{1}, x_{2})$ we have $| | x | | = \sqrt{x_{1}^{2} + x_{2}^{2}}$ . Similarly if $x = (x_{1}, x_{2}, x_{3})$ then we have $| | x | | = \sqrt{x_{1}^{2} + x_{2}^{2} + x_{3}^{2}}$ . In $R^{n}$ we have:

| | x | | = \sqrt{x_{1}^{2} + \dots + x_{n}^{2}}

The norm itself isn't linear on $R^{n}$ , but to inject linearity we define:

dot product

For $x, y \in R^{n}$ the dot product of $x, y$ , denoted $x \cdot y$ , is defined by:

x \cdot y = x_{1} y_{1} + \dots + x_{n} y_{n}

where $x = (x_{1}, . . ., x_{n})$ and $y = (y_{1}, . . ., y_{n})$ .

Keep in mind the dot product is a binary operator: $\cdot : R^{n} \times R^{n} \to R$ . Obviously $x \cdot x = | | x | |^{2}$ . We notice the following properties of this dot product:

$x \cdot x \geq 0$ for all $x \in R^{n}$ . (Distance is always a positive quantity)
$x \cdot x = 0$ iff $x = 0$ . (The distance $| | x | |$ to the origin is 0 when we have the zero vector)
For $y \in R^{n}$ fixed, the map from $R^{n}$ to $R$ that sends $x \in R^{n}$ to $x \cdot y$ is linear. (Namely here $y \cdot (v + u) = y \cdot v + y \cdot u$ , which is distributive property)
$x \cdot y = y \cdot x$ for all $x, y \in R^{n}$ (commutativity)

The inner product is a generalization of this dot product. But keep in mind that the properties above are true mainly for real spaces, but we need something to deal with complex spaces. Namely, if $λ = a + b i$ where $a, b \in R$ then:

$| λ | = \sqrt{a^{2} + b^{2}}$
$\overset{―}{λ} = a - b i$
$| λ |^{2} = λ \overset{―}{λ}$

So for $z = (z_{1}, . . ., z_{n}) \in C^{n}$ we have the norm as:

| | z | | = \sqrt{| z_{1} |^{2} + \dots + | z_{n} |^{2}}

Notice we need $| z_{i} |^{2}$ instead of $z_{i}^{2}$ since it's possible for $z_{i}^{2}$ to be a negative number (which is bad under a square root, as in the example $z = (i)$ then $| | z | | = i$ in that case). Note that:

| | z | |^{2} = z_{1} \overset{―}{z_{1}} + \dots + z_{n} \overset{―}{z_{n}}

You want to think of $| | z | |^{2}$ as the inner product of $z$ with itself, similar to the dot product above. As such, then this implies that if $w$ is defined similar to $z$ then the inner product of $w$ with $z$ is:

w_{1} \overset{―}{z_{1}} + \dots + w_{n} \overset{―}{z_{n}}

If $w$ and $z$ are swapped, we get the complex conjugate, suggesting that the inner product of $w$ with $z$ is the complex conjugate of the inner product of $z$ and $w$ .

Some things before we define the inner product:

If $λ \in C$ then $λ \geq 0$ implies that $λ$ is both real and nonnegative.
We use $⟨ u, v ⟩$ to denote the inner product.

inner product

An inner product on $V$ is a function that takes each ordered pair $(u, v)$ of elements of $V$ to a number $⟨ u, v ⟩ \in F$ and has the following properties:

(positivity): $⟨ v, v ⟩ \geq 0$ for all $v \in V$
(definiteness): $⟨ v, v ⟩ = 0$ iff $v = 0$
(additivity in the first slot): $⟨ u + v, w ⟩ = ⟨ u, w ⟩ + ⟨ v, w ⟩$ for all $u, v, w \in V$
(homogeneity in the first slot): $⟨ λ u, v ⟩ = λ ⟨ u, v ⟩$ for all $λ \in F$ and all $u, v \in V$
(conjugate symmetry): $⟨ u, v ⟩ = \overset{―}{⟨ v, u ⟩}$ for all $u, v \in V$

Since any real is it's own complex conjugate, if we're dealing with a real vector space then the last condition just says that $⟨ u, v ⟩ = ⟨ v, u ⟩$ for all $u, v \in V$ .

An Inner Product on Functions

An inner product can be defined on the vector space of continuous real-valued functions over the interval $[- 1, 1]$ by:

⟨ f, g ⟩ = \int_{- 1}^{1} f (x) g (x) d x

Another Inner Product on Functions

An inner product can be defined on $P (R)$ by:

⟨ p, q ⟩ \int_{0}^{\infty} p (x) q (x) e^{- x} d x

inner product space

An inner product space is a vector space $V$ along with an inner product on $V$ .

If you're given $V = R^{n}$ you can usually assume that $⟨ \cdot ⟩$ refers to the standard dot product we talked about earlier (called the Euclidean Inner Product).

For the sake of brevity we make the following assumption:

V

For the rest of this chapter, $V$ denotes an inner product space over $F$ .

Note the abuse of language here. $V$ itself is an inner product space, meaning it talks about it's own vectors space $V$ (same name, different thing), with an obvious (from context) inner product.

Note

Note that the inner product from the examples were "obvious" because they took the idea of the Euclidean dot product, of multiplying similar numbers and adding them up, and wrapped the adding up part into an integral, which means essentially the same thing.

Basic Properties of an Inner Product

(a) For each fixed $u \in V$ the function that takes $v$ to $⟨ v, u ⟩$ is linear from $V$ to $F$
(b) $⟨ 0, u ⟩ = 0$ for every $u \in V$
(c) $⟨ u, 0 ⟩ = 0$ for every $u \in V$
(d) $⟨ u, v + w ⟩ = ⟨ u, v ⟩ + ⟨ u, w ⟩$ for all $u, v, w \in V$
(e) $⟨ u, λ v ⟩ = \overset{―}{λ} ⟨ u, v ⟩$ for all $λ \in F$ and $u, v \in V$

Proof
(a): This comes from the conditions of additivity in the first slot and homogeneity in the first slot in the definition of an inner product.

(b): Follows from (a) and the result that every linear map takes $0$ to $0$ .

(c): Follows from (a) and the conjugate symmetry property in the definition of an inner product.

(d): Suppose $u, v, w \in V$ . Then:

\begin{aligned} ⟨ u, v + w ⟩ & = \overset{―}{⟨ v + w, u ⟩} \\ = \overset{―}{⟨ v, u ⟩ + ⟨ w, u ⟩} \\ = \overset{―}{⟨ v, u ⟩} + \overset{―}{⟨ w, u ⟩} \\ = ⟨ u, v ⟩ + ⟨ u, w ⟩ \end{aligned}

(e): Suppose $λ \in F$ and $u, v \in V$ then:

\begin{aligned} ⟨ u, λ v ⟩ & = \overset{―}{⟨ λ v, u ⟩} \\ = \overset{―}{λ ⟨ v, u ⟩} \\ = \bar{λ} \overset{―}{⟨ v, u ⟩} \\ = \bar{λ} ⟨ u, v ⟩ \end{aligned}

☐

Norms

Our initial desire was to define distances for other spaces. Now we see that the inner product determines this norm:

norm,

| | v | |

For $v \in V$ the norm of $v$ , denoted $| | v | |$ , is defined by:

| | v | | = \sqrt{⟨ v, v ⟩}

For instance, the norm of $R^{n}$ is:

| | x | | = \sqrt{x_{1}^{2} + \dots + x_{n}^{2}}

Example

In the vector space of continuous real-valued functions on $[- 1, 1]$ with the inner product given from Chapter 6 - Inner Product Spaces#^b56893 is:

| | f | | = \sqrt{\int_{- 1}^{1} (f (x))^{2} d x}

Basic Properties of the norm

Suppose $v \in V$ :

(a) $| | v | | = 0$ iff $v = 0$
(b) $| | λ v | | = | λ | | | v | |$ for all $λ \in F$

Proof
(a): Comes from the fact that $⟨ v, v ⟩ = 0$ iff $v = 0$ from the properties of the inner product.

(b): Suppose $λ \in F$ , then:

\begin{aligned} | | λ v | |^{2} & = ⟨ λ v, λ v ⟩ \\ = λ ⟨ v, λ v ⟩ \\ = λ \bar{λ} ⟨ v, v ⟩ \\ = | λ |^{2} | | v | |^{2} \end{aligned}

Taking the square roots of both sides finishes the proof.
☐

Notice that the proof above used the norm squared. In general, it's better to do proofs in this way, because the norm is never negative (so the $\pm$ we usually get gets nullified).

orthogonal

Two vectors $u, v \in V$ are called orthogonal if $⟨ u, v ⟩ = 0$ .

Notice that the order here doesn't matter since even in a complex vector space the complex conjugate of $0$ is $0$ , so $⟨ u, v ⟩ = 0 = ⟨ v, u ⟩$ .

In HW 7 - Inner Product Spaces#13 we show that if $u, v \in R^{2}$ are non-zero then we get that:

⟨ u, v ⟩ = | | u | | | | v | | \cos (θ)

where $θ$ is the angle between $u$ and $v$ (thinking of $u$ and $v$ as arrows pointing from the origin). Thus, the two vectors are othogonal, using the Euclidean inner product, iff $\cos (θ) = 0$ or when $θ = π / 2$ or equivalent. Thus, we're able to take the words perpendicular and orthogonal as meaning the same thing.

Orthogonality and

0

(a) $0$ is orthogonal to every vector in $V$
(b) $0$ is the only vector in $V$ that is orthogonal to itself

Proof
(a): Part (b) from Chapter 6 - Inner Product Spaces#^6bc3f4 states that $⟨ 0, u ⟩ = 0$ for every $u \in V$ .

(b): If $v \in V$ and $⟨ v, v ⟩ = 0$ then $v = 0$ by the definition of the inner product.
☐
For the special case $V = R^{2}$ the proof of the next thing is super classic. But now, we can abstract it away!

Pythagorean Theorem

Suppose $u, v$ are orthogonal vectors in $V$ . Then:

| | u + v | |^{2} = | | u | |^{2} + | | v | |^{2}

Proof

\begin{aligned} | | u + v | |^{2} & = ⟨ u + v, u + v ⟩ \\ = ⟨ u, u ⟩ + ⟨ u, v ⟩ + ⟨ v, u ⟩ + ⟨ v, v ⟩ \end{aligned}

Here notice that $⟨ u, v ⟩ = \overset{―}{⟨ v, u ⟩}$ so then:

\begin{aligned} ⟨ u, v ⟩ + ⟨ v, u ⟩ & = ⟨ u, v ⟩ + \overset{―}{⟨ u, v ⟩} \\ = 2 R (⟨ u, v ⟩) \end{aligned}

In general. In this case since $u, v$ are orthogonal then the real part becomes $0$ , so then:

| | u + v | |^{2} = ⟨ u, u ⟩ + ⟨ v, v ⟩ = | | u | |^{2} + | | v | |^{2}

☐
Suppose $u, v \in V$ with $v \neq 0$ . We would like to write $u$ as a scalar multiple of $v$ plus a vector $w$ orthogonal to $v$ , as suggest by:

Pasted image 20240308221135.png

We want to get the vector $w$ above. Notice here that, where $c \in F$ :

u = c v + \underset{w}{\underset{⏟}{(u - c v)}}

we want to get the vector perpendicular to the value above, so namely:

0 = ⟨ u - c v, v ⟩ = ⟨ u, v ⟩ - c | | v | |^{2}

Since $v \neq 0$ then $| | v | |^{2} \neq 0$ so then solve for $c$ :

c = \frac{⟨ u, v ⟩}{| | v | |^{2}}

So then plug it back in to get:

u = \frac{⟨ u, v ⟩}{| | v | |^{2}} v + (u - \frac{⟨ u, v ⟩}{| | v | |^{2}} v)

Thus, we proved the following:

An orthogonal decomposition

Suppose $u, v \in V$ , with $v \neq 0$ . Then set $c = \frac{⟨ u, v ⟩}{| | v | |^{2}}$ and $w = u - \frac{⟨ u, v ⟩}{| | v | |^{2}} v$ . Then:

⟨ w, v ⟩ = 0

and:

u = c v + w

Cauchy-Schwarz Inequality

Suppose $u, v \in V$ . Then:

| ⟨ u, v ⟩ | \leq | | u | | | | v | |

This inequality is an equality iff one of $u, v$ is a scalar multiple of the other.

Proof
If $v = 0$ then we get an equality, so let $v \neq 0$ . Then consider the orthogonal decomposition:

u = \frac{⟨ u, v ⟩}{| | v | |^{2}} v + w

given by our above decomposition, where $w, v$ are orthogonal. By the Pythagorean Theorem:

\begin{aligned} | | u | |^{2} & = {‖ \frac{⟨ u, v ⟩}{| | v | |^{2}} ‖}^{2} + ‖ w ‖^{2} \\ = \frac{| ⟨ u, v ⟩ |^{2}}{‖ v ‖^{2}} + ‖ w ‖^{2} \\ \geq \frac{| ⟨ u, v ⟩ |^{2}}{‖ v ‖^{2}} \end{aligned}

Multiply both sides by $‖ v ‖^{2}$ then square root both sides to get the inequality above.

Notice equality only happens when $‖ w ‖^{2} = 0$ which only happens when $w = 0$ . But $w = 0$ iff $u$ is a multiple of $v$ via Chapter 6 - Inner Product Spaces#^315ed2, so then we only get equality iff $u$ is a scalar multiple of $v$ or $v$ is a scalar multiple of $u$ .
☐

Example

If $f, g$ are continuous real-valued functions on $[- 1, 1]$ , then:

{(\int_{- 1}^{1} f (x) g (x) d x)}^{2} \leq (\int_{- 1}^{1} (f (x))^{2} d x) (\int_{- 1}^{1} (g (x))^{2} d x)

Triangle Inequality

Suppose $u, v \in V$ . Then:

‖ u + v ‖ \leq ‖ u ‖ + ‖ v ‖

where we get equality iff one of $u, v$ is a non-negative multiple of the other.

Proof
We have:

\begin{aligned} ‖ u + v ‖^{2} & = ⟨ u + v, u + v ⟩ \\ = ⟨ u, u ⟩ + ⟨ v, v ⟩ + ⟨ u, v ⟩ + ⟨ v, u ⟩ \\ = ⟨ u, u ⟩ + ⟨ v, v ⟩ + 2 R (⟨ u, v ⟩) & (⟨ u, v ⟩ + \overset{―}{⟨ v, u ⟩} = 2 R (⟨ u, v ⟩)) \\ \leq ‖ u ‖^{2} + ‖ v ‖^{2} + 2 | ⟨ u, v ⟩ | \\ \leq ‖ u ‖^{2} + ‖ v ‖^{2} + 2 ‖ u ‖ ‖ v ‖ & (Triangle Inequality) \\ = (‖ u ‖ + ‖ v ‖)^{2} \end{aligned}

Square rooting both sides gives the identity. Notice that we have an equality only if we hav equality from the top to the bottom, requiring from both $\leq$ 's that:

⟨ u, v ⟩ = ‖ u ‖ ‖ v ‖

where notice that if one of $u, v$ are nonnegative multiples of the other, then we get the equation above. Conversely, if the equation holds, then the condition for equality for the Cauchy-Scharz Inequality (see Chapter 6 - Inner Product Spaces#^2ef5ba) implies that one of $u, v$ is a scalar multiple of the other, forcing the scalar in question to be nonnegative as needed.
☐
Similar to the triangle in equality, geometric interpretations suggest a parallelogram equality:

Pasted image 20240308225041.png

Parallelogram Equality

Suppose $u, v \in V$ . Then:

‖ u + v ‖^{2} + ‖ u - v ‖^{2} = 2 (‖ u ‖^{2} + ‖ v ‖^{2})

Proof
We have:

\begin{aligned} ‖ u + v ‖^{2} + ‖ u - v ‖^{2} & = ⟨ u + v, u + v ⟩ + ⟨ u - v, u - v ⟩ \\ = ‖ u ‖^{2} + ‖ v ‖^{2} + ⟨ u, v ⟩ + ⟨ v, u ⟩ + ‖ u ‖^{2} + ‖ v ‖^{2} - ⟨ u, v ⟩ - ⟨ v, u ⟩ \\ = 2 (‖ u ‖^{2} + ‖ v ‖^{2}) \end{aligned}

☐

6.B: Orthonormal Bases

orthonormal

A list of vectors is called orthonormal if each vector in the list has norm 1 and is orthogonal to all the other vectors in the list. In other words, a list $e_{1}, . . ., e_{m}$ of vectors in $V$ is orthonormal if:

⟨ e_{j}, e_{k} ⟩ = {\begin{cases} 1 & j = k \\ 0 & j \neq k \end{cases}

For instance, the standard basis in $F^{n}$ is an orthonormal list.

The norm of an orthonormal linear combination

If $e_{1}, . . ., e_{m}$ is an orthonormal list of vectors in $V$ , then:

‖ a_{1} e_{1} + \dots + a_{m} e_{m} ‖^{2} = | a_{1} |^{2} + \dots + | a_{m} |^{2}

for all $a_{i} \in F$ .

Proof
Since each $e_{j}$ has norm 1, then we can just apply Chapter 6 - Inner Product Spaces#^4c14b5 over $m - 1$ iterations.
☐

An orthonormal list is LI

Every orthonormal list of vectors is LI.

Proof
Suppose $e_{1}, . . ., e_{m}$ is an orthonormal list of vectors in $V$ and $a_{1}, . . ., a_{m} \in F$ such that:

a_{1} e_{1} + \dots + a_{m} e_{m} = 0

Then $| a_{1} |^{2} + \dots + | a_{m} |^{2} = 0$ from Chapter 6 - Inner Product Spaces#^845f7d. Thus, all $a_{j} = 0$ showing our list of vectors is LI.
☐

orthonormal basis

An orthonormal basis of $V$ is an orthonormal list of vectors in $V$ that is also a basis of $V$ .

For instance, the standard basis is an orthonormal basis of $F^{n}$ .

An orthonormal list of the right length is an orthonormal basis

Every orthonormal list of vectors in $V$ with length $dim (V)$ is an orthonormal basis of $V$

Proof
By Chapter 6 - Inner Product Spaces#^73b58d, and since we have the right number of vectors by having $dim (V)$ of them, then it's a basis.
☐
In general, given a basis $e_{1}, . . ., e_{n}$ of $V$ , and a vector $v \in V$ , we know that there's choices of scalars $a_{1}, . . ., a_{n} \in F$ such that:

v = a_{1} e_{1} + \dots + a_{n} e_{n}

But how do we find all these $a_{i}$ 's in an efficient manner? The next results will help us in doing that:

Writing a vector as a linear combination of orthonormal basis

Suppose $e_{1}, . . ., e_{n}$ is an orthonormal basis of $V$ and $v \in V$ . Then:

v = ⟨ v, e_{1} ⟩ e_{1} + \dots + ⟨ v, e n ⟩ e_{n}

and:

‖ v ‖^{2} = | ⟨ v, e_{1} ⟩ |^{2} + \dots + | ⟨ v, e_{n} ⟩ |^{2}

Proof
Because $e_{1}, . . ., e_{n}$ is a basis of $V$ , there are $a_{i}$ such that:

v = a_{1} e_{1} + \dots + a_{n} e_{n}

Since $e_{1}, . . ., e_{n}$ is orthonormal, taking the inner product of both sides with $e_{j}$ gives $⟨ v, e_{j} ⟩ = a_{j}$ . This shows the first equation of our lemma.

The second equation follows immediately from using the first equation with Chapter 6 - Inner Product Spaces#^845f7d.
☐
See Lecture 29 - More on Orthonormality#Gram Schmidt Process for a more in-depth look as to what's going on here.

We see how it's useful to have an orthonormal basis, so how do we get one? This is the Gram-Schmidt Procedure:

Gram-Schmidt Procedure

Suppose $v_{1}, . . ., v_{m}$ is a LI list of vectors from $V$ . Let $e_{1} = v_{1} / ‖ v_{1} ‖$ . For $j = 2, . . ., m$ , define $e_{j}$ inductively by:

e_{j} = \frac{v_{j} - ⟨ v_{j}, e_{1} ⟩ e_{1} - \dots - ⟨ v_{j}, e_{j - 1} ⟩ e_{j - 1}}{‖ v_{j} - ⟨ v_{j}, e_{1} ⟩ e_{1} - \dots - ⟨ v_{j}, e_{j - 1} ⟩ e_{j - 1} ‖}

Then $e_{1}, . . ., e_{m}$ is an orthonromal list of vectors in $V$ such that:

span (v_{1}, . . ., v_{j}) = span (e_{1}, . . ., e_{j})

for $j = 1, . . ., m$

Proof
We'll use induction over $j$ . Start with $j = 1$ . Notice that $span (v_{1}) = span (e_{1})$ since $v_{1}$ is a positive multiple of $e_{1}$ .

Suppose $1 < j < m$ and we have it that:

span (v_{1}, . . ., v_{j - 1}) = span (e_{1}, . . ., e_{j - 1})

Notice that $v_{j} \notin span (v_{1}, . . ., v_{j - 1})$ since $v_{1}, . . ., v_{m}$ is LI. Thus $v_{j} \notin span (e_{1}, . . ., e_{j - 1})$ by our inductive hypothesis. Hence, we are not dividing by 0 in the new definition of $e_{j}$ given by the lemma. Dividing a vector by its norm produces a new vector with norm 1, so $‖ e_{j} ‖ = 1$ .

Let $1 \leq k < j$ . Then:

\begin{aligned} ⟨ e_{j}, e_{k} ⟩ & = ⟨ \frac{v_{j} - ⟨ v_{j}, e_{1} ⟩ e_{1} - \dots - ⟨ v_{j}, e_{j - 1} ⟩ e_{j - 1}}{‖ v_{j} - ⟨ v_{j}, e_{1} ⟩ e_{1} - \dots - ⟨ v_{j}, e_{j - 1} ⟩ e_{j - 1} ‖}, e_{k} ⟩ \\ = \frac{⟨ v_{j}, e_{k} ⟩ - ⟨ v_{j}, e_{k} ⟩}{‖ v_{j} - ⟨ v_{j}, e_{1} ⟩ e_{1} - \dots - ⟨ v_{j}, e_{j - 1} ⟩ e_{j - 1} ‖} \\ = 0 \end{aligned}

Thus $e_{1}, . . ., e_{j}$ is an orthonormal list.

From the definition of $e_{j}$ given by the lemma, we see that $v_{j} \in span (e_{1}, . . ., e_{j})$ , and combining this information with the inductive hypothesis gives:

span (v_{1}, . . ., v_{j}) \subset span (e_{1}, . . ., e_{j})

Both lists are LI (the $v$ 's by hypothesis, the $e$ 's by orthonormality and Chapter 6 - Inner Product Spaces#^ef122d). Thus, both subspaces above have dimension $j$ , and hence they are equal, completing the proof.
☐

An Example

We'll find an orthonormal basis of $P_{2} (R)$ , where the inner product is given by:

⟨ p, q ⟩ \int_{- 1}^{1} p (x) q (x) d x

Apply Gram-Schmidt to the basis $1, x, x^{2}$ . To get started, we see that:

‖ 1 ‖ = \int_{- 1}^{1} 1^{2} d x = 2

Thus $‖ 1 ‖ = \sqrt{2}$ so then $e_{1} = \frac{\sqrt{2}}{2}$ .

Now the numerator for $e_{2}$ should be:

x - ⟨ x, e_{1} ⟩ e_{1} = x - \int_{- 1}^{1} x \frac{\sqrt{2}}{2} d x \cdot \frac{\sqrt{2}}{2} = x

We have:

‖ x ‖^{2} = \int_{- 1}^{1} x^{2} d x = \frac{2}{3}

Thus have $e_{2} = \sqrt{\frac{3}{2}} x$ . Now the numerator for $e_{3}$ is:

\begin{aligned} x^{2} & - ⟨ x^{2}, e_{1} ⟩ e_{1} - ⟨ x^{2}, e_{2} ⟩ e_{2} \\ = x^{2} - \sqrt{\frac{1}{2}} \int_{- 1}^{1} x^{2} \cdot \sqrt{\frac{1}{2}} d x - \sqrt{\frac{3}{2}} x \int_{- 1}^{1} x^{2} \cdot \sqrt{\frac{3}{2}} x d x \\ = x^{2} - \frac{1}{3} \end{aligned}

And then:

‖ x^{2} - 1 / 3 ‖^{2} = \int_{- 1}^{1} (x^{2} - 1 / 3)^{2} d x = \frac{8}{45}

Thus $e_{3} = \sqrt{\frac{45}{8}} (x^{2} - 1 / 3)$ .

Thus $e_{1}, e_{2}, e_{3}$ is $\sqrt{\frac{1}{2}}, \sqrt{\frac{3}{2}} x, \sqrt{\frac{45}{8}} (x^{2} - 1 / 3)$ , which is our orthonormal list of length 3 in our vector space. Hence, this orthonormal list is an orthonormal basis of $P_{2} (R)$ since it's LI (from orthonormality), and is of the right dimension.

Existence of orthonormal basis

Every finite-dimensional inner product space has an orthonormal basis.

Proof
If $V$ is finite-dimensional, then there's a basis $v_{1}, . . ., v_{n}$ for $V$ . Apply Gram-Schmidt to get an orthonormal list with length $dim (V) = n$ . This orthonormal list is LI, so it's an orthonormal basis of $V$ .
☐

Orthornormal list extends to orthonormal basis

Suppose $V$ is finite-dimensional. Then every orthonormal list of vectors in $V$ can be extended to an orthonormal basis of $V$ .

Proof
If $e_{1}, . . ., e_{m}$ is an orthonormal list of vectors in $V$ , then $e_{1}, . . ., e_{m}$ is LI. This list can also be extended to a basis as a result, to $e_{1}, . . ., e_{m}, v_{1}, . . ., v_{n}$ of $V$ . Applying Gram-Schmidt, we can an orthonormal list:

e_{1}, . . ., e_{m}, f_{1}, . . ., f_{n}

here the first $m$ vectors are unchanged since they are already orthonormal. The list above is an orthonormal basis of $V$ since it's the right length.
☐

Recall that a matrix is upper triangular if all the entries below the diagonal equal 0. From Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^2e8389, we would like to know whether there exists an orthonormal basis specifically, with respect to which we have an upper-triangular matrix.

Upper-triangular matrix with respect to orthonormal basis

Suppose $T \in L (V)$ . If $T$ has an upper-triangular matrix with respect to some basis of $V$ , then $T$ has an upper-triangular matrix with respect to some orthonormal basis of $V$ .

Proof
Suppose $T$ has an upper-triangular matrix with respect to some basis $v_{1}, . . ., v_{n}$ of $V$ . Then $span (v_{1}, . . ., v_{j})$ is invariant under $T$ for each $j = 1, . . ., n$ via Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^a5a043.

Apply the Gram-Schmidt Procedure to $v_{1}, . . ., v_{n}$ , producing an orthonormal basis $e_{1}, . . ., e_{n}$ of $V$ . Because:

span (e_{1}, . . ., e_{j}) = span (v_{1}, . . ., v_{j})

for each $j$ via our Gram-Schmidt procedure, we can conclude that $span (e_{1}, . . ., e_{j})$ is invariant under $T$ for each $j = 1, . . ., n$ . Thus, by our invariant property, $T$ has an upper-triangular matrix with respect to the orthonormal basis $e_{1}, . . ., e_{n}$ .
☐

Schur's Theorem

Suppose $V$ is a finite-dimensional complex vector space and $T \in L (V)$ . Then $T$ has an upper-triangular matrix with respect to some orthonormal basis of $V$ .

Proof
Recall that $T$ has an upper-triangular matrix with respect to some basis of $V$ via Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^2e8389. Apply Gram-Schmidt and Chapter 6 - Inner Product Spaces#^b22ab5.
☐

Linear Functionals on Inner Product Spaces

linear functional

A linear functional on $V$ is a linear map from $V$ to $F$ . In other words, a linear functional is an element of $L (V, F)$ .

For instance, the function $ϕ : F^{3} \to F$ given by:

ϕ (z_{1}, z_{2}, z_{3}) = 2 z_{1} - 5 z_{2} + z_{3}

is a linear function on $F^{3}$ . We could write this linear functional in the form:

ϕ (z) = ⟨ z, u ⟩

for all $z \in F^{3}$ where $u = (2, - 5, 1)$ .

We won't need to cover linear functionals for the time being, so we just end here!

6.C: Orthogonal Complements and Minimization Problems

(I'll see you back here next quarter!!!)