Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces

5.A: Invariant Subspaces

Suppose $T \in L (V)$ . If we have a direct sum decomposition:

V = U_{1} \oplus \dots \oplus U_{m}

where each $U_{j}$ is a proper subspace of $V$ , then to understand $T : V \to V$ then we only need to understand each $T |_{U_{j}}$ , restricting $T$ to our domain for a subspace $U_{j}$ . But the problem is that $T |_{U_{j}}$ may not map back to $U_{j}$ itself, a requirement for being an operator (see Chapter 3 - Linear Maps#Operators as to why this is the case). So we must only consider decompositions of $V$ that allow that property to arise.

invariant subspace

Suppose $T \in L (V)$ . A subspace $U$ of $V$ is an invariant under $T$ if $u \in U$ implies that $T u \in U$ .

This allows $U$ to be invariant when $T |_{U}$ is an operator on $U$ .

Some examples of invariant subspaces under $T$ over $V$ :

${\vec{0}}$ since $u = \vec{0}$ and $T \vec{0} = \vec{0} \in U$
$V$ since if $v \in V$ then already $T v \in V$
$null (T)$ since if $v \in null (T)$ then $T v = \vec{0} \in null (T)$
$range (T)$ since if $v \in range (T)$ then $T u \in range (T)$ .

So the big question is if $T \in L (V)$ has invariant subspaces that aren't the ones above, as these are the so-called trivial invariants. Notice that this is because $null (T)$ may very well be the first set ${\vec{0}}$ and likewise $range (T) = V$ is also a possibility.

Eigenvalues/vectors

Let's look at invariant subspaces of dimension 1, the simplest one we know.

Take any $v \in V$ where $v \neq 0$ and let $U$ be:

U = {λ v : λ \in F} = span (v)

Then $U$ is a 1-dimensional subspace of $V$ . If $U$ is invariant under and operator $T \in L (V)$ then $T v \in U$ and thus there is some scalar $λ \in F$ such that:

T v = λ v

Conversely, if $T v = λ v$ for some $λ \in F$ then $span (v)$ is a 1-dimensional subspace of $V$ invariant under $T$ .

eigenvalue

Suppose $T \in L (V)$ . A number $λ \in F$ is called an eigenvalue of $T$ if there exists $v \in V$ such that $v \neq 0$ and $T v = λ v$ .

Thus $T$ has a 1-dimensional invariant subspace iff $T$ has an eigenvalue.

Lemma

Suppose $V$ is finite-dimensional and $T \in L (V)$ and $λ \in F$ . Then the following are equivalent.

$λ$ is an eigenvalue of $T$ ;
$T - λ I$ is not injective;
$T - λ I$ is not surjective;
$T - λ I$ is not invertible.

The proof of this is in Lecture 22 (online) - Invariant Subspaces#^dedd4b.

eigenvector

Suppose $T \in L (V)$ and $λ \in F$ is an eigenvalue of $T$ . A vector $v \in V$ is an eigenvector of $T$ corresponding to $λ$ if $v \neq 0$ and $T v = λ v$ .

Since $T v = λ v \Rightarrow (T - λ I) v = \vec{0}$ then a vector $v \in V$ is an eigenvector corresponding to $λ$ iff $v \in null (T - λ I)$ .

Linearly Independent Eigenvectors

Let $T \in L (V)$ . Suppose $λ_{1}, . . ., λ_{m}$ are distinct eigenvalues of $T$ and $v_{1}, . . ., v_{m}$ are corresponding eigenvectors. Then $v_{1}, . . ., v_{m}$ is LI.

The proof is at Lecture 22 (online) - Invariant Subspaces#^4f4e13.

Lemma

Suppose $V$ is a finite-dimensional vector space. Then each linear transformation $T$ on $V$ has at most $dim (V)$ distinct eigenvalues and corresponding eigenvectors.

The proof is at Lecture 22 (online) - Invariant Subspaces#^cecd64.

Restriction and Quotient Operators

If $T \in L (V)$ and $U$ is a subspace of $V$ invariant under $T$ , then $U$ determines two other operators $T |_{U} \in L (U)$ and $T / U \in L (V / U)$ :

T |_{U}

and

T / U

Suppose $T \in L (V)$ and $U$ is a subspace of $V$ invariant under $T$

The restriction operator $T |_{U} \in L (U)$ is defined by:

T |_{U} (u) = T u

for $u \in U$ .

The quotient operator $T / U \in L (V / U)$ is defined by:

(T / U) (v + U) = T v + U

for $v \in V$ .

Here we have it that $v + U$ is the set $v + U = {v + u : u \in U}$ . Notice if $v + U = w + U$ then $T v + U = T w + U$ . Suppose $v + U = w + U$ then $v - w \in U$ . To be honest, we're not covering these, so see section 3.D and 3.E later on when I get to there to use these in more detail.

5.B: Eigenvectors and Upper-Triangular Matrices

Polynomials Applied to Operators

T^{m}

Suppose $T \in L (V)$ and $m$ is a positive integer.

$T^{m}$ is defined by $\underset{m times}{\underset{⏟}{T \circ T \circ \dots \circ T}}$
$T^{0}$ is defined to be the identity operator $I$ on $V$ .
If $T$ is invertible with inverse $T^{- 1}$ then $T^{- m}$ is defined by: $T^{- m} = (T^{- 1})^{m}$ .

p (T)

Suppose $T \in L (V)$ and $p \in P (F)$ is a polynomial given by:

p (z) = a_{0} + a_{1} z + a_{2} z^{2} + \dots + a_{m} z^{m}

for $z \in F$ . Then $p (T)$ is the operator defined by:

p (T) = a_{0} I + a_{1} T + a_{2} T^{2} + \dots + a_{m} T^{m}

If we fix $T \in L (V)$ then the function from $P (F)$ to $L (V)$ given by $p \mapsto p (T)$ is linear.

product of polynomials

If $p, q \in P (F)$ then $p q \in P (F)$ is the polynomial defined by:

(p q) (z) = p (z) q (z)

for $z \in F$

We get some properties as a result:

Multiplicative properties

Suppose $p, q \in P (F)$ and $T \in L (V)$ then:

$(p q) (T) = p (T) q (T)$
$p (T) q (T) = q (T) p (T)$

The proof is pretty straightforward, but if you're curious check out Year3/Winter2024/MATH306-LinearAlgebraII/2015_Book_LinearAlgebraDoneRight.pdf#page=144.

Existence of Eigenvalues

Operators on complex vector spaces have an eigenvalue

Every operator on a finite-dimensional, nonzero, complex vector space has an eigenvalue.

We talked about the proof in Lecture via Lecture 23 - Polynomial Operator#^9bac66.

Upper-Triangular Matrices

matrix of an operator,

M (T)

Suppose $T \in L (V)$ and $v_{1}, . . ., v_{n}$ is a basis of $V$ . The matrix of $T$ with respect to this basis is the $n \times n$ matrix:

M (T) = (\begin{matrix} A_{1, 1} & \dots & A_{1, n} \\ ⋮ & ⋮ \\ A_{n, 1} & \dots & A_{n, n} \end{matrix})

whose entries $A_{j, k}$ are defined by:

T v_{k} = A_{1, k} v_{1} + \dots + A_{n, k} v_{n}

If the basis is not clear from context, $M (T, β)$ is used instead.

A really simple basis to have is to have the first vector be nothing but zeroes, after a non-zero term $λ$ , then repeating this process:

(\begin{matrix} λ & \dots \\ 0 & * & \dots \\ 0 & 0 & * \\ ⋮ & ⋱ \end{matrix})

If $V$ is finite-dimensional and a complex vector space, we know that an eigenvalue like $λ$ exists, with its associated eigenvector that we use as the first vector of the basis. We can then repeat with the smaller matrix with the first row and column removed, and get the same thing, all the way down!

diagonal of a matrix

The diagonal of a square matrix consists of the entries along the line from the upper left corner to the bottom right corner.

upper-triangular matrix

A matrix is called upper-triangular if all the entries below the diagonal equal 0.

Typically they have the shape of:

(\begin{matrix} λ_{1} & * \\ ⋱ \\ 0 & λ_{n} \end{matrix})

Conditions for upper-triangular matrix

Suppose $T \in L (V)$ and $v_{1}, . . ., v_{n}$ is a basis of $V$ . Then the following are equivalent:

The matrix of $T$ ( $M (T)$ ) with respect to $v_{1}, . . ., v_{n}$ is upper-triangular
$T v_{j} \in span (v_{1}, . . ., v_{j})$ for each $j = 1, . . ., n$
$span (v_{1}, . . ., v_{j})$ is invariant under $T$ for each $j = 1, . . ., n$

To see the intuition for the proof below, refer to Lecture 23 - Polynomial Operator#Upper Triangular for an overview of how the proof works.

Proof
(1) equals (2) from the definitions. (3) implies (2) is easy to show, so to finish we only prove (2) implies (c).

Suppose (2). Fix $j = 1, . . ., n$ . From (2) we know that:

\begin{array}{r} T v_{1} \in span (v_{1}) \subset span (v_{1}, . . ., v_{j}) \\ T v_{2} \in span (v_{1}, v_{2}) \subset span (v_{1}, . . ., v_{j}) \\ ⋮ \\ T v_{j} \in span (v_{1}, . . . v_{j}) \end{array}

Thus if $v$ is a linear combination of $v_{1}, . . ., v_{j}$ then:

T v \in span (v_{1}, . . ., v_{j})

So then $span (v_{1}, . . ., v_{j})$ is invariant under $T$ .
☐
Now we really want to show that each operator on a finite-dimensional complex vector space has a matrix of the operator with only 0's below the diagonal.

Over

C

, every operator has an upper-triangular matrix

Suppose $V$ is a finite-dimensional complex vector space and $T \in L (V)$ . Then $T$ has an upper-triangular matrix with respect to some basis of $V$ .

See the proof at Lecture 24 - Finishing Eigenstuff#^98fa4c. Notice that to construct such a proof, one usually creates it from the back, then proves it forwards. In this case, $U$ seems arbitrary, but notice that it's a good choice because the bottom right matrix at the end of the proof specifically is guaranteed to have $λ$ 's on the diagonal.

Determination of invertibility from upper-triangular matrix

Suppose $T \in L (V)$ has an upper triangular matrix with respect to some basis of $V$ . Then $T$ is invertible iff all the entries on the diagonal of that upper-triangular matrix are non-zero.

Again the proof we did in lecture is at Lecture 24 - Finishing Eigenstuff#^d13947 for the first half, and Lecture 25 - Eigenvalues (cont.)#^a6bb24 for the more detailed proof of the second half.

This will be used for a really important lemma, so hold onto your butts.

Determination of eigenvalues from upper-triangular matrix

Suppose $T \in L (V)$ has an upper-triangular matrix with respect to some basis of $V$ . Then the eigenvalues of $T$ are precisely the entries on the diagonal of that upper-triangular matrix.

The proof is at Lecture 25 - Eigenvalues (cont.)#^8972ff.

5.C: Eigenspaces and Diagonal Matrices

diagonal matrix

A diagonal matrix is a square matrix that is 0 everywhere except possibly along the diagonal.

For instance:

(\begin{matrix} 8 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & 5 \end{matrix}) = diag (8, 5, 5)

Note that every diagonal matrix is upper triangular, so all the properties from the previous section apply here.

Using Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^bb525c, if an operator $T$ has a diagonal matrix with respect to some basis, then the entries along the diagonal are precisely the eigenvalues themselves.

eigenspace,

E (λ, T)

Suppose $T \in L (V)$ and $λ \in F$ . The eigenspace of $T$ corresponding to $λ$ , denoted $E (λ, T)$ , is defined by:

E (λ, T) = null (T - λ I)

In other words, $E (λ, T)$ is the set of all eigenvectors of $T$ corresponding to $λ$ , along with the $\vec{0}$ vector.

Notice that for $T \in L (V)$ and $λ \in F$ , the eigenspace $E (λ, T)$ is a subspace of $V$ because the null space of each linear map on $V$ is a subspace of $V$ . Thus, these definitions imply that $λ$ is an eigenvalue of $T$ iff $E (λ, T) \neq {\vec{0}}$ .

For example, consider the $diag (8, 5, 5)$ matrix above. Here:

\begin{aligned} E (8, T) & = span (v_{1}) \\ E (5, T) & = span (v_{2}, v_{3}) \end{aligned}

Now notice that if we restrict $T$ to just $E (λ, T)$ , then all that happens is vectors get scaled:

Sum of eigenspaces is a direct sum

Suppose $V$ is finite-dimensional and $T \in L (V)$ . Suppose also that $λ_{1}, . . ., λ_{m}$ are distinct eigenvalues of $T$ . Then:

E (λ_{1}, T) + \dots + E (λ_{m}, T)

is a direct sum, and furthermore,

dim (E (λ_{1}, T)) + \dots + dim (E (λ_{m}, T)) \leq dim (V)

Proof
To show that $E (λ_{1}, T) + \dots + E (λ_{m}, T)$ is a direct sum, suppose:

u_{1} + \dots + u_{m} = 0

where each $u_{j} \in E (λ_{j}, T)$ . Since eigenvectors corresponding to distinct eigenvalues are LI (from Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^45cb5f), then each $u_{j} = 0$ . Thus, we have a direct sum via Chapter 1 - Vector Spaces#^038942. Now:

\begin{aligned} dim (E (λ_{1}, T)) + \dots + dim (E (λ_{m}, T)) & = dim (E (λ_{1}, T) \oplus \dots \oplus E (λ_{m}, T)) \\ \leq dim (V) \end{aligned}

where the $=$ comes from HW 2 - Finite Dimensional Vector Spaces#16.
☐

diagonalizaable

An operator $T \in L (V)$ is diagonalizable if the operator has a diagonal matrix with respect to some basis of $V$ .

For instance, consider $T \in L (R^{2})$ where:

T (x, y) = (41 x + 7 y, - 20 x + 74 y)

The matrix of $T$ with respect to the standard basis of $R^{2}$ is:

(\begin{matrix} 41 & 7 \\ - 20 & 74 \end{matrix})

which isn't diagonal. But $T$ is diagonalizable, since $T$ with respect to the basis $(1, 4), (7, 5)$ is:

(\begin{matrix} 69 & 0 \\ 0 & 46 \end{matrix})

which is diagonal.

Conditions equivalent to diagonalizability

Suppose $V$ is finite-dimensional and $T \in L (V)$ . Let $λ_{1}, . . ., λ_{m}$ denote the distinct eigenvalues of $T$ . Then the following are equivalent:

$T$ is diagonalizable;
$V$ has a basis consisting of eigenvectors of $T$ ;
There exist 1-dimensional subspaces $U_{1}, . . ., U_{n}$ of $V$ , each invariant under $T$ , such that:

V = U_{1} \oplus \dots \oplus U_{n}

$V = E (λ_{1}, T) \oplus \dots \oplus E (λ_{m}, T)$ ;
$dim (V) = dim (E (λ_{1}, T)) + \dots + dim (E (λ_{m}, T))$

Proof
(a) = (b). An operator $T \in L (V)$ has a diagonal matrix $diag (λ_{1}, . . ., λ_{n})$ with respect to basis $v_{1}, . . ., v_{n}$ iff $T v_{j} = λ_{j} v_{j}$ for each $j$ . Thus, (a) and (b) are equivalent above.

(b) $\to$ (c). Suppose (b). Then $V$ has a basis $v_{1}, . . ., v_{n}$ consisting of eigenvectors of $T$ . For each $j$ , let $U_{j} = span (v_{j})$ . Each $U_{j}$ is 1-dimensional subspace that is invariant under $T$ . Because $v_{1}, . . ., v_{n}$ is a basis of $V$ , each vector in $V$ can be written uniquely as a linear combination of $v_{1}, . . ., v_{n}$ . Hence, each vector in $V$ can be written uniquely as a sum $u_{1} + \dots + u_{n}$ , where each $u_{j} \in U_{j}$ . Thus, $V = U_{1} \oplus \dots \oplus U_{n}$ , so (b) implies (c).

(c) $\to$ (b). Suppose (c); so there are 1-dimensional subspaces $U_{1}, . . ., U_{n}$ of $V$ , each invariant under $T$ , such that $V = U_{1} \oplus \dots \oplus U_{n}$ . For each $j$ , let $v_{j}$ be a non-zero vector in $U_{j}$ . Then each $v_{j}$ is an eigenvector of $T$ , as $U_{j}$ is 1-dimensional. Hence, because each vector in $V$ is uniquely written as a sum $u_{1} + \dots + u_{n}$ where each $u_{j} \in U_{j}$ (so each $u_{j}$ is a scalar multiple of $v_{j}$ ), we see that $v_{1}, . . ., v_{n}$ is a basis of $V$ . So (c) implies (b).

We know that (a,b,c) are all equivalent. We finish showing (b) implies (d), (d) implies (e), and (e) implies (b)

Suppose (b) holds; thus $V$ has a basis consisting of eigenvectors of $T$ . Hence, every vector in $V$ is a linear combinations of eigenvectors of $T$ , which means that:

V = E (λ_{1}, T) + \dots + E (λ_{m}, T)

so then Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^c87cad shows that (d) holds specifically.

(d) implies from (e) comes straight, again, from HW 2 - Finite Dimensional Vector Spaces#16.

Finally, suppose (e) holds; so:

dim (V) = dim (E (λ_{1}, T)) + \dots + dim (E (λ_{m}, T))

Choose a basis of each $E (λ_{j}, T)$ , put all these bases together to form a list $v_{1}, . . ., v_{n}$ of eigenvectors of $T$ , where $n = dim (V)$ via our equation of dimension above. To show $v_{1}, . . ., v_{n}$ is LI, suppose:

a_{1} v_{1} + \dots + a_{n} v_{n} = 0

For each $j = 1, . . ., m$ let $u_{j}$ denote the sum of all the terms $a_{k} v_{k}$ such that $v_{k} \in E (λ_{j}, T)$ . Thus each $u_{j}$ is in $E (λ_{j}, T)$ , and:

u_{1} + \dots + u_{m} = 0

Because eigenvectors corresponding to distinct eigenvalues are LI via Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^45cb5f, this implies each $u_{j} = 0$ . Because each $u_{j} = \sum a_{k} v_{k}$ where the $v_{k}$ 's are chosen to be a basis of $E (λ_{j}, T)$ , this implies that all $a_{k} = 0$ . Thus $v_{1}, . . ., v_{n}$ is LI, and hence a basis of $V$ (since we have the right number of vectors). Thus (e) implies (b).
☐

Okay that was a long proof. Feel free to digest this, then move onto the next thing.

The sad thing though is that not all $T \in L (V)$ have diagonalizable matrices $M (T)$ . For example, if $T \in L (C^{2})$ where:

T (w, z) = (z, 0)

is not diagonalizable. $0$ is the only eigenvalue of $T$ and furthermore $E (0, T) = {(w, 0) \in C^{2} : w \in C}$ , so all (b - e) in our theorem above fail, so (a) of it fails and thus $T$ isn't diagonalizable.

However, we can guaruntee that we can be diagonalizable if we follow the following lemma:

Enough eigenvalues implies diagonalizablility

If $T \in L (V)$ has $dim (V)$ distinct eigenvalues, then $T$ is diagonalizable.

Proof
Suppose $T \in L (V)$ has $dim (V)$ distinct eigenvalues $λ_{1}, . . ., λ_{dim (V)}$ . For each $j$ , let $v_{j} \in V$ be eigenvector corresponding to the eigenvalue $λ_{j}$ . Because eigenvectors corresponding to distinct eigenvalues are LI via Chapter 5 - Eigenvalues, Eigenvectors, and Invariant Subspaces#^45cb5f, then $v_{1}, . . ., v_{dim (V)}$ is LI. A LI list of $dim (V)$ is a basis of $V$ , so $v_{1}, . . ., v_{dim (V)}$ is a basis of $V$ . With respect to this basis consisting of eigenvectors, $T$ has a diagonal matrix.
☐
Note that the converse is not true; namely, we could have not all distinct eigenvalues while still being diagonalizable.