Chapter 3 - Linear Maps

3.A: The Vector Space of Linear Maps

Linear Map

A linear map from $V$ to $W$ is a function $T : V \to W$ such that:

$T (u + v) = T u + T v$ for all $u, v \in V$ (additivity)
$T (λ v) = λ (T v)$ for all $λ \in F, v \in V$ . (homogeneity)

Examples include:

Identity map
Differentiation
Integration
Multiplication by $x^{2}$
...

Linear Maps and Basis of Domain

Suppose $v_{1}, . . ., v_{n}$ is a basis of $V$ and $w_{1}, . . ., w_{n} \in W$ . Then there exists a unique linear map $T : V \to W$ such that:

T v_{j} = w_{j}

for each $j \in (1, . . ., n)$ .

Algebraic Operations on $L (V, W)$

Addition and scalar multiplication on

L (V, W)

Suppose $S, T \in L (V, W)$ and $λ \in F$ . The sum $S + T$ and the product $λ T$ are the linear maps from $V$ to $W$ defined by:

(S + T) (v) = S v + T v, (λ T) (v) = λ (T v)

As a result, $L (V, W)$ is a vector space (the zero linear map $0 v = 0$ is $\vec{0}$ here).

For the rest of this section suppose $U$ is a vector space:

Product of Linear Maps

If $T \in L (U, V)$ and $S \in L (V, W)$ then the product $S T \in L (U, W)$ is:

(S T) (u) = S (T (u))

Algebraic Properties of products of Linear Maps

Linear maps are:

Associative: $(T_{1} T_{2}) T_{3} = T_{1} (T_{2} T_{3})$
Identity: $T I = I T = T$
Distributive Properties: $(S_{1} + S_{2}) T = S_{1} T + S_{2} T$ and $S (T_{1} + T_{2}) = S T_{1} + S T_{2}$

Linear maps take

\vec{0}

\vec{0}

Suppose $T$ is a linear map from $V \to W$ . Then $T (0) = 0$

Proof
By additivity:

T (0) = T (0 + 0) = T (0) + T (0)

So adding the additive inverse to both sides gives $T (0) = 0$ .
☐

3.B: Null Spaces and Ranges

null space,

null (T)

For $T \in L (V, W)$ the null space of $T$ , denoted $null (T)$ , is the subset of $V$ consisting of those vectors that $T$ maps to ${\vec{0}}_{W}$ . So $null (T) = {v \in V : T v = {\vec{0}}_{W}}$ .

Some examples include:

The zero map $T v = \vec{0}$ for any $v \in V$ has $null (T) = V$ .
Suppose $D$ is the differentiation map from $P (R)$ to itself, where $D p = p^{'}$ . Only $D p = 0$ when $p = c$ for some constant $c \in R$ . As such, $null (D) = {c \in P (R)}$
Suppose $T \in L (F^{\infty}, F^{\infty})$ is the backward shift given by: $T (x_{1}, x_{2}, . . .) = (x_{2}, . . .)$ then clearly $T (x_{1}, . . .) = 0$ iff $x_{2}, x_{3}, . . . = 0$ so then $null (T) = {(a, 0, 0, . . .) : a \in F}$ .

The nullspace is a subspace

Suppose $T \in L (V, W)$ . Then $null (T)$ is a subspace of $V$ .

Proof
Since $T$ is linear, then $T (\vec{0}) = \vec{0}$ by Chapter 3 - Linear Maps#^c97a61. Thus, $\vec{0} \in null (T)$ .

Suppose $u, v \in null (T)$ . Then:

T (u + v) = T u + T v = \vec{0} + \vec{0} = \vec{0}

Similarly, we can show closure under scalar multiplication.
☐

injective

A function $T : V \to W$ is injective if $T u = T v$ implies that $u = v$

Injectivity is equivalent to null space equals

{\vec{0}}

Let $T \in L (V, W)$ . Then $T$ is injective iff $null T = {\vec{0}}$ .

Proof
Consider $\to$ . We already know ${\vec{0}} \subset null T$ since it's a subspace of $V$ and thus must contain the zero-vector. We'll show the other way now, so suppose $v \in null (T)$ . Then:

T v = \vec{0} = T (0)

and because $T$ is injective, then that implies that $v = \vec{0}$ , completing the proof for this direction.

Consider $\leftarrow$ . Suppose $u, v \in V$ are arbitrary, and that $T u = T v$ . Then:

\vec{0} = T u - T v = T (u - v)

thus $u - v \in null) (T)$ which equals ${\vec{0}}$ , so then $u - v = \vec{0}$ so $u = v$ so then $T$ is injective.
☐

Range

For $T : V \to W$ the range of $T$ is the subset of $W$ consisting of those vectors that are of the form $T v$ for some $v \in V$ :

range (T) = {T v : v \in V}

Some examples include:

If $T$ is the zero map, ie $T v = 0$ for all $v$ , then $range (T) = {\vec{0}}$ .
The differentiation operator has the range of $P (R)$ .

The range is a subspace

If $T \in L (V, W)$ then the $range (T)$ is a subspace of $W$ .

Proof

$\vec{0} \in range (T)$ as $T (0) = 0$ .
Since $T$ is linear then $T (v_{1} + v_{2}) = T v_{1} + T v_{2} = w_{1} + w_{2} \in range (T)$
A similar argument is made for closure under s. mult.
So $range (T)$ is a subspace of $W$ .
☐

surjective

A function $T : V \to W$ is surjective if its range equals $W$ .

For instance, $D$ is surjective since the range of $D$ is the entire polynomial space.

Fundamental Theorem of Linear Maps

Fundamental Theorem of Linear maps

Suppose $V$ is finite-dimensional and $T \in L (V, W)$ . Then $range (T)$ is finite dimensional and:

dim (V) = dim (null (T)) + dim (range (T))

A proof is seen in Lecture 14 - Linear Transformations ++#^6e22f5.

Some interesting properties:

Suppose $V, W$ are finite-dimensional vector spaces where $dim (V) > dim (W)$ . Then no linear maps from $V$ to $W$ is injective. Use the lemma above to show this which implies that $dim null (T) > 0$ so then there's other basis vectors that aren't zero in this space, so $T$ is not injective via Chapter 3 - Linear Maps#^ba1b7d.
If instead $dim (V) < dim (W)$ then no linear map from $V$ to $W$ is surjective as using the above theorem implies that $dim (range (T)) < dim (W)$ so then $range (T) \neq W$ so then $T$ is not surjective.

Some other Notes

We can use this idea to show when and when not a system of linear equations has a solution. There's a lot more information on relationships to systems of linear equations, which you can find at Year3/Winter2024/MATH306-LinearAlgebraII/2015_Book_LinearAlgebraDoneRight.pdf#page=65 and onward. Some lemmas:

Summary of Lemmas

Homogeneous sytem of linear equations with more variables than equations has nonzero solutions.
An inhomogeneous system of linear equations with more equations than variables has no solution for some choice of the constant terms.

3.C: Matrices

Representing matrices

matrix,

A_{j, k}

Let $m, n$ denote positive integers. An $m \times n$ matrix is a rectangular array of elements of $F$ with $m$ rows and $n$ columns:

A = (\begin{matrix} A_{1, 1} & \dots & A_{1, n} \\ ⋮ & ⋮ \\ A_{m, 1} & \dots & A_{m, n} \end{matrix})

where $A_{j, k}$ denotes the entry in row $j$ and column $k$ of $A$ .

matrix of a linear map,

M (T)

Suppose $T \in L (V, W)$ and $v_{1}, . . ., v_{n}$ is a basis of $V$ and $w_{1}, . . ., w_{m}$ is a basis of $W$ . The matrix of $T$ with respect to these bases is the $m \times n$ matrix $M (T)$ whose entries $A_{j, k}$ are defined by:

T v_{k} = A_{1, k} w_{1} + \dots + A_{m, k} w_{m}

If the bases are not clear from the context, then the notation $M (T, (v_{1}, . . ., v_{n}), (w_{1}, . . ., w_{m}))$ is used.

You can use the following to help construct the matrix:
Pasted image 20240207235858.png

Thus:

T v_{k} = \sum_{j = 1}^{m} A_{j, k} w_{j}

Addition and Scalar Multiplication of Matrices

We've done matrix addition before and scalar multiplication:

$(A + C)_{j, k} = A_{j, k} + C_{j, k}$
$(α A)_{j, k} = α A_{j, k}$

As a result, then:

Matrix sum of linear maps

Suppose $S, T \in L (V, W)$ . Then $M (S + T) = M (S) + T$ .

and:

The matrix of a scalar times a linear map

Suppose $λ \in F$ and $T \in L (V, W)$ . Then $M (λ T) = λ M (T)$ .

As such, we define this as a new vector space $F^{m, n}$ . As a lemma:

dim (F^{m, n}) = m n

Matrix Multiplication

See Lecture 17 - Continuing Matrices#^1be6e5. It explains where matrix multiplication comes from.

Matrix Multiplication

Suppose $A$ is an $m \times n$ matrix and $C$ is $n \times p$ . Then $A C$ is defined to be the $m \times p$ matrix whose entry in row $j$ and column $k$ is given by:

(A C)_{j, k} = \sum_{r = 1}^{n} A_{j, r} C_{r, k}

Essentially, this is a dot product of the $j$ -th row of $A$ and the $k$ -th column of $C$ . Note that matrix multiplication isn't commutative.

The matrix of the product of linear maps

If $T \in L (U, V)$ and $S \in L (V, W)$ then $M (S T) = M (S) M (T)$ .

The proof of this is actually the derivation of matrix multiplication via Lecture 17 - Continuing Matrices#^1be6e5.

Notation

Suppose $A$ is $m \times n$ . Then:

If $1 \leq j \leq m$ then $A_{j, \cdot}$ denotes the $1 \times n$ matrix consisting of row $j$ of $A$ .
If $1 \leq k \leq n$ then $A_{\cdot, k}$ denotes the $k \times 1$ matrix consisting of column $k$ of $A$ .

As such, then multiplication of $A$ left by $C$ via $A C$ is:

(A C)_{j, k} = A_{j, \cdot} \cdot C_{\cdot, k}

for $j \in (1, . . ., m$ ) and $k \in (1, . . ., p)$ .

Column of matrix product equals matrix times column

Suppose $A_{m \times n}$ and $C_{n \times p}$ ,. Then:

(A C)_{\cdot, k} = A C_{\cdot, k}

For example, consider:
Pasted image 20240208001646.png
Pasted image 20240208001656.png
Pasted image 20240208001731.png

We generalize the results from the examples above to give:

Linear Combinations of Columns

Suppose $A$ is an $m \times n$ matrix and $c = [\begin{matrix} c_{1} \\ ⋮ \\ c_{n} \end{matrix}]$ is an $n \times 1$ matrix. Then:

A c = c_{1} A_{\cdot, 1} + \dots + c_{n} A_{\cdot, n}

In other words, $A c$ is a linear combination of the columns of $A$ , with the scalars that multiply the columns coming from $c$ .

3.D: Invertibility and Isomorphic Vector Spaces

Invertible Linear Maps

invertible, inverse

A linear map $T \in L (V, W)$ is called invertible if there exists a linear map $S \in L (W, V)$ such that $S T$ equals the identity map on $V$ and $T S$ equals the identity map on $W$ .
A linear map $S \in L (W, V)$ satisfying $S T = I_{V}$ and $T S = I_{W}$ is called the inverse of $T$ .

Note that the inverse is unique:

Proof
Suppose $T$ is invertible and $S_{1}, S_{2}$ are inverses of $T$ . Then:

S_{1} = S_{1} I_{W} = S_{(} T S_{2}) = (S_{1} T) S_{2} = I_{V} S_{2} = S_{2}

Thus $S_{1} = S_{2}$ .
☐
Since $S$ is unique, we call it $T^{- 1}$ to denote it as $T$ 's inverse.

Invertibility equals bijectivity

A linear map is invertible iff it is bijective (injective and surjective).

A proof is at Year3/Winter2024/MATH306-LinearAlgebraII/2015_Book_LinearAlgebraDoneRight.pdf#page=81. Some examples of non-invertible linear maps include:

The multiplication of $x^{2}$ from $P (R)$ to itself is not invertible because it is not surjective (1 is not in the range).
The backward shift linear map from $F^{\infty}$ to itself is not invertible because it is not injective ( $(1, 0, 0, . . .)$ is in the null space).

Isomorphic Vector Spaces

isomorphism, isomorphic

An isomorphism is an invertible linear map. Two vector spaces are isomorphic if there is an isomorphism from one vector space onto the other one.

We are essentially relabeling all the $v \in V$ to some new label $T v = w \in W$ . Hence, this is why certain spaces are so similar to one another, like $R^{3}$ and $P_{2} (R)$ . You might wonder if dimension has anything to do with it.

Dimension shows vector spaces as isomorphic

Two finite-dimensional vector spaces over $F$ are isomorphic iff they have the same dimension.

Proof
First, suppose $V$ and $W$ are isomorphic finite-dimensional vector spaces. Thus there exists an isomorphism $T : V \to W$ . Because $T$ is invertible then $null (T) = {\vec{0}}$ and $range (T) = W$ So then:

dim (V) = dim (null (T)) + dim (range (T)) = 0 + dim (W)

Via Chapter 3 - Linear Maps#^15829c. Thus $dim (V) = dim (W)$ .

For the other way, suppose $V, W$ are finite-dimensional vector spaces with the same dimension. Then $v_{1}, . . ., v_{n}$ and $w_{1}, . . ., w_{n}$ are bases for $V, W$ respectively. Let $T \in L (V, W)$ be defined by:

T (c_{1} v_{1} + \dots + c_{n} v_{n}) = c_{1} w_{1} + \dots + c_{n} w_{n}

Then $T$ is a well-defined linear map because $v_{1}, . . ., v_{n}$ is a basis for $V$ via Chapter 3 - Linear Maps#^9a80a8. Also, $T$ is surjective since $w_{1}, . . ., w_{n}$ spans $W$ . Furthermore $null (T) = {\vec{0}}$ since $w_{1}, . . ., w_{n}$ is LI, so $T$ is injective. Because $T$ is both injective and surjective, it's an isomorphism, so $V, W$ are isomorphic as desired.
☐

Thus any vector space $V$ where $dim (V) = n$ is isomorphic to $F^{n}$ . If $v_{1}, . . ., v_{n}$ is a basis for $V$ and $w_{1}, . . ., w_{m}$ is a basis for $W$ then for each $T \in L (V, W)$ we have a matrix $M (T) \in F^{m, n}$ . So in other words, once bases have been set for $V, W$ then $M$ is a function from $L (V, W)$ to $F^{m, n}$ , so since $M$ via Chapter 3 - Linear Maps#^bfb9a4, then $M$ is invertible.

L (V, W)

and

F^{m, n}

are isomorphic

Suppose $v_{1}, . . ., v_{n}$ is a basis of $V$ and $w_{1}, . . ., w_{m}$ is a basis of $W$ . Then $M$ is an isomorphism between $L (V, W)$ and $F^{m, n}$ .

Proof
We already noted that $M$ is linear. We need to prove that $M$ is injective and surjective.

With injectivity, if $T \in L (V, W)$ and $M (T) = \vec{0}$ then $T v_{k} = \vec{0}$ for all $k = 1, . . ., n$ and because $v_{1}, . . ., v_{n}$ is a basis of $V$ this implies $T = 0$ , showing $M$ as injective via Chapter 3 - Linear Maps#^ba1b7d.

For surjectivity, suppose $A \in F^{m, n}$ . Let $T$ be the linear map from $V$ to $W$ such that:

T v_{k} = \sum_{j = 1}^{m} A_{j, k} w_{j}

for $k = 1, . . ., n$ (see Chapter 3 - Linear Maps#^9a80a8). Obviously then $M (T)$ equals $A$ and thus the range of $M$ equals $F^{m, n}$ as desired.
☐

dim L (V, W) = (dim (V)) (dim (W))

Suppose $V, W$ are finite dimensional. Then $L (V, W)$ is finite dimensional where:

dim L (V, W) = (dim (V)) (dim (W))

This comes from Chapter 3 - Linear Maps#^0f4054, Chapter 3 - Linear Maps#^3e5a28, and Chapter 3 - Linear Maps#^ee27e8.

Linear Maps Thought of as Matrix Multiplication

matrix of a vector,

M (v)

Suppose $v \in V$ and $v_{1}, . . ., v_{n}$ is a basis for $V$ . The matrix of $v$ with respect to this basis is the $n$ -by-1 matrix:

M (v) = (\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{n} \end{matrix})

where $c_{1}, . . ., c_{n}$ are the scalars such that:

v = c_{1} v_{1} + \dots + c_{n} v_{n}

Notice that the matrix of $v$ depends on the given basis $v_{1}, . . ., v_{n}$ , and usually it's clear from context.

Some examples:

The matrix of $2 - 7 x + 5 x^{3}$ with respect to the standard basis of $P_{3} (R)$ is:

(\begin{matrix} 2 \\ - 7 \\ 0 \\ 5 \end{matrix})

The matrix of a vector $x \in F^{n}$ with respect to the standard basis is obtained by writing the coordinates of $x$ as the entries in an $n$ -by-1 matrix. So if $x = (x_{1}, . . ., x_{n})$ then:

M (x) = (\begin{matrix} x_{1} \\ ⋮ \\ x_{n} \end{matrix})

Note that the function $M$ is essentially a remapping/isomorphism from our vector $v \in V$ and gives us the related matrix of $v$ .

Recall that if $A$ is $m \times n$ then $A_{\cdot, k}$ denotes the $k$ -th column of $A$ , thought of as an $m \times 1$ matrix. Next, $M (v_{k})$ is computed with respect to the basis $w_{1}, . . ., w_{m}$ of $W$ :

M (T)_{\cdot, k} = M (v_{k})

Suppose $T \in L (V, W)$ and $v_{1}, . . ., v_{n}$ is a basis of $V$ and $w_{1}, . . ., w_{m}$ is a basis of $W$ . Let $1 \leq k \leq n$ . Then the $k$ -th column of $M (T)$ which is denoted by $M (T)_{\cdot, k}$ equals $M (v_{k})$ .

This follows directly from the definitions of both.

Linear Maps act like matrix multiplication

Suppose $T \in L (V, W)$ and $v \in V$ . Suppose $v_{1}, . . ., v_{n}$ is a basis of $V$ and $w_{1}, . . ., w_{m}$ is a basis of $W$ . Then:

M (T v) = M (T) M (v)

Proof
Suppose $v = c_{1} v_{1} + \dots + c_{n} v_{n}$ , where all $c_{i} \in F$ . Then:

T v = c_{1} T v_{1} + \dots + c_{n} T v_{n}

Hence:

\begin{aligned} M (T v) & = c_{1} M (T v_{1}) + \dots + c_{n} M (T v_{n}) \\ = c_{1} M (T)_{\cdot, 1} + \dots + c_{n} M (T)_{\cdot, n} \\ = M (T) v \end{aligned}

The first equality comes from linearity of $M$ , the second equality from Chapter 3 - Linear Maps#^fa150d, and the last one comes from Chapter 3 - Linear Maps#^93d307.
☐

Operators

operator,

L (V)

A linear map from a vector space to itself is called an operator. The notation $L (V)$ denotes the set of all operators on $V$ . In other words, $L (V) = L (V, V)$ .

Removing $W$ from this may suggest that we can remove one condition from bijectivity of $T$ , but recall from our earlier examples Chapter 3 - Linear Maps#^153ab0. But both of these are still not invertible despite being themselves operators. This is actually because both are infinite-dimensional vector spaces. But when they are finite dimensional, we get a remarkable result:

Injectivity is equivalent to surjectivity in finite dimensions

Suppose $V$ is finite-dimensional and $T \in L (V)$ . Then the following are equivalent:

$T$ is invertible
$T$ is injective
$T$ is surjective

Proof
Clearly if (i) holds then we get (ii, iii) for free. Suppose instead (ii) just holds. Then $T$ is injective so then $dim (null (T)) = 0$ so we just have the zero vector in the null space, so via the FTOLM:

dim (range (T)) = dim (V) - dim (null (T)) = dim (V)

so then $range (T) = V$ implying surjectivity. So we get (iii), which gives us (i).

Instead if (iii) holds then $T$ is surjective, so $range (T) = V$ so then via the FTOLM we have $dim (null (T)) = dim (V) - dim (range (T)) = 0$ so then the null space is just the zero vector, showing injectivity. Hence $T$ is invertible.
☐

3.A: The Vector Space of Linear Maps

Algebraic Operations on L(V,W)

3.B: Null Spaces and Ranges

Fundamental Theorem of Linear Maps

Some other Notes

3.C: Matrices

Representing matrices

Addition and Scalar Multiplication of Matrices

Matrix Multiplication

3.D: Invertibility and Isomorphic Vector Spaces

Invertible Linear Maps

Isomorphic Vector Spaces

Linear Maps Thought of as Matrix Multiplication

Operators

Algebraic Operations on $L (V, W)$