Chapter 1 - Vector Spaces

Intro

Linear Algebra is the study of linear maps on spaces. We'll look at using $R$ in conjunction with $C$ when looking at our scalar fields. Then we'll generalize using $R^{n}, C^{n}$ to a general notion of a vector space. Lastly, we'll talk about subspaces.

1.A : $R^{n}$ and $C^{n}$

Be familiar with the properties of $R$ and $C$ :

Complex Numbers

A complex number is an ordered pair $(a, b)$ where $a, b \in R$ but we write this as $a + b i$
The set of all complex numbers is denoted by $C = {a + b i : a, b \in R}$
Addition and mulitiplication on $C$ are defined by:
- $(a + b i) + (c + d i) = (a + c) + (b + d) i$
- $(a + b i) (c + d i) = (a c - b d) + (a d + b c) i$
- Here $a, b, c, d \in R$

If $a \in R$ we identify $a + 0 i$ with a real number $a$ . So $R \subseteq C$ . Usually $0 + b i$ is just $b i$ and $0 + 1 i = i$ . You can verify that $i^{2} = - 1$ by definition.

Complex numbers follow these properties:

Properties of Complex Arithmetic

For the following, let $α, β, λ \in C$ be arbitrary:

Commutativity: $α + β = β + α$ and $α β = β α$
Associativity: $(α + β) + λ = α + (β + λ)$ and $α (β λ) = (α β) λ$
Identities: $λ + 0 = λ$ and $λ 1 = λ$
Additive Inverse: For every $α \in C$ , there exists some number $γ \in C$ such that $α + γ = 0$
Multiplicative Inverse: For every $α i n C$ there is some number $λ \in C$ such that $α λ = 1$ .
Distributive Property: $\lambda(\alpha + \beta) = \lambda\alpha + \lambda\beta

We want to really emphasize the proof of showing one of these properties from solely the definitions alone:

Multiplicative Associativity

Show that $α β = β α$ for any $α, β \in C$ .

Proof
Let $α = a + b i \in C$ and $β = c + d i \in C$ be arbitrary ( $a, b, c, d \in R$ ). Then notice that using the rules of complex multiplication and addition we get the following:

\begin{aligned} α β & = (a + b i) (c + d i) \\ = (a c - b d) + (a d + b c) i \end{aligned}

And similarly:

\begin{aligned} β α & = (c + d i) (a + b i) \\ = (c a - d b) + (c d + d a) i \end{aligned}

The commutativity of multiplication and addition of $R$ shows that $α β = β α$
☐

- α

, subtraction, division

Let $α, β \in C$ :

Let $- α$ denote the additive inverse of $α$ . So $- α$ is the unique complex number such that:

α + (- α) = 0

Subtraction on $C$ is defined by:

β - α = β + (- α)

For $α \neq 0$ let $\frac{1}{α}$ denote the multiplicative inverse of $α$ . Thus $\frac{1}{α}$ is the unique complex number such that:

α (\frac{1}{α}) = 1

Division on $C$ is defined by:

\frac{β}{α} = β (\frac{1}{α})

For future reference, to allow a lot of future theorems and proofs to apply to both $R$ and $C$ , we denote the generic field $F$ to stand for either or:

Elements of $F$ are called scalars.
For $α \in F$ and $m \in N$ we define $α^{m} = \underset{m times}{\underset{⏟}{α \cdot α \cdot . . . \cdot α}}$ . Clearly $(α^{m})^{n} = α^{m n}$ and $(α β)^{m} = α^{m} β^{m}$ for all $α, β \in F$ .

Lists

We define $R^{2}, R^{3}$ as the following:

R^{2}, R^{3}

The set $R^{2} = {(x, y) : x, y \in R}$
$R^{3} = {(x, y, z) : x, y, z \in R}$

To allow the generalization of any $R^{n}$ we need to talk about lists:

list, length

Suppose $n \in N$ . A list of length $n$ is an ordered collection of $n$ elements, separated by commas:

(x_{1}, . . ., x_{n})

Two lists are equal iff they have the same length and the same elements in the same order.

Note that sometimes we don't specify the length of a list. But if the length of a list is infinite, as is $(x_{1}, x_{2}, . . .)$ , then it is not a list.

A list of length 0 is $()$ with length $n = 0$ . Lists are different from sets in that order matters for lists, and repetitions are allowed (whereas with sets that's not the case).

Lists vs. Sets

The lists $(3, 5)$ and $(5, 3)$ aren't equal, but the sets ${3, 5}$ and ${5, 3}$ are equal.
The lists $(4, 4)$ and $(4, 4, 4)$ aren't equal, but the sets ${4, 4}$ and ${4, 4, 4}$ are (equivalent to ${4}$ ).

Thus we get:

F^{n}

$F^{n}$ is the set of all lists of length $n$ of elements of $F$ . For $(x_{1}, . . ., x_{n}) \in F^{n}$ and $j \in {1, . . ., n}$ we say that $x_{j}$ is the $j$ -th coordinatte of $(x_{1}, . . ., x_{n})$ .

We geometrically cannot comprehend $R^{n}$ when $n \geq 4$ , or for $C^{n}$ when $n \geq 2$ , so we have to consider the algebraic manipulations instead:

Addition in

F^{n}

Addition in $F^{n}$ is defined by adding corresponding coordinates:

(x_{1}, . . ., x_{n}) + (y_{1}, . . ., y_{n}) = (x_{1} + y_{1}, . . ., x_{n} + y_{n})

Commutativity of addition in

F^{n}

If $x, y \in F^{n}$ then $x + y = y + x$ .

Proof
Let $x = (x_{1}, . . ., x_{n}), y = (y_{1}, . . ., y_{n}) \in F^{n}$ be arbitrary. Then:

\begin{aligned} x + y & = (x_{1}, . . ., x_{n}) + (y_{1}, . . ., y_{n}) \\ = (x_{1} + y_{1}, . . ., x_{n} + y_{n}) \\ = (y_{1} + x_{1}, . . ., y_{n} + x_{n}) \\ = (y_{1}, . . ., y_{n}) + (x_{1}, . . ., x_{n}) \\ = y + x \end{aligned}

☐

We define $0 = (0, . . ., 0)$ for any $n$ entries for the corresponding $F^{n}$ .

We may want to draw these vectors in $R^{2}$ but keep in mind that this is more of a visual aid to aid in what's going on. Normally we won't care about the geometry of, say 5-dimensional space, but we'll need to use the algebra to actually verify and prove that our intuition is correct in those spaces:

Additive Inverse in

F^{n}

For $x \in F^{n}$ the additive inverse of $x$ , denoted $- x$ , is the vector $- x \in F^{n}$ such that:

x + (- x) = 0

In other words, if $x = (x_{1}, . . ., x_{n})$ then $- x = (- x_{1}, . . ., - x_{n})$

Geometrically, it means flipping the original vector 180 degrees:

Scalar Multiplication in

F^{n}

The product of a number $λ$ and a vector in $F^{n}$ is computed by multiplying each coordinate of the vector by $λ$ :

λ (x_{1}, . . ., x_{n}) = (λ x_{1}, . . ., λ x_{n})

here $λ \in F$ and $(x_{1}, . . ., x_{n}) \in F^{n}$

Here $λ$ scales the size of $\vec{x}$ by a factor, and if $λ$ is negative then it also flips the vector and scales it:

Fields

A field is a set containing at least two distinct elements called 0 and 1, along with operations of addition and multiplication satisfying all the properties listed in Chapter 1 - Vector Spaces#^4af3e2. So then $R, C$ are fields, as well as $Q$ . The set ${0, 1}$ is also a field with the usually operations except we define $1 + 1 = 0$ . in this set

1.B: Definition of Vector Space

Addition, Scalar Multiplication

An addition on a set $V$ is a function that assigns an element $u + v \in V$ to each pair of elements $u, v \in V$ .
A scalar multiplication on a set $V$ is a function that assigns an element $λ v \in V$ to each $λ \in F$ and each $v \in V$ .

Vector Space

A vector space is a set $V$ along with an addition on $V$ and a scalar multiplication on $V$ such that the following properties hold:

Commutativity: $u + v = v + u$ for all $u, v \in V$ .
Associativity: $(u + v) + w = u + (v + w)$ and $(a b) v = a (b v)$ for all $u, v, w \in V$ and all $a, b \in F$ .
Additive Identity: There exists an element $0 \in V$ such that $v + 0 = v$ for all $v \in V$ .
Additive Inverse: For every $v \in V$ there exists $w \in V$ such that $v + w = 0$ .
Multiplicative Identity: $1 v = v$ for all $v \in V$ .
Distributive Properties: $a (u + v) = a u + a v$ and $(a + b) v = a v + b v$ for all $a, b \in F$ and all $u, v \in V$ .

Elements of a vector space are called vectors or points. We say, based on the scalar multiplication operation, that $V$ is a vector space over $F$ . A vector space over $R$ is a real vector space, and over $C$ is a complex vector space.

Example

$F^{\infty}$ is defined to be the set of all sequences of elements of $F$ :

F^{\infty} = {(x_{1}, x_{2}, . . .) : x_{j} \in F \forall j \in N}

Addition and scalar multiplication are as expected. $F^{\infty}$ is a vector space, as you can verify.

The book references Lecture 2 - Vector Spaces (cont.)#^86311a as an example. But notice that we can think about how $F^{n}$ and $F^{\infty}$ are really similar. They are special cases of the vector space $F^{S}$ since a list of length $n$ of numbers in $F$ can be thought of as a function from ${1, 2, 3, . . ., n}$ to $F$ , and a sequence of numbers in $F$ can be thought of as a function from the set of positive integers to $F$ . Ie: $F^{n}$ is really just $F^{{1, 2, . . ., n}}$ and $F^{\infty}$ is just $F^{{1, 2, . . .}}$ .

Unique Additive Identity

A vector space $V$ has a unique additive identity $\vec{0}$ .

Proof
We already know that $\vec{0}$ exists via the definition. Suppose $\vec{0}$ and $\vec{0^{'}}$ are both additive identities for our vector space $V$ . Then:

\vec{0^{'}} = \vec{0^{'}} + \vec{0} = \vec{0} + \vec{0^{'}} = \vec{0}

The first equality comes from applying the definition of the additive identity, and same with the last equality. The middle equality comes from the commutativity of vector addition. Thus, $\vec{0} = \vec{0^{'}}$ .
☐

Unique Additive Inverse

Every element $v \in V$ in vector space $V$ has a unique additive inverse.

Proof
Suppose $V$ is a vector space, and let $v \in V$ be arbitrary. We know from the definitions that $w \in V$ , the additive inverse of $v$ , exists. Now, consider $w', w' \in V $, and suppose they're different. Notice that:

w = w + 0 = w + (v + w^{'}) = (w + v) + w^{'} = 0 + w^{'} = w^{'}

☐
For notation's sake, we say that if $v \in V$ , then the additive inverse is just $- v \in V$ . Further, instead of writing $w + (- v)$ we'll just write $w - v$ instead. Further, in general since we are restating that $V$ is a vector space, in general we implicitly say that $V$ is a vector space over $F$ .

The number

0

times a vector

$0 v = 0$ for every $v \in V$ .

Proof
For $v \in V$ , we have:

0 v = (0 + 0) v = 0 v + 0 v

Add the additive inverse of $0 v$ to both sides to get that $0 = 0 v$ as expected.
☐
Clearly here we note that the vectors and scalars are getting mixed, so we'll usually denote the vectors with the arrow ( $\vec{0}$ ) while we'll use no arrow for scalars ( $0$ ).

A number times the vector

\vec{0}

$a \vec{0} = \vec{0}$ for all $a \in F$ .

Proof
For $a \in F$ , we have:

a \vec{0} = a (\vec{0} + \vec{0}) = a \vec{0} + a \vec{0}

Again, add the additive inverse of $a \vec{0}$ to both sides to get that $\vec{0} = a \vec{0}$ as expected.
☐

The number -1 times a vector

$(- 1) \vec{v} = \vec{- v}$ for every $v \in V$ .

Proof
Let $\vec{v} \in V$ be arbitrary. Then:

\vec{v} + (- 1) \vec{v} = 1 \vec{v} + (- 1) \vec{v} = (1 + (- 1)) \vec{v} = 0 \vec{v} = \vec{0}

Thus $(- 1) \vec{v}$ is the additive inverse for $\vec{v}$ , so then it's $\vec{- v}$ .
☐

1.C: Subspaces

Subspace

A subset $U \subseteq V$ is called a subspace of $V$ if $U$ is also a vector space (using the same addition and scalar multiplication as on $V$ ).

For instance, the set ${(x_{1}, x_{2}, 0 : x_{1}, x_{2} \in F} \subseteq F^{3}$ , is a subspace. We can check for if a subspace is still a subspace if we have the following:

Conditions for a subspace

A subset $U \subseteq V$ is a subspace of $V$ iff $U$ satisfies the following three conditions:

Additive Identity: $\vec{0} \in U$
Closed Under Addition: $\vec{u}, \vec{w} \in U$ implies that $\vec{u} + \vec{w} \in U$
Closed Under Scalar Multiplication: $a \in F$ and $\vec{u} \in U$ implies that $a \vec{u} \in U$

Proof
If $U$ is a subspace of $V$ then $U$ satisfies the three conditions above (by definition). Conversely, suppose $U$ satisfies the three conditions above. The first condition ensures that the additive identity of $V$ is in $U$ . The second condition ensures addition makes sense on $U$ . Similarly for the 3rd condition.

If $\vec{u} \in U$ , then $- \vec{u} \in U$ since $(- 1) \vec{u} = - \vec{u}$ by Chapter 1 - Vector Spaces#^21334b, so the additive inverse always exists. Commutativity, associativity, distributivity, and so on all are satisfied in $U$ because they are satisfied in $V$ .

Therefore, $U$ is a subspace of $V$ .
☐

Sums of Subspaces

Sum of Subsets

Suppose $U_{1}, . . ., U_{m}$ are subsets of $V$ . The sum of $U_{1}, . . ., U_{m}$ , denoted $U_{1} + . . . + U_{m}$ , is the set of all possible sums of elements of $U_{1}, . . ., U_{m}$ . More precisely:

U_{1} + \dots + U_{m} = {u_{1} + \dots + u_{m} : u_{1} \in U_{1}, . . ., u_{m} \in U_{m}}

Let's look at an example:

Example

Suppose $U$ is the set of all elements of $F^{3}$ whose second and third coordinates equal $0$ , and $W$ is the set of all elements of $F^{3}$ whose first and third coordinates equal $0$ :

U = {(x, 0, 0) \in F^{3} : x \in F}, W = {(0, y, 0) \in F^{3} : y \in F}

Then:

U + W = {(x, y, 0) : x, y \in F}

Example

Suppose that $U = {(x, x, y, y) \in F^{4} : x, y \in F}, W = {(x, x, x, y) \in F^{4} : x, y \in F^{4}}$ . Then:

U + W = {(x, x, y, z) \in F^{4} : x, y, z \in F}

As a fact, if $U_{1}, . . ., U_{m}$ are subspaces of $V$ , then $U_{1} + . . . + U_{m}$ is the smallest subspace of $V$ containing $U_{1}, . . ., U_{m}$ .

Proof
For clarity, call $U_{c} = U_{1} + \dots + U_{m}$ . Clearly $\vec{0} \in U_{c}$ , and that $U_{c}$ is closed under addition and scalar multiplication. So therefore Chapter 1 - Vector Spaces#^2bacbe implies that $U_{c}$ is a subspace of $V$ .

Now for size. Clearly $U_{1}, . . ., U_{m}$ all are contained in $U_{c}$ , since we can just choose ${\vec{u}}_{n} \in U_{n}$ solely from any $U_{i}$ above, and the rest of the vectors would be ${\vec{0}}_{j} \in U_{j}$ where $i \neq j$ . This justifies the largeness of our set.

Conversely, every subspace of $V$ containing $U_{1}, . . ., U_{m}$ contains $U_{c}$ (because subspaces must contain all finite sums of their elements), so $U_{c}$ is as small as can be (we can't make any other subspaces that are smaller without missing some $U_{i}$ ).

Therefore, $U_{c}$ must be the smallest subspace of $V$ containing $U_{1}, . . ., U_{m}$ .
☐

Direct Sums

Suppose $U_{1}, . . ., U_{m}$ are subspaces of $V$ . Every element of $U_{1} + \dots + U_{m}$ can be written in the form:

u_{1} + \dots + u_{m}

where each $u_{j}$ is in $U_{j}$ . We create a new definition using this idea:

Direct Sum

Suppose $U_{1}, . . ., U_{m}$ are subspaces of $V$ .

The sum $U_{1} + \dots + U_{m}$ is called a direct sum if each element of $U_{1} + \dots + U_{m}$ can be written in only one way as a sum $u_{1} + \dots + u_{m}$ , where each $u_{j} \in U_{j}$ .
If $U_{1} + \dots + U_{m}$ is a direct sum, then $U_{1} \oplus \dots \oplus U_{m}$ denotes $U_{1} + \dots + U_{m}$ where the $\oplus$ notation serves as an indication that this is a direct sum.

This looks like a confusing definition, but the examples really help here:

An Example

Suppose $U$ is the subspace of $F^{3}$ whose vectors' last coordinate equals 0, and $W$ is the subspace of $F^{3}$ whose vectors' first two coordinates equals 0. Then $F^{3} = U \oplus V$ .

A Counterexample

Let $U_{1} = {(x, y, 0) \in F^{3} : x, y \in F}$ , $U_{2} = {(0, 0, z) \in F^{3} : z \in F}$ , and $U_{3} = {(0, y, y) \in F^{3} : y \in F}$ . $U_{1} + U_{2} + U_{3}$ is not a direct sum because every vector in $F^{3}$ can be composed of:

(x, y, z) = (x, y, 0) + (0, 0, z) + (0, 0, 0)

Where the first vector is from $U_{1}$ , the second from $U_{2}$ , and the third from $U_{3}$ . But $F^{3}$ isn't the direct sum of these sets since $(0, 0, 0)$ can be written in two different ways as a sum $u_{1} + u_{2} + u_{3}$ with each $u_{j} \in U_{j}$ . Specifically:

(0, 0, 0) = (0, 1, 0) + (0, 0, 1) + (0, - 1, - 1)

or even:

(0, 0, 0) = (0, 0, 0) + (0, 0, 0) + (0, 0, 0)

Where again the first vector is from $U_{1}$ , the second from $U_{2}$ , and the third from $U_{3}$ .

The definition of direct sum requires that every vector in the sum have a unique representation as an appropriate sum.

This definition can be hard to use to prove things, but luckily there's an easy condition that we can use instead:

Condition for a direct sum

Suppose $U_{1}, \dots, U_{m}$ are subspaces of $V$ . Then $U_{1} + \dots + U_{m}$ is a direct sum iff the only way to write $0$ as a sum $u_{1} + \dots + u_{m}$ , where each $u_{j} \in U_{j}$ . is by taking each $u_{j} = 0$ .

I would write the proof, but it's very non-intuitive and the proof can be seen in Year3/Winter2024/MATH306-LinearAlgebraII/2015_Book_LinearAlgebraDoneRight.pdf#pages=23 if you're so curious.

Lastly, we have an important definition that you'll want to definitely use!

Direct sum of two subspaces

Suppose $U$ and $W$ are subspaces of $V$ . Then $U + W$ is a direct sum iff $U \cap W = {0}$ .

Proof
First suppose that $U + W$ is a direct sum. If $v \in U \cap W$ then $0 = v + (- v)$ , where $v \in U$ and $- v \in W$ . By the unique representation of $0$ as the sum of a vector in $U$ and a vector in $W$ , we have $v = 0$ . Thus $U \cap W = {0}$ .

For the other way, suppose $U \cap W = {0}$ . To prove that $U + W$ is a direct sum, suppose $u \in U, w \in W$ , and $0 = u + w$ . We only need to show that $u = w = 0$ . The equation implies that $u = - w \in W$ , so then $u \in U \cap W$ . Hence $u = 0$ , so therefore $w = 0$ .
☐

Note that this only works for two subspaces!!! If you have any more you are considering, then it doesn't work.

Intro

1.A : Rn and Cn

Lists

Fields

1.B: Definition of Vector Space

1.C: Subspaces

Sums of Subspaces

Direct Sums

1.A : $R^{n}$ and $C^{n}$