HW 6 - Polar & Singular Value Decomposition

2

Question

Give an example of $T \in L (C^{2})$ such that $0$ is the only eigenvalue of $T$ and the singular values of $T$ are $5, 0$ .

Proof
Consider:

T (z_{1}, z_{2}) = (5 z_{2}, 0)

Then:

T (e_{1}) = (0, 0), T (e_{2}) = 5 e_{1} \Rightarrow M (T) = [\begin{matrix} 0 & 5 \\ 0 & 0 \end{matrix}]

Notice that here $λ^{2} = 0 \Leftrightarrow λ = 0$ is the only eigenvalue, with eigenvector $e_{1}$ in this case. However, calculating $T^{*}$ :

\begin{aligned} ⟨ x, T^{*} y ⟩ & = ⟨ T x, y ⟩ \\ = ⟨ (5 x_{2}, 0), (y_{1}, y_{2}) ⟩ \\ = 5 x_{2} y_{1}^{*} \\ = ⟨ (x_{1}, x_{2}), (0, 5 y_{1}) ⟩ \end{aligned}

Thus $T^{*} (x) = (0, 5 x_{2})$ . Namely:

T^{*} T (x) = T^{*} (5 x_{2}, 0) = (5 x_{2}, 0)

This is confirmed by calculating $M (T^{*}) = M (T)^{*}$ manually:

M (T^{*}) = [\begin{matrix} 0 & 0 \\ 5 & 0 \end{matrix}]

Thus:

M (T^{*} T) = [\begin{matrix} 0 & 0 \\ 5 & 0 \end{matrix}] [\begin{matrix} 0 & 5 \\ 0 & 0 \end{matrix}] = [\begin{matrix} 0 & 0 \\ 0 & 25 \end{matrix}]

Thus the eigenvalues of $T^{*} T$ are 0 and 25, so then it's singular values are the square roots, namely 0 and 5 are the singular values for $T$ .
☐

4

Question

Suppose $T \in L (V)$ and $s$ is a singular value of $T$ . Prove that there exists a vector $v \in V$ such that $‖ v ‖ = 1$ and $‖ T v ‖ = s$ .

Proof
Since $s$ is a singular value of $T$ , then $s$ is the eigenvalue of $\sqrt{T^{*} T}$ by definition, and thus there exists a corresponding eigenvector to $s$ , we'll name $v$ . Thus $\sqrt{T^{*} T} v = s v$ . But notice:

\begin{aligned} T^{*} T v & = . . . \\ {\sqrt{T^{*} T}}^{2} v & = \sqrt{T^{*} T} (s v) \\ = s (\sqrt{T^{*} T} v) \\ = s^{2} v \end{aligned}

Thus:

\begin{aligned} ⟨ T^{*} T v, v ⟩ & = ⟨ T v, T v ⟩ \\ ⟨ s^{2} v, v ⟩ & = ‖ T v ‖^{2} \\ s^{2} ⟨ v, v ⟩ & = ‖ T v ‖^{2} \\ s^{2} ‖ v ‖^{2} & = ‖ T v ‖^{2} \\ s ‖ v ‖ & = ‖ T v ‖ \end{aligned}

So notice this. Choose $v^{'} \in V$ where $v^{'} = \frac{v}{‖ v ‖}$ . Notice that $v^{'}$ is still an eigenvector of $\sqrt{T^{*} T}$ with eigenvalue $s$ , so the finding above works if we replace all the $v$ 's with $v^{'}$ 's. But notice that now:

Our chosen vector $‖ v^{'} ‖ = 1$ as required.
Because of this, plugging into the equation up top gives that $‖ T v^{'} ‖ = s ‖ v^{'} ‖ = s$ as required.
☐

5

Question

Suppose $T \in L (C^{2})$ is defined by $T (x, y) = (- 4 y, x)$ . Find the singular values of $T$ .

Proof
We should first find $T^{*}$ . Notice for a fixed $y \in C^{2}$ and arbitrary $x \in C^{2}$ :

\begin{aligned} ⟨ T x, y ⟩ & = ⟨ (- 4 x_{2}, x_{1}), (y_{1}, y_{2}) ⟩ \\ = - 4 x_{2} y_{1}^{*} + x_{1} y_{2}^{*} \\ = x_{1} y_{2}^{*} - 4 x_{2} y_{1}^{*} \\ = ⟨ (x_{1}, x_{2}), (y_{2}, - 4 y_{1}) ⟩ \\ ⟨ x, T^{*} y ⟩ & = ⟨ x, (y_{2}, - 4 y_{1}) ⟩ \end{aligned}

Thus $T^{*} (x, y) = (y, - 4 x)$ . We should now determine $M (T^{*} T)$ and then find the square roots of these eigenvalues. First notice that $T^{*} T (x, y) = T^{*} (- 4 y, x) = (x, 16 y)$ , so:

M (T^{*} T) = [\begin{matrix} 1 & 0 \\ 0 & 16 \end{matrix}]

As such, then we have a diagonal matrix, so the eigenvalues are on the diagonal. Thus, $λ_{1} = 1, λ_{2} = 16$ with the corresponding eigenvectors as the standard basis vectors for $C^{2}$ .

Therefore, our singular values are the square roots of these:

4, 1

☐

6

Question

Find the singular values of the differentiation operator $D \in P (R^{2})$ defined by $D p = p^{'}$ , with the inner product given by:

⟨ p, q ⟩ = \int_{- 1}^{1} p (x) q (x) d x

Proof
The example 6.33 gives the standard basis for $P_{2} (R)$ :

β = {\sqrt{\frac{1}{2}}, \sqrt{\frac{3}{2}} x, \sqrt{\frac{45}{8}} (x^{2} - \frac{1}{3})}

Notice what the differentiation operator $D$ does to our standard basis vectors:

\begin{aligned} D e_{1} & = 0 \\ D e_{2} & = \sqrt{\frac{3}{2}} = \sqrt{3} e_{1} \\ D e_{3} & = \sqrt{\frac{45}{8}} \cdot 2 x = \sqrt{\frac{45}{2}} x = \sqrt{15} e_{2} \end{aligned}

Thus:

M (D, β) = [\begin{matrix} 0 & \sqrt{3} & 0 \\ 0 & 0 & \sqrt{15} \\ 0 & 0 & 0 \end{matrix}]

So:

M (D^{*} D, β) = [\begin{matrix} 0 & 0 & 0 \\ \sqrt{3} & 0 & 0 \\ 0 & \sqrt{15} & 0 \end{matrix}] [\begin{matrix} 0 & \sqrt{3} & 0 \\ 0 & 0 & \sqrt{15} \\ 0 & 0 & 0 \end{matrix}] = [\begin{matrix} 0 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 15 \end{matrix}]

Thus our singular values, from largest to smallest, are:

15, 3, 0

☐

7

Question

Define $T \in L (F^{3})$ by:

T (z_{1}, z_{2}, z_{3}) = (z_{3}, 2 z_{1}, 3 z_{2})

Find (explicitly) an isometry $S \in L (F^{3})$ such that $T = S \sqrt{T^{*} T}$ .

Proof
For this case we can just deal with everything in terms of matrices since we're finite dimensional! Notice:

M (T) = [\begin{matrix} 0 & 0 & 1 \\ 2 & 0 & 0 \\ 0 & 3 & 0 \end{matrix}]

Finding $T^{*}$ :

\begin{aligned} ⟨ T x, y ⟩ & = ⟨ (x_{3}, 2 x_{1}, 3 x_{2}), (y_{1}, y_{2}, y_{3}) ⟩ \\ = x_{3} y_{1}^{*} + 2 x_{1} y_{2}^{*} + 3 x_{2} y_{3}^{*} \\ = 2 x_{1} y_{2}^{*} + 3 x_{2} y_{3}^{*} + x_{3} y_{1}^{*} \\ = ⟨ (x_{1}, x_{2}, x_{3}), (2 y_{2}, 3 y_{3}, y_{1}) ⟩ \\ = ⟨ x, T^{*} y ⟩ \end{aligned}

Thus $T^{*} (z_{1}, z_{2}, z_{3}) = (2 z_{2}, 3 z_{3}, z_{1})$ :

M (T^{*}) = [\begin{matrix} 0 & 2 & 0 \\ 0 & 0 & 3 \\ 1 & 0 & 0 \end{matrix}] = M (T)^{T}

which makes sense! As such:

M (T^{*} T) = M (T^{*}) M (T) = [\begin{matrix} 4 & 0 & 0 \\ 0 & 9 & 0 \\ 0 & 0 & 1 \end{matrix}]

Since this is a diagonal matrix, clearly then:

M (\sqrt{T^{*} T}) = [\begin{matrix} 2 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 1 \end{matrix}]

Notice that $M (\sqrt{T^{*} T})$ can be inverted:

M ({\sqrt{T^{*} T}}^{- 1}) = [\begin{matrix} \frac{1}{2} & 0 & 0 \\ 0 & \frac{1}{3} & 0 \\ 0 & 0 & 1 \end{matrix}]

Multiplication of $M (\sqrt{T^{*} T}) M ({\sqrt{T^{*} T}}^{- 1})$ (both ways) gives the identity to verify this. As such, we can assume, and later verify, that:

M (S) = M (T) M ({\sqrt{T^{*} T}}^{- 1}) = [\begin{matrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{matrix}]

This I claim that I choose the transformation:

S (z_{1}, z_{2}, z_{3}) = (z_{3}, z_{1}, z_{2})

is a valid isometry where $T = S \sqrt{T^{*} T}$ . To validate this, first let $v \in V$ be arbitrary. Then:

\begin{aligned} ‖ S v ‖^{2} & = ‖ S (v_{1}, v_{2}, v_{3}) ‖^{2} \\ = ‖ (v_{3}, v_{1}, v_{2}) ‖^{2} \\ = | v_{3} |^{2} ‖ e_{1} ‖^{2} + | v_{1} |^{2} ‖ e_{2} ‖^{2} + | v_{2} |^{2} ‖ e_{3} ‖^{2} \\ = | v_{3} |^{2} ‖ e_{3} ‖^{2} + | v_{1} |^{2} ‖ e_{1} ‖^{2} + | v_{2} |^{2} ‖ e_{2} ‖^{2} & \forall i (‖ e_{i} ‖^{2} = 1) \\ = ‖ (v_{1}, v_{2}, v_{3}) ‖^{2} \\ = ‖ v ‖^{2} \end{aligned}

using the Pythagorean Theorem (our $e_{i}$ 's are orthonormal). Thus, square rooting both sides gives $‖ S v ‖ = ‖ v ‖$ so $S$ is a valid isometry.

To show that $T = S \sqrt{T^{*} T}$ , let $v \in V$ be arbitrary. We know that $v = \sum_{i = 1}^{3} α_{i} e_{i}$ :

\begin{aligned} S \sqrt{T^{*} T} v & = S \sqrt{T^{*} T} (\sum_{i = 1}^{3} α_{i} e_{i}) \\ = \sum_{i = 1}^{3} α_{i} S \sqrt{T^{*} T} e_{i} \\ = 2 α_{1} S e_{1} + 3 α_{2} S e_{2} + α_{3} S e_{3} & Use M (\sqrt{T^{*} T}) \\ = 2 α_{1} e_{2} + 3 α_{2} e_{3} + α_{3} e_{1} \\ = (α_{3}, 2 α_{1}, 3 α_{2}) \\ = T v \end{aligned}

So since $v$ was arbitrary, it follows $S \sqrt{T^{*} T} = T$ .
☐

8

Question

Suppose $T \in L (V)$ , while $S \in L (V)$ is an isometry, and $R \in L (V)$ is a positive operator such that $T = S R$ . Prove that $R = \sqrt{T^{*} T}$ .

Proof
Using polar decomposition, we already know that since $T \in L (V)$ that there is some isometry $S^{'} \in L (V)$ such that:

T = S^{'} \sqrt{T^{*} T}

Furthermore, we know from the question that we are given $S$ and $R$ where:

T = S R

Thus:

S R = S^{'} \sqrt{T^{*} T}

Thus, notice that since both $S, S^{'}$ are isometries, by their properties then they are invertible and $S^{- 1} = S^{*}$ and likewise $S^{' - 1} = S^{' *}$ . So then $R = S^{*} T$ and $S^{' *} T = \sqrt{T^{*} T}$ . Consider:

\begin{aligned} 0 & = ‖ T v ‖^{2} - ‖ T v ‖^{2} \\ = ‖ S^{*} T v ‖^{2} - ‖ S^{' *} T v ‖^{2} \\ = ‖ R v ‖^{2} - ‖ \sqrt{T^{*} T} v ‖^{2} \\ = ⟨ R v, R v ⟩ - ⟨ \sqrt{T^{*} T} v, \sqrt{T^{*} T} v ⟩ \\ = ⟨ R^{*} R v, v ⟩ - ⟨ T^{*} T v, v ⟩ \\ = ⟨ (R^{*} R - T^{*} T) v, v ⟩ \end{aligned}

Thus we have $R^{*} R - T^{*} T = \vec{0}$ . Since $R$ is a positive operator then $R$ is self-adjoint. Thus $R = R^{*}$ so then $R^{2} = T^{*} T$ . Then $R$ has a unique positive square root which is $R = \sqrt{T^{*} T}$ .
☐

10

Question

Suppose $T \in L (V)$ is self-adjoint. Prove that the singular values of $T$ equal the absolute values of the eigenvalues of $T$ , repeated appropriately.

Proof
Since $T$ is self-adjoint then $T = T^{*}$ so then $T$ only has real-valued eigenvalues $λ_{i} \in R$ . For our cases, say that $\dim (V) = n$ and $λ_{1}, . . ., λ_{n} \in R$ are eigenvalues of $T$ , with some repeats possible.

Since $T$ is self-adjoint, then no matter what by either Spectral Theorem we have an orthonormal basis consisting of eigenvectors of $T$ . We'll label them $β = {e_{1}, . . ., e_{n}}$ in this case, and each $e_{i}$ gets paired with it's corresponding eigenvalues $λ_{i}$ . In essence:

M (T, β) = diag (λ_{1}, . . ., λ_{n})

Since $T$ is self-adjoint, then $T^{*} T = T^{*} T = T T = T^{2}$ . As result, then $T = \sqrt{T^{*} T}$ because it's a possible square root (we don't know if it's unique or not).

Consider an eigenvector of $\sqrt{T^{*} T}$ , we'll denote $f_{i}$ . We'll show that $f_{i} \in β$ . Notice that it has an eigenvalue $s$ . Notice that:

{\sqrt{T^{*} T}}^{2} f_{i} = s^{2} f_{i} = T^{*} T f_{i} = T^{2} f_{i} \Rightarrow (T^{2} - s^{2}) f_{i} = \vec{0}

As such notice that:

\Rightarrow (T - s I) (T + s I) f_{i} = \vec{0}

So $f_{i}$ is an eigenvector of $T$ . We know that we have an eigenbasis for $T$ namely $β$ thus $f_{i} \in β$ .

To end, let $s_{i}$ be a singular value of $T$ . Thus it's an eigenvalue of $\sqrt{T^{*} T}$ . We know any $e_{i}$ is an eigenvector of both transformations. Thus:

T e_{i} = λ_{i} e_{i} \Leftrightarrow \sqrt{T^{*} T} e_{i} = s_{i} e_{i}

But:

{\sqrt{T^{*} T}}^{2} e_{i} = s_{i}^{2} e_{i} = T^{2} e_{i} = λ_{i}^{2} e_{i} \Rightarrow (s_{i}^{2} - λ_{i}^{2}) e_{i} = \vec{0}

Since all $e_{i} \neq \vec{0}$ then $s_{i}^{2} - λ_{i}^{2} = 0$ . Thus $s_{i}^{2} = λ_{i}^{2} \Rightarrow s_{i} = | λ_{i} |$ . Notice that $s_{i}$ cannot be negative since singular values are nonnegative square roots of the eigenvalues of $T^{*} T$ .
☐

14

Question

Suppose $T \in L (V)$ Then $\dim (range (T))$ is equal to the number of nonzero singular values of $T$ .

Proof
Say $\dim (V) = n$ for clarity. Let's first prove that $range (T) = range (T^{*} T)$ . We know from a previous exercise in HW 3 - Self-Adjoint and Normal Operators#5 that $range (T) = range (T^{*})$ .

$\subseteq$ : Suppose $w \in range (T)$ . Since $range (T) = range (T^{*})$ then $w \in range (T^{*})$ so $\exists$ $v$ such that $T^{*} v = w$ . We can write $v = v^{'} + v^{″}$ where $v^{'} \in null (T^{*})$ and $v^{″} \in null (T^{*})^{⊥}$ . But since $null (T^{*})^{⊥} = range (T)$ then notice that $\exists$ $u$ where $T u = v^{″}$ . Thus:

T^{*} T u = T^{*} v^{″} = w

Because notice that since $v^{'} \in null (T^{*})$ then $T^{*} w = T^{*} (v^{'} + v^{″}) = T^{*} v^{'} + T^{*} v^{″} = \vec{0} + T^{*} v^{″} = T^{*} v^{″}$ . As such, we can always choose vector $u$ here such that $T^{*} T u = w$ , so then $w \in range (T^{*} T)$ .

$\supseteq$ : We know that $range (T^{*} T) \subseteq range (T^{*}) = range (T)$ .

Thus we know that $range (T^{*} T) = range (T)$ . Since $\sqrt{T^{*} T}$ is diagonalizable as it is self-adjoint, then the number of nonzero singular values of $T$ is the dimension of $range (\sqrt{T^{*} T}) = range (T^{*} T) = range (T)$ .

☐

17

Question

Suppose $T \in L (V)$ has singular value decomposition given by:

T v = s_{1} ⟨ v, e_{1} ⟩ f_{1} + \dots + s_{n} ⟨ v, e_{n} ⟩ f_{n}

for every $v \in V$ where $s_{1}, . . ., s_{n}$ are the singular values of $T$ and $e_{1}, . . ., e_{n}$ and $f_{1}, . . ., f_{n}$ are orthonormal bases of $V$ .

Prove that if $v \in V$ , then:

T^{*} v = s_{1} ⟨ v, f_{1} ⟩ e_{1} + \dots + s_{n} ⟨ v, f_{n} ⟩ e_{n}

Prove that if $v \in V$ , then:

T^{*} T v = s_{1}^{2} ⟨ v, e_{1} ⟩ e_{1} + \dots + s_{n}^{2} ⟨ v, e_{n} ⟩ e_{n}

Prove that if $v \in V$ , then:

\sqrt{T^{*} T} v = s_{1} ⟨ v, e_{1} ⟩ e_{1} + \dots + s_{n} ⟨ v, e_{n} ⟩ e_{n}

Suppose $T$ is invertible. Prove that if $v \in V$ , then:

T^{- 1} v = \sum_{i = 1}^{n} \frac{⟨ v, f_{i} ⟩ e_{i}}{s_{i}}

for every $v \in V$ .

Proof
(1) Notice that:

T e_{j} = s_{j} f_{j}

for all $1 \leq j \leq n$ . Notice then that if $β_{e} = {e_{1}, . . ., e_{n}}$ and $β_{f} = {f_{1}, . . ., f_{n}}$ then we have:

M (T, β_{e}, β_{f}) = [\begin{matrix} s_{1} & 0 & \dots & 0 \\ 0 & s_{2} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & 0 \\ 0 & 0 & \dots & s_{n} \end{matrix}]

Thus:

M (T^{*}, β_{f}, β_{e}) = M (T, β_{e}, β_{f})^{*} = [\begin{matrix} s_{1} & 0 & \dots & 0 \\ 0 & s_{2} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & 0 \\ 0 & 0 & \dots & s_{n} \end{matrix}]

Because all $s_{i} \in R$ so the conjugate operation does nothing. As a result, then reading the matrix this implies:

T^{*} f_{j} = s_{j} e_{j}

As a result, because we can use the bases $β_{e}, β_{f}$ then we can say that, using Graham Schmidt:

T^{*} v = s_{1} ⟨ v, f_{1} ⟩ e_{1} + \dots + s_{n} ⟨ v, f_{n} ⟩ e_{n}

(2) Apply $T^{*}$ to the left of $T$ via the matrices:

M (T^{*} T, β_{e}, β_{e}) = M (T^{*}, β_{f}, β_{e}) M (T, β_{e}, β_{f}) = [\begin{matrix} s_{1}^{2} & 0 & \dots & 0 \\ 0 & s_{2}^{2} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & 0 \\ 0 & 0 & \dots & s_{n}^{2} \end{matrix}]

As such then:

T^{*} T e_{j} = s_{j}^{2} e_{j}

Using Graham Schmidt:

T^{*} T v = \sum_{i = 1}^{n} s_{i}^{2} ⟨ v, e_{i} ⟩ e_{i}

(3) Since $T^{*} T$ is positive, then $\sqrt{T^{*} T}$ is unique. From (2), we know that the eigenvalues are given by $s_{j}^{2}$ for each eigenvector $e_{j}$ . As such $\sqrt{T^{*} T}$ 's eigenvalues are just the square roots of $T^{*} T$ 's eigenvalues, which is $\sqrt{s_{j}^{2}} = | s_{j} | = s_{j}$ because singular values are non-negative. As such, then:

\sqrt{T^{*} T} e_{j} = s_{j} e_{j}

by this finding, so then applying Graham Schmidt:

\sqrt{T^{*} T} v = \sum_{i = 1}^{n} s_{j} ⟨ v, e_{j} ⟩ e_{j}

(4) Apply $T T^{- 1}$ and $T^{- 1} T$ and see they are the identity:

T^{- 1} T v = T^{- 1} (\sum_{i = 1}^{n} s_{i} ⟨ v, e_{i} ⟩ f_{i}) = \sum_{j = 1}^{n} \frac{⟨ \sum_{i = 1}^{n} s_{i} ⟨ v, e_{i} ⟩ f_{i}, f_{j} ⟩ e_{j}}{s_{j}} = \sum_{j = 1}^{n} \frac{s_{j} ⟨ v, e_{j} ⟩ e_{j}}{s_{j}} = \sum_{j = 1}^{n} ⟨ v, e_{j} ⟩ e_{j} = v

T T^{- 1} v = T (\sum_{i = 1}^{n} \frac{⟨ v, f_{i} ⟩ e_{i}}{s_{i}}) = \sum_{j = 1}^{n} s_{j} ⟨ \sum_{i = 1}^{n} \frac{⟨ v, f_{i} ⟩ e_{i}}{s_{i}}, e_{j} ⟩ f_{j} = \sum_{j = 1}^{n} \frac{s_{j}}{s_{j}} ⟨ v, f_{j} ⟩ f_{j} = v

thus $T^{- 1}$ is the correct definition for the inverse of $T$ .
☐