Lecture 13 - Applications of Quadratic Forms

This lecture is much more of a lecture on the applications of what we've learned so far.

Quadratic Forms

Quadratic Forms

A quadratic form in $n$ commuting variables is a homogeneous polynomial of degree 2 in terms of those variables.

Here homogeneous here just says that every monomial in the polynomial is of degree 2. As an example:

3 x^{2} + y^{2} - z^{2} + 4 x z - 5 x y

is a quadratic form of $3$ commuting variables $x, y, z$ . This is homogeneous since each term is of degree 2 or less. If we added a $+ 5 y$ then it wouldn't fit the form since we need exactly the second degree of a combination of our $x, y, z$ .

If we have a quadratic form $Q (x_{1}, . . ., x_{n})$ then:

Q (x_{1}, . . ., x_{n}) = \sum_{i, j = 1}^{n} a_{i j} x_{i} x_{j}

We don't really need to do the entire summation, as $x_{i} x_{j} = x_{j} x_{i}$ . As such, we may restrict to have $i \leq j$ . Notice:

\begin{aligned} Q (x_{1}, . . ., x_{n}) & = \sum_{i = 1}^{n} x_{i} \underset{the i -th entry of A \vec{x}}{\underset{⏟}{(\sum_{j = 1}^{n} a_{i j} x_{j})}} \\ = \sum_{i = 1}^{n} x_{i} (A \vec{x})_{i} \\ = ⟨ \vec{x}, A \vec{x} ⟩ & Dot Product \\ = {\vec{x}}^{T} A \vec{x} & (⟨ \vec{x}, \vec{y} ⟩ = {\vec{x}}^{T} \vec{y} for dot product) \end{aligned}

You may guess that since $x_{i} x_{j}$ is commuting that we have some freedom. This will be the case for $A$ .

As an instance of this, consider:

3 x_{1}^{2} + 2 x_{1} x_{2} + x_{1} x_{3} - 3 x_{3}^{2} = [\begin{matrix} x_{1} & x_{2} & x_{3} \end{matrix}] [\begin{matrix} 3 & a_{1} & b_{1} \\ a_{2} & 0 & 0 \\ b_{2} & 0 & - 3 \end{matrix}] [\begin{matrix} x_{1} \\ x_{2} \\ x_{3} \end{matrix}]

We want to convert the LHS to the matrix representation we noticed before. We got the entries in the matrix by doing the actual matrix transformations and confirming these things. This works as long as:

a_{1} + a_{2} = 2, b_{1} + b_{2} = 1

We want to make $A$ symmetric, so we can do that by splitting our terms! Thus have $a_{1} = a_{2} = 1$ and $b_{1} = b_{2} = \frac{1}{2}$ . Thus:

A = [\begin{matrix} 3 & 1 & \frac{1}{2} \\ 1 & 0 & 0 \\ \frac{1}{2} & 0 & 0 \end{matrix}]

is a symmetric matrix. Notice in general for all the $^{2}$ terms we have the entries on the diagonal, and the antidiagonal is the sums of the other entries for $Q$ .

Symmetry in

A

Every quadratic form $Q$ can be put in the form:

Q (\vec{x}) = {\vec{x}}^{T} A \vec{x}

where $A$ is a symmetric matrix.

The operator $T$ where $M (T) = A$ then must be normal and self-adjoint (if we restrict $A \in M (R^{n})$ ). Namely, $T : R^{n} \to R^{n}$ is defined by $T \vec{x} = A \vec{x}$ . Thus $T$ is self-adjoint since $M (T^{*}) = A^{*} = A$ . Using the Spectral Theorem, then we must have an orthonormal eigenbasis $β = {u_{1}, . . ., u_{n}}$ for which $M (T, β)$ is diagonal.

If we let:

P = [\begin{matrix} 1 & 1 & \dots & 1 \\ u_{1} & u_{2} & \dots & u_{n} \\ 1 & 1 & \dots & 1 \end{matrix}] \Rightarrow P^{T} = [\begin{matrix} 1 & u_{1} & 1 \\ 1 & u_{2} & 1 \\ ⋮ & ⋮ & ⋮ \\ 1 & u_{n} & 1 \end{matrix}]

Thus:

P^{T} P = I \Rightarrow P^{T} = P^{- 1}

In particular, we have $P^{T} A P = D$ , or $A = P D P^{T}$ , where $D = diag (λ_{1}, . . ., λ_{n})$ for each $u_{1}, . . ., u_{n}$ eigenvector.

So given a quadratic form:

\begin{aligned} Q (\vec{x}) & = \vec{x^{T}} A \vec{x} & (Let \vec{y} = P^{T} \vec{x} so \vec{x} = P \vec{y}) \\ Q (\vec{x}) & = Q (P \vec{y}) \\ = (P \vec{y})^{T} A (P \vec{y}) \\ = {\vec{y}}^{T} P^{T} A P \vec{y} \\ = {\vec{y}}^{T} D \vec{y} & D = diag (λ_{1}, . . ., λ_{n}) \\ = λ_{1} y_{1}^{2} + \dots + λ_{n} y_{n}^{2} \end{aligned}

This is a nice theorem that we just proved:

Theorem

Given a symmetric matrix $A$ , there is an orthogonal change of variables $\vec{x} = P \vec{y}$ or $(\vec{y} = P^{T} \vec{x})$ that transforms the quadratic form ${\vec{x}}^{T} A \vec{x}$ into ${\vec{y}}^{T} D \vec{y}$ where there are no cross terms. Here $D = diag (λ_{1}, . . ., λ_{n})$ .

Using these!

When you graph quadratic forms, ellipses, hyperbolas, etc. are all different shapes that had constrained properties. The theorem says we can make a change of variables to allow for an elimination of other potential cases.

For instance, try to graph:

2 x^{2} - 4 x y + 5 y^{2} = 1

We want to try to graph this out. Here we have a quadratic form:

[\begin{matrix} x & y \end{matrix}] [\begin{matrix} 2 & - 2 \\ - 2 & 5 \end{matrix}] [\begin{matrix} x \\ y \end{matrix}]

So we have an orthonormal basis for $A$ . We can go through our Linear Analysis I methods to find our eigenvectors. Doing so yields that $v_{1} = (- 1, 2)$ and $v_{2} = (2, 1)$ with $λ_{1} = 6$ and $λ_{2} = 1$ . Notice their dot product is 0, as expected as they are orthogonal (although they are not normal) but we can normalize. As such:

u_{1} = \frac{1}{\sqrt{5}} [\begin{matrix} - 1 \\ 2 \end{matrix}], u_{2} = \frac{1}{\sqrt{5}} [\begin{matrix} 2 \\ 1 \end{matrix}]

so then define:

P = \frac{1}{\sqrt{5}} [\begin{matrix} - 1 & 2 \\ 2 & 1 \end{matrix}]

notice that $P^T = P here. So then we introduce our new coordinates:

\vec{x^{'}} = [\begin{matrix} x^{'} \\ y^{'} \end{matrix}] = P^{T} \vec{x} = P^{T} [\begin{matrix} x \\ y \end{matrix}] = \frac{1}{\sqrt{5}} [\begin{matrix} - x + 2 y \\ 2 x + y \end{matrix}]

Thus:

\begin{aligned} Q (\vec{x}) & = {\vec{x^{'}}}^{T} [\begin{array}{c} 6 & 0 \\ 0 & 1 \end{array}] \vec{x^{'}} \\ = 6 (x^{'})^{2} + (y^{'})^{2} \end{aligned}

So we can graph the ellipse in the $x^{'}, y^{'}$ coordinate system, noting that it's equivalent to its standard form:

\frac{(x^{'})^{2}}{(\frac{1}{\sqrt{6}})^{2}} + \frac{(y^{'})^{2}}{1^{2}} = 1

Then we just convert from $x^{'}, y^{'}$ back to $x, y$ . Recall that we had:

\vec{x^{'}} = P^{T} \vec{x} \Rightarrow \vec{x} = P \vec{x^{'}}, P = [\begin{matrix} - 1 / \sqrt{5} & 2 / \sqrt{5} \\ 2 / \sqrt{5} & 1 / \sqrt{5} \end{matrix}]

Thus we just transform our $\vec{x^{'}}$ to $\vec{x}$ in a transforming way:

Here $P$ reflects across $y = x$ and rotates CCW by some angle $\alpha$. Thus, we transform into the new axes:

Taylor Series with Multiple Variables

In Calc IV you learn about the second-derivative test. We now have all the tools using Linear Algebra to show why that works entirely.

We first look at the extremal values of a quadratic form. For instance if we have:

Q (\vec{x}) = {\vec{x}}^{T} A \vec{x}

then:

Q (α \vec{x}) = (α \vec{x})^{T} A (α \vec{x}) = α^{2} ({\vec{x}}^{T} A \vec{x}) = α^{2} Q (\vec{x})

If we know what $Q$ does to unit vectors, then we know what it does to all vectors. So we can just look at the unit sphere for these vectors. Suppose $‖ x ‖ = 1$ , so it lives there. The question we'll look at (tomorrow) is: what are the extreme values of $Q (x)$ when $‖ x ‖ = 1$ ?