Taken from a talk on 11/15/2024 by Jessie Loucks - Tavitas from Sac State.

1: What is Computer Vision?

Specifically from mathematicians' perspective. Two motivating questions:

1. Given cameras (positions, angles) + images (color data), recover the object: Traingulation
2. Given objects + images, recover the camera: Resectioning

2: What is a pinhole camera?

This is the idea of the hole in a box projecting the world's image onto the backboard of the box. Mathematically, the camera is a map $A:{\mathbb{R}}^3\to {\mathbb{R}}^2$. Where $(x,y,z)\to (\frac{x}{z},\frac{y}{z})$.

But notice that this map is not invertible, and if $z=0$ then we're toast! It's also non-linear, so we can't use a matrix for it.

The other thing is that in 3D space two parallel lines don't intersect; but their map via $A$ may have this (think of the train tracks' lines intersecting at the horizon).

The fix...

3: Perspective & Projective Geometry

The idea is that we look at this via perspective geometry, where we actually have these converging lines. This is in contrast to orthographic geometry, where the 2D counterparts also are parallel if they are in the ${\mathbb{R}}^3$.

You can think of the left set as ${\mathbb{R}}^3$ and the RHS as the Limit Points of ${\mathbb{R}}^3$. The rules are:

1. $(\alpha x:\alpha y:\alpha z:\alpha w)=(x:y:z:w)$ where $\alpha \ne 0$.
2. At least one non-zero coordinate $(0,0,0,0)\notin {\mathbb{P}}^3$.

For example, if a line through $\overrightarrow{0}$ in ${\mathbb{R}}^3$ is:

So the end point of this line is just this limit point $(x:y:z:0)$.

For ${\mathbb{P}}^2$ it's the same idea, except with three coordinates instead of 4.

With projective space then we get linearity. The map $A:{\mathbb{P}}^3\to {\mathbb{P}}^2$ is a valid linear map and is:

4: Triangulation and Resectioning

Triangulation

Say we have $m$ cameras. Then we have a map $\bar{A}=(\overline{A_1},\dots ,\overline{A_m})$. A multiview configuration is a tuple of cmaeras, capturing multiple scene points. The multiview variety of $\bar{A}$ is:

Here $n$ is the number of world-points that we get images of. The idea is $\mathrm{\Gamma }$ is useful for reconstructing scene points.

Resectioning

This is the meat and potatoes here. A hypercamera configuration is a tuple of world points $q_1,\dots ,q_n$ being captured by multiple cameras $m$ of them. This recovers camera structure.

5: Duality

Like with the Chapter 3 (cont.) - Products and Quotients of Vector Spaces#3.F Duality, a lot of things have duality like Graphs!

This gets into Carlsson-Weinshall Duality.