Covariance

Definition

The covariance between two rvs $X, Y$ is:

\begin{aligned} Cov (X, Y) & = E [(X - μ_{X}) (Y - μ_{Y})] \\ = {\begin{cases} \sum_{x} \sum_{y} (x - μ_{X}) (y - μ_{Y}) p (x, y) \\ \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} (x - μ_{X}) (y - μ_{Y}) f (x, y) d x d y \end{cases} \end{aligned}

based on whether $X, Y$ are continuous or discrete.

For an example of its usage, see Ch 7 - 23, 25 for discrete examples and Ch 7 - 13, 17, 33 for continuous examples.

Properties of Covariance

Proposition

For any two random variables $X, Y$ :

$Cov (X, Y) = Cov (Y, X)$
$Cov (X, X) = Var (X)$
(Covariance Shortcut Formula): $Cov (X, Y) = E [X Y] - μ_{X} μ_{Y}$
(Distributive Property of Covariance): For any rv $Z$ and any constants $a, b, c$ :

Cov (a X + b Y + c, Z) = a Cov (X, Z) + b Cov (Y, Z)

Proof

Cov (X, X) = E [(X - μ_{X})^{2}] = Var (X)

\begin{aligned} Cov (X, Y) & = E [(X - μ_{X}) (Y - μ_{Y})] \\ = E [X Y - μ_{X} Y - μ_{Y} X + μ_{X} μ_{Y}] \\ = E [X Y] - μ_{X} E [Y] - μ_{Y} E [X] + μ_{X} μ_{Y} \\ = E [X Y] - μ_{X} μ_{Y} - μ_{Y} μ_{X} + μ_{X} μ_{Y} \\ = E [X Y] - μ_{X} μ_{Y} \end{aligned}

☐