Correlation

The problem with Covariance is that it is very unit independent. What we really want is to instead find a way to normalize the value we get for covariance into a simpler range. The correlation here helps do that.

correlation

The correlation coefficient of $X, Y$ denoted by $Corr (X, Y)$ or $ρ_{X, Y}$ or just $ρ$ is defined by:

ρ_{X, Y} = E [\frac{X - μ_{X}}{σ_{X}} \cdot \frac{Y - μ_{Y}}{σ_{Y}}] = \frac{Cov (X, Y)}{σ_{X} σ_{Y}}

Properties of Correlation

Proposition

For any two rvs $X, Y$ :

$Corr (X, Y) = Corr (Y, X)$
$Corr (X, X) = 1$
(Scale Invariance Property): If $a, b, c, d$ are constants and $a c > 0$ :

Corr (a X + b, c Y + d) = Corr (X, Y)

$Corr (X, Y) \in [- 1, 1]$

Proof

See Covariance's properties (1).

Corr (X, X) = \frac{Cov (X, X)}{σ_{X}^{2}} = \frac{Var (X, X)}{σ_{X}^{2}} = \frac{σ_{X}^{2}}{σ_{X}^{2}} = 1

Again using the Covariance's properties gives this result.
Same as (3). Notice that the maximum of the covariance part is when both variables are the same (gives (2)) or all the way uncorrelated (gives -1 as we expected).

☐

Propsition

If $X, Y$ are independent then $ρ = 0$ . However, $ρ = 0$ does not imply independence.
$ρ = 1, - 1$ iff $Y = a X + b$ for some numbers $a, b$ where $a \neq 0$ .

When $ρ = 0$ we specifically say that $X, Y$ are uncorrelated.

Uncorrelated iff expectation and multiplication can be swapped.

Two rvs $X, Y$ are said to be uncorrelated iff $E (X Y) = μ_{X} μ_{Y}$ .

Proof

Uncorrelated implies $Cov (X, Y) = 0$ and applying the covariance shortcut:

ρ = 0 ⟺ Cov (X, Y) = 0 ⟺ E (X Y) - μ_{X} μ_{Y} = 0 ⟺ E (X Y) = μ_{X} μ_{Y}

☐