Parameters and Families of Distributions

Recall the definition of bernoulli random variables. If we only have $0$ and $1$ events, then if $p (0) = .8 \Rightarrow p (1) = 1 - p (0) = .2$ . In general, $p (1) = α \Rightarrow p (0) = 1 - α$ where $0 < α < 1$ . Since $p (x)$ for some $X$ discrete random variable and it depends on $α$ , we write it as $p (x; α)$ :

p (x; α) = {\begin{cases} 1 - α & x = 0 \\ α & x = 1 \end{cases}

parameter, family

Suppose $p (x)$ depends on one or more quantities, each of which can be assigned any one of a number of possible values, with each different value determining a different probability distribution. Each such quantity is a parameter of the distribution. The collection of all probability distributions for different values of the parameter(s) is called a family of probability distributions

For example, $α$ above is a parameter. Each $0 < α < 1$ determines a different member of a family of distributions, such as $p (x; .2)$ or $p (x; .08)$ . The bernoulli random variables have the same form, so it's the family of Bernoulli distributions.

Example

Let $A$ be an event and define $θ = P (A)$ . Say $X$ is some discrete event (like a number of occurences). Then $θ$ is a parameter of the Probability Distribution, PMF of $X$ . Experimental outcomes are $A, A^{'}, A^{'} A^{'} A, \dots$ with $X$ values $1, 2, 3, . . .$ counting the occurrence before (and including) we hit $A$ . Then:

\begin{aligned} p (1; θ) & = P (X = 1) = P (A) = θ \\ p (2; θ) & = P (X = 2) = P (A^{'} A) = P (A^{'}) P (A) = (1 - θ) θ \\ p (3; θ) & = P (X = 3) = P (A^{'} A^{'} A) = P (A^{'})^{2} P (A) = (1 - θ)^{2} θ \end{aligned}

In general:

p (x; θ) = (1 - θ)^{x - 1} θ

for $x = 1, 2, 3, . . .$ .

What about expected value in this case? See:

\begin{aligned} E (X) & = \sum_{x = 1}^{\infty} x p (x; θ) \\ = \sum_{x = 1}^{\infty} x \cdot (1 - θ)^{x - 1} θ \\ = θ \sum_{x = 1}^{\infty} [\frac{- d}{d θ} (1 - θ)^{x}] \\ = - θ \frac{d}{d θ} \sum_{x = 1}^{\infty} (1 - θ)^{x} \\ = - θ \frac{d}{d θ} \frac{1 - θ}{θ} \\ = - θ (- θ)^{- 2} \\ = \frac{1}{θ} \end{aligned}

These families of distributions are called the geometric distributions due to the solving of the geometric series.