STT 861 Theory of Prob and STT I Lecture Note - 4

2017-09-27

Expectation and its theorems, Geometry distribution, Negative Binomial distribution and their examples; Theorem of linerarity, PMF of a pair of random variables, Markov's inequality; variance and its examples, uniform distribution.

Portal to all the other notes

Lecture 04 - Sept 27 2017

Expectation

A v g (X_{1}) + A v g (X_{2}) + . . . + A v g (X_{n}) = \frac{n}{p}

Definition: Let $X$ be a (discrete) r.v. with PMF $p_{k} = P [X = k], k \in Z$ . We say that the expectation of $X$ is

E [X] := \sum_{k = - \infty}^{\infty} x_{k} p_{k}

Example 1

A game of dice. Throw 1 die, win \$ 1 if outcome is even, win \$ outcome/2 if outcome is odd.

E [x] = \frac{1 + 1 + 1}{6} + \frac{0.5 + 1.5 + 2.5}{6} = \frac{7.5}{6} = 1.25

Example 2

Let $X \sim B e r n (p)$ , thus $E (X) = p$ .

Example 3

Let $X \sim B i n (n, p)$ , $E (X) = \sum_{k = 0}^{n} k C_{n}^{k} p^{k} (1 - p)^{n - k}$ , this is the dumb way to solve.

Another way, use linearity,

E [X] = \sum E [X_{i}] = n \times p

Example 4

Let $X \sim G e o m (p)$ , then $E [x] = \frac{1}{p}$ (prove this at home). Here is a link to it. I don’t want to type it again.

Example 5

Let $X \sim N e g B i n (X)$ , then

E [X] = \sum E [X_{i}] = \frac{n}{p}

Try at home

Find the definition, PMF and expectation for the “Multinomial” distribution.

Theorem of Linearity: Let $X$ amd $Y$ be two r.v.’s and $α, β \in R$ Let $Z = α X + β Y$ .

E [Z] = E [α X + β Y] = α E [X] + β E [Y]

Don’t require $X$ and $Y$ be independent.

Theorem Let $X$ and $Y$ be two independent r.v.’s, then

E [X Y] = E [X] E [Y]

(try to prove it at home)

PMF of $(X, Y)$

For $(X, Y)$ a pair of r.v.s, we can define PMF of $(X, Y)$ :

p_{x, y} = P [X = x a n d Y = y]

Note: if $X$ and $Y$ are independent, then $p_{x, y} = p_{x} p_{y} = P [X = x] P [Y = y]$ .

Example 6

Let $Y = X^{2}$ where $X$ is a r.v. Assume $P [X = k] = p_{k}$ , then

E [Y] = E [X^{2}] = \sum_{k = - \infty}^{\infty} (x_{k})^{2} p_{k}

Theorem: Let $X$ be a r.v.with PMF $P [X = x_{k}] = p_{k}$ , let $F$ be a function from $R$ to $R$ , let $Y = F (X)$ , Then

E [Y] = E [F (X)] = \sum_{k = - \infty}^{\infty} F (x_{k}) p_{k}

Theorem: Let $X$ be a r.v. such that $X \geq 0$ ( $P [X \geq 0] = 1$ ). Then $E [X] \geq 0$ . Proven by the definition.

Theorem (Markov’s inequality): Let $X$ be a non-negative r.v. Let $X$ be fixed real $> 0$ . Then

P [X > C] \leq \frac{E [X]}{C}

(Chebyshev inequality is related to Markov inequality).

Variance (Chapter 1.6)

Empirically the variance is the average squared deviation from the mean.

With data $x_{1}, x_{2}, \dots, x_{n}$ , let

μ = \frac{1}{n} \sum_{i = 1}^{n} x_{i}

and

σ^{2} = v a r i a n c e = \frac{1}{n} \sum_{i = 1}^{n} (x_{i} - μ)^{2}

Mathematically, variance is defined as:

Let $X$ be a r.v.

V a r (X) = E [(X - E [X])^{2}]

(General formula)

Here we used the approximate corresponding $\frac{1}{n} \leftrightarrow X$ , $μ \leftrightarrow E [X]$ .

Proposition: $V a r (X) = E [X^{2}] - (E [X])^{2}$ . This is extremely important, especially in doing homework. :-)

Proof:

\begin{aligned} V a r (X) & = E [(X - E [X])^{2}] \\ = E [X^{2} - 2 X E [X] + (E [X])^{2}] \\ = E [X^{2}] - E [2 X E [X]] + E [(E [X])^{2}] \\ = E [X^{2}] - 2 E [X]^{2} + E [X]^{2} \\ = E [X^{2}] - (E [X])^{2} \end{aligned}

Usually if PMF of $X$ is given, it is easier compute to $V a r (X)$ using the 2nd formula than the original definition.

Usual Notation

Let $X$ be a r.v.

$μ := E [X]$
$σ^{2} := V a r [X]$
$σ := \sqrt{V a r [X]}$ aka “the standard deviation”

Example 7

Let $X$ have this PMF: for $k = 1, 2, 3, 4$ , $p_{k} = P [X = k] = \frac{k}{10}$ .

Then $μ = E [X] = \frac{1 + 4 + 9 + 16}{10} = 3,$

E [X^{2}] = \frac{1 + 8 + 27 + 64}{10} = 10

Finally, $V a r [X] = E [X^{2}] - E [X]^{2} = 10 - 9 = 1$

Properties of the variance: Let $c \in R$ , let $X$ & $Y$ be independent r.v. Then

$V a r (c X) = c^{2} V a r (X)$
$V a r (X + c) = V a r (X)$
$V a r (X + Y) = V a r (X) + V a r (Y)$ (prove this at home)

Example 8

Let $X \sim B i n o m (n, p)$ , therefore,

X = \sum_{i = 1}^{n} X_{i}

where $X_{i}$ are i.i.d $B e r n o u l l i (p)$ . Thus

V a r (X) = V a r (X_{1}) + V a r (X_{2}) + . . . + V a r (X_{n})

Variance of Bernoulli:

V a r (X_{i}) = E [X_{i}^{2}] - E [X_{i}]^{2} = p - p^{2} = p (1 - p)

Thus,

V a r (X) = n p (1 - p)

In Ch2, we will calculate $V a r (X \sim G e o m (p)) = \frac{1 - p}{p^{2}}$ .

Let $X \sim N e g B i n (n, p)$ , then $V a r (X) = \frac{n (1 - p)}{p^{2}}$ .

Proposition: Let $X$ be a r.v. Let $c \in R$ , then

E [(X - c)^{2}] = V a r (X) + (c - μ)^{2}

where $μ = E [X]$ .

Proof:

\begin{aligned} E [(X - c)^{2}] & = E (X - μ + μ - c)^{2} \\ = E [(X - μ)^{2} + 2 (X - μ) (μ - c) + (μ - c)^{2}] \\ = V a r (X) + 0 + (μ - c)^{2} \\ = V a r (X) + (μ - c)^{2} \end{aligned}

Indeed, for $c = x$ , we get the definition of $V a r (X)$ . For $c \neq μ$ we get

E [(X - c)^{2}] > V a r (X)

Example 9

Let $X$ be a uniform r.v. is the set of integers from 1 to $N$ , $P [X = k] = \frac{1}{N}$ for $k = 1, 2, 3, \dots, n$ (definition).

Then

E [N] = \frac{N + 1}{2},

V a r (N) = \frac{N^{2} - 1}{12}

End note

Sometimes people (the professor) use different notation for expectation and variance,

$E [X]$ or $E (X)$
$V a r [X]$ or $V a r (X)$

Just choose whichever you like. I prefer ( ) to [ ].

← Older-STT 861 Theory of Prob and STT I Lecture Note - 3

STT 861 Theory of Prob and STT I Lecture Note - 5-Newer →

Disqus Comment （0）

LanternD's Castle

An electronics enthusiast - survive technically

STT 861 Theory of Prob and STT I Lecture Note - 4

Portal to all the other notes

Lecture 04 - Sept 27 2017

Expectation

Example 1

Example 2

Example 3

Example 4

Example 5

Try at home

PMF of $(X, Y)$

Example 6

Variance (Chapter 1.6)

Usual Notation

Example 7

Example 8

Example 9

End note

Portal to all the other notes

Lecture 04 - Sept 27 2017

Expectation

Example 1

Example 2

Example 3

Example 4

Example 5

Try at home

PMF of (X,Y)(X,Y)

Example 6

Variance (Chapter 1.6)

Usual Notation

Example 7

Example 8

Example 9

End note

PMF of $(X, Y)$