STT 861 Theory of Prob and STT I Lecture Note - 10

2017-11-08

Solving a problem in midterm exam; conditional distribution, definition, example and its expectation and variance.

Portal to all the other notes

Lecture 10 - Nov 08 2017

Exam question

Let $X \sim U (50, 90)$ , $Z \sim U (70, 90)$ , $L a w (Z) = L a w (X | X \geq 70)$ .

Q: Let $X \sim U (a, b)$ , $c \in (a, b)$ , find law ( $X | X > c$ ).

A: let $F$ be the CDF of $X | X > c$ . Let $x \in [c, b]$ .

F (x) = P (X \leq x | x > c) = \frac{P (X \leq x, X | c)}{P (X > c)} = \frac{\frac{x - c}{b - a}}{\frac{b - c}{b - a}} = \frac{x - c}{b - c}

It is uniform in $(c, b)$ .

Conditional distribution - Chapter 5

Recall joint density of a vector $(X, Y)$ is $f_{X, Y} (x, y)$ , $x$ and $y$ are real numbers.

Definition (Theorem): The conditional law of $X$ given $Y$ is

f_{X | Y} (x | y) = \frac{f_{X, Y} (x, y)}{f_{Y} (y)}

Consequently, the CDF of $X$ given $Y = y$ is

F_{X | Y} = P (X \leq x | Y = y) = \int_{- \infty}^{x} f_{X | Y} (u, v) d u

Discrete case using the same idea.

For $Y$ , no change in notation.

For $X$ , use $\sum$ instead of integrals.

Example 1

Let $N \sim P o i (λ)$ and $X \sim B i n (N, θ)$ , here we see law $(X | N = k) = B i n (k, θ)$ .

This type of example is called a mixture. A mixture is when the parameter of one random variable is determined by the value of another random variable.

In our example, we would like to find law $(X)$ unconditional on $N$ . We want to find $P (X = i)$ for any $k = 0, 1, 2, \dots$ .

\begin{aligned} P (X = i) & = \sum_{k = i}^{\infty} P (X = i a n d N = k) \\ = \sum_{k = i}^{\infty} P (X = i | N = k) P (N = k) \\ = \sum_{k = i}^{\infty} C_{k}^{i} θ^{i} (1 - θ)^{k - i} e^{- λ} \frac{λ^{k}}{k!} \\ = \sum_{k = i}^{\infty} \frac{k!}{i! (k - i)!} θ^{i} (1 - θ)^{k - i} e^{- λ} \frac{λ^{k}}{k!} \\ = \frac{e^{- λ} θ^{i}}{i!} \sum_{k = i}^{\infty} \frac{(1 - θ)^{k - i}}{(k - i)!} λ^{k} \\ = \frac{e^{- λ} θ^{i}}{i!} \sum_{k^{'} = 0}^{\infty} \frac{(i - θ)^{k^{'}}}{k^{'}!} λ^{k^{'} + i} \\ = \frac{e^{- λ} (θ λ)^{i}}{i!} \sum_{k^{'} = 0}^{\infty} \frac{(i - θ)^{k^{'}}}{k^{'}!} λ^{k^{'}} \\ = \frac{e^{- λ} (θ λ)^{i}}{i!} e^{λ (1 - θ)} \\ = e^{- λ θ} \frac{(λ θ)^{i}}{i!} \end{aligned}

It is Poisson distribution with parameter $λ^{'} = λ θ$ .

Home exercise

Read and understand the section of the book in Chapter 5 on Markov Chains.

Conditional expectations - Chapter 5.2

E (X | Y = y) = \int_{- \infty}^{\infty} f_{X | Y} (x | y) x d x

For the discrete case,

E (X | Y = y) = \sum_{k = - \infty}^{\infty} P_{X | Y} (x_{k} | y) x_{k}

Super important theorem (definition)

let $X$ and $Y$ be given. Let

g (x) = E (Y | X = x)

v (x) = V a r (Y | X = x) = E (Y^{2} | X = x) - (E (Y | X = x))^{2} = E ((Y - g (x))^{2} | X = x)

Theorem: $E (X Y) = E (X g (x))$

(the idea of “tower property”: $E (E (Z | X)) = E (Z)$ . “Given $X$ ” means $X$ is not random).

Proof: use the property to prove the theorem

\begin{aligned} E (X Y) & = E (E (X Y | X)) \\ = E (X E (Y | X)) \\ = E (X g (X)) \end{aligned}

Notice:

this is the function $g$ with $x$ replaced by $X$ .
$E (h (X) Y) = E (h (X) g (X))$ . If we put $X$ in another function.

Now recall $v (x)$ . It turns out

V a r (Y) = E (v (X)) + V a r (g (X)) = E (v (x)) + v a r (E (Y | X))

Explanation: The unconditional variance of $Y$ is the expected conditional variance plus the variance of the conditional expectation.

Example 2

Let $X | N \sim B i n (N, θ)$ , $N \sim P o i (λ)$ .

(We found that $X \sim P o i (λ)$ )

From the theorem, ignoring the fact above, we apply $E (X) = E (E (X | N))$

If we give some information and then take the information away, it is equivalent to not giving information at the first place.

E (X) = E (E (X ‖ N)) = θ E (N) = λ θ

E (P o i (λ θ)) = λ θ

We verify the theorem above.

For variance

V a r (X) = E (v (N)) + V a r (E (X | N)) = N θ (1 - θ) + N θ = θ (1 - θ) λ + θ^{2} λ = λ θ

This is also correct.

← Older-LTE Telecommunication Primer - Part 1

LTE Telecommunication Primer - Part 2-Newer →

Disqus Comment （0）

LanternD's Castle

An electronics enthusiast - survive technically