STT 861 Theory of Prob and STT I Lecture Note - 10
2017-11-08
Solving a problem in midterm exam; conditional distribution, definition, example and its expectation and variance.
Portal to all the other notes
- Lecture 01 - 2017.09.06
- Lecture 02 - 2017.09.13
- Lecture 03 - 2017.09.20
- Lecture 04 - 2017.09.27
- Lecture 05 - 2017.10.04
- Lecture 06 - 2017.10.11
- Lecture 07 - 2017.10.18
- Lecture 08 - 2017.10.25
- Lecture 09 - 2017.11.01
- Lecture 10 - 2017.11.08 -> This post
- Lecture 11 - 2017.11.15
- Lecture 12 - 2017.11.20
- Lecture 13 - 2017.11.29
- Lecture 14 - 2017.12.06
Lecture 10 - Nov 08 2017
Exam question
Let X∼U(50,90), Z∼U(70,90), Law(Z)=Law(X|X≥70).
Q: Let X∼U(a,b), c∈(a,b), find law (X|X>c).
A: let F be the CDF of X|X>c. Let x∈[c,b].
F(x)=P(X≤x|x>c)=P(X≤x,X|c)P(X>c)=x−cb−ab−cb−a=x−cb−cIt is uniform in (c,b).
Conditional distribution - Chapter 5
Recall joint density of a vector (X,Y) is fX,Y(x,y), x and y are real numbers.
Definition (Theorem): The conditional law of X given Y is
fX|Y(x|y)=fX,Y(x,y)fY(y)Consequently, the CDF of X given Y=y is
FX|Y=P(X≤x|Y=y)=∫x−∞fX|Y(u,v)duDiscrete case using the same idea.
For Y, no change in notation.
For X, use ∑ instead of integrals.
Example 1
Let N∼Poi(λ) and X∼Bin(N,θ), here we see law (X|N=k)=Bin(k,θ).
This type of example is called a mixture. A mixture is when the parameter of one random variable is determined by the value of another random variable.
In our example, we would like to find law (X) unconditional on N. We want to find P(X=i) for any k=0,1,2,….
P(X=i)=∞∑k=iP(X=i and N=k)=∞∑k=iP(X=i|N=k)P(N=k)=∞∑k=iCikθi(1−θ)k−ie−λλkk!=∞∑k=ik!i!(k−i)!θi(1−θ)k−ie−λλkk!=e−λθii!∞∑k=i(1−θ)k−i(k−i)!λk=e−λθii!∞∑k′=0(i−θ)k′k′!λk′+i=e−λ(θλ)ii!∞∑k′=0(i−θ)k′k′!λk′=e−λ(θλ)ii!eλ(1−θ)=e−λθ(λθ)ii!It is Poisson distribution with parameter λ′=λθ.
Home exercise
Read and understand the section of the book in Chapter 5 on Markov Chains.
Conditional expectations - Chapter 5.2
E(X|Y=y)=∫∞−∞fX|Y(x|y)xdxFor the discrete case,
E(X|Y=y)=∞∑k=−∞PX|Y(xk|y)xkSuper important theorem (definition)
let X and Y be given. Let
g(x)=E(Y|X=x) v(x)=Var(Y|X=x)=E(Y2|X=x)−(E(Y|X=x))2=E((Y−g(x))2|X=x)Theorem: E(XY)=E(Xg(x))
(the idea of “tower property”: E(E(Z|X))=E(Z). “Given X” means X is not random).
Proof: use the property to prove the theorem
E(XY)=E(E(XY|X))=E(XE(Y|X))=E(Xg(X))Notice:
- this is the function g with x replaced by X.
- E(h(X)Y)=E(h(X)g(X)). If we put X in another function.
Now recall v(x). It turns out
Var(Y)=E(v(X))+Var(g(X))=E(v(x))+var(E(Y|X))Explanation: The unconditional variance of Y is the expected conditional variance plus the variance of the conditional expectation.
Example 2
Let X|N∼Bin(N,θ), N∼Poi(λ).
(We found that X∼Poi(λ))
From the theorem, ignoring the fact above, we apply E(X)=E(E(X|N))
If we give some information and then take the information away, it is equivalent to not giving information at the first place.
E(X)=E(E(X‖N))=θE(N)=λθ E(Poi(λθ))=λθWe verify the theorem above.
For variance
Var(X)=E(v(N))+Var(E(X|N))=Nθ(1−θ)+Nθ=θ(1−θ)λ+θ2λ=λθThis is also correct.