STT 861 Theory of Prob and STT I Lecture Note - 12
2017-11-20
Examples on continuous distirbution conditional on discrete distribution; bivariate normal distribution. Not much for today.
Portal to all the other notes
- Lecture 01 - 2017.09.06
- Lecture 02 - 2017.09.13
- Lecture 03 - 2017.09.20
- Lecture 04 - 2017.09.27
- Lecture 05 - 2017.10.04
- Lecture 06 - 2017.10.11
- Lecture 07 - 2017.10.18
- Lecture 08 - 2017.10.25
- Lecture 09 - 2017.11.01
- Lecture 10 - 2017.11.08
- Lecture 11 - 2017.11.15
- Lecture 12 - 2017.11.20 -> This post
- Lecture 13 - 2017.11.29
- Lecture 14 - 2017.12.06
Lecture 12 - Nov 20 2017
Some notes
- Exponential distribution and geometry distribution are the only distributions which have the memoryless property (continuous and discrete).
- Geometry is not scalable. It only takes integer values. $ T=\frac{1}{60}Geom(p) $ is not a geometry distribution.
Exercise: see the textbook’s treatment of discrete mixtures of continuous r.v.’s.
Example 1
Let $ X\sim Exp(\lambda) $ and $ Y\sim Exp(\mu) $, $ \varepsilon \sim Bernoulli(\frac{1}{2}) $. They are all independent.
let $ Z=X $ if $ \varepsilon=0 $ and $ Z=-Y $ if $ \varepsilon = 1 $.
Exercise: See book on `continuous mixtures’.
Example 2
$ X\sim Poisson(\lambda) $. Assume $ \lambda $ itself is random. $ \lambda\sim \Gamma(m,\theta) $.
Easy to say, $ X $ is Poisson conditional on $ \lambda $, but what is the unconditional distribution of $ X $?
Answer is in the book. We just want to compute $ P(X=k)$ for $ k=0,1,2,… $.
Bivariate Normal
Let $ X \sim N(\mu,\sigma^2)$. We know we can represent $ X $ as $ X=\mu+\sigma Z $ where $ Z\sim N(0,1) $.
More generally, let $ X_1, X_2 $ be bivariate normal. It turns out that we can represent $ X_2 $ using $ X_1 $ and an independent component $ \varepsilon_2 $ like this:
\[X_2=a+bX_1+\varepsilon_2\]where $ a \& b $ are constants. $ \varepsilon_2 $ is a normal r.v. independent of $ X $ , with $ E(\varepsilon_2) $.
We would like to compute $ a $ and $ b $ and $ Var(\varepsilon_2) $. All we know is
\[Var(X_1)=\sigma_1^2\] \[Var(X_2)=\sigma_2^2\] \[Corr(X,Y)=\rho\]To simplify, assume $ \sigma_1=\sigma_2 $, then we know, from linear prediction, that $ b=\rho $. Then for $ Var(\varepsilon_2)$:
\[\begin{align*} Var(X_2) & = 1= Var(bX_1) + Var(\varepsilon_2) \\ &= Var(\varepsilon_2) = 1-\rho^2 \end{align*}\]Also note: by taking expectation of the whole model,
\[\mu_2=a+b\mu_1+0\]Therefore, $ a=\mu_2-\rho\mu_2 $.
Going back to our work with densities for the multivariate normal. We find the following density for the pair $ X=(X_1,X_2) $:
\[f(x_1,x_2) = const\cdot \exp\big(-\frac{1}{2d}(x_1^2-2\rho x_1x_2 + x_2^2)\big)\]where $ d=1-\rho^2 $, and const $ =\frac{1}{2\pi d} $. Notice that
\[d=1-\rho^2=\det\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\]Also note: the expression
\[Q(x)=Q(x_1,x_2) = (x_1^2-2\rho x_1x_2+x_2^2)/(2d)\]is the ``quadratic form’’, which we encountered as the term $ 1/2 x^T[cov(X)]^{-1}x $. Go back and check this is true.
When $ \rho=0 $, const $\frac{1}{2\pi}$.
\[f(x_1,x_2) = \frac{1}{2\pi}\exp(-\frac{1}{2}(x_1^2+x_2^2)) = \frac{1}{2\pi} \exp({-\frac{1}{2}}x_1^2)\cdot \exp(-\frac{1}{2}x_2^2)\]This proves the independence ($\rho=0$).
- ← Older-STT 861 Theory of Prob and STT I Lecture Note - 11
- Things to Do After Installing Ubuntu-Newer →