STT 861 Theory of Prob and STT I Lecture Note - 12

2017-11-20

Examples on continuous distirbution conditional on discrete distribution; bivariate normal distribution. Not much for today.

Portal to all the other notes

Lecture 12 - Nov 20 2017

Some notes

Exponential distribution and geometry distribution are the only distributions which have the memoryless property (continuous and discrete).
Geometry is not scalable. It only takes integer values. $T = \frac{1}{60} G e o m (p)$ is not a geometry distribution.

Exercise: see the textbook’s treatment of discrete mixtures of continuous r.v.’s.

Example 1

Let $X \sim E x p (λ)$ and $Y \sim E x p (μ)$ , $ε \sim B e r n o u l l i (\frac{1}{2})$ . They are all independent.

let $Z = X$ if $ε = 0$ and $Z = - Y$ if $ε = 1$ .

Exercise: See book on `continuous mixtures’.

Example 2

$X \sim P o i s s o n (λ)$ . Assume $λ$ itself is random. $λ \sim Γ (m, θ)$ .

Easy to say, $X$ is Poisson conditional on $λ$ , but what is the unconditional distribution of $X$ ?

Answer is in the book. We just want to compute $P (X = k)$ for $k = 0, 1, 2, \dots$ .

Bivariate Normal

Let $X \sim N (μ, σ^{2})$ . We know we can represent $X$ as $X = μ + σ Z$ where $Z \sim N (0, 1)$ .

More generally, let $X_{1}, X_{2}$ be bivariate normal. It turns out that we can represent $X_{2}$ using $X_{1}$ and an independent component $ε_{2}$ like this:

X_{2} = a + b X_{1} + ε_{2}

where $a & b$ are constants. $ε_{2}$ is a normal r.v. independent of $X$ , with $E (ε_{2})$ .

We would like to compute $a$ and $b$ and $V a r (ε_{2})$ . All we know is

V a r (X_{1}) = σ_{1}^{2}

V a r (X_{2}) = σ_{2}^{2}

C o r r (X, Y) = ρ

To simplify, assume $σ_{1} = σ_{2}$ , then we know, from linear prediction, that $b = ρ$ . Then for $V a r (ε_{2})$ :

\begin{aligned} V a r (X_{2}) & = 1 = V a r (b X_{1}) + V a r (ε_{2}) \\ = V a r (ε_{2}) = 1 - ρ^{2} \end{aligned}

Also note: by taking expectation of the whole model,

μ_{2} = a + b μ_{1} + 0

Therefore, $a = μ_{2} - ρ μ_{2}$ .

Going back to our work with densities for the multivariate normal. We find the following density for the pair $X = (X_{1}, X_{2})$ :

f (x_{1}, x_{2}) = c o n s t \cdot \exp (- \frac{1}{2 d} (x_{1}^{2} - 2 ρ x_{1} x_{2} + x_{2}^{2}))

where $d = 1 - ρ^{2}$ , and const $= \frac{1}{2 π d}$ . Notice that

d = 1 - ρ^{2} = det (\begin{matrix} 1 & ρ \\ ρ & 1 \end{matrix})

Also note: the expression

Q (x) = Q (x_{1}, x_{2}) = (x_{1}^{2} - 2 ρ x_{1} x_{2} + x_{2}^{2}) / (2 d)

is the quadratic form’’, which we encountered as the term $1 / 2 x^{T} [c o v (X)]^{- 1} x$ . Go back and check this is true.

When $ρ = 0$ , const $\frac{1}{2 π}$ .

f (x_{1}, x_{2}) = \frac{1}{2 π} \exp (- \frac{1}{2} (x_{1}^{2} + x_{2}^{2})) = \frac{1}{2 π} \exp (- \frac{1}{2} x_{1}^{2}) \cdot \exp (- \frac{1}{2} x_{2}^{2})

This proves the independence ( $ρ = 0$ ).

← Older-STT 861 Theory of Prob and STT I Lecture Note - 11

Things to Do After Installing Ubuntu-Newer →

Disqus Comment （0）

LanternD's Castle

An electronics enthusiast - survive technically