Hypergeometric distribution; Poisson Law; Brownian motion; continous random variables, exponential distribution, cumulative distribution function, uniform distribution; expectation and variance of continuous random variable.

Portal to all the other notes

• Lecture 01 - 2017.09.06
• Lecture 02 - 2017.09.13
• Lecture 03 - 2017.09.20
• Lecture 04 - 2017.09.27
• Lecture 05 - 2017.10.04
• Lecture 06 - 2017.10.11 -> This post
• Lecture 07 - 2017.10.18
• Lecture 08 - 2017.10.25
• Lecture 09 - 2017.11.01
• Lecture 10 - 2017.11.08
• Lecture 11 - 2017.11.15
• Lecture 12 - 2017.11.20
• Lecture 13 - 2017.11.29
• Lecture 14 - 2017.12.06

Lecture 06 - Oct 11 2017

Hypergeometric distribution (continued)

(See Page 66)

A hypergeometric r.v. $X$ with parameters $n,N,R$:

Have a set of size $N$, a subset of size $R$ (‘red’). Pick a sample of size $n$ without replacement. Then $X$ is the number of elements of type $R$ (‘red’) in the sample.

Then for $k=0,\dots ,n$,

Important: let $p=\frac{R}{N}$

The correction factor $\frac{N-n}{N-1}$ is called the finite sample correction factor.

Poisson Law

Let $N$ and $M$ be $Poi\left(\lambda \right)$ and $Poi\left(\mu \right)$ and independent. Then $N+M$ is $Pos(\lambda +\mu )$.

(Think of it as arrivals in a store on 2 different days.)

Therefore, $N\left(1\right)$ is $Poi\left(\lambda \right)$. By the same construction $N\left(t\right)$ is $Poi\left(\lambda t\right)$.

Now as $t$ funs from 0 to 1, we have a collection or r.v’s $\left\{N\right(t):t\in [0,1\left]\right\}$. This is the Poisson process with parameter $\lambda$.

These are the properties of this $Poi\left(\lambda \right)$ process. [a stochastic process, like $N$, is a random function where the rules of randomness are specified.]

The word stochastic comes from the Greek word “stochos” ($\sigma \tau o\chi o\sigma$). It means “target”.

It comes from the stroy of Aristotle and target distribution.

Let $0<s<t$, then,

1. $N\left(t\right)-N\left(s\right)\sim Poi\left(\lambda \right(t-s\left)\right)$
2. $N\left(s\right)$ and $N\left(t\right)-N\left(s\right)$ are independent.

Vocabulary: (2) is called independence of increments. And (1) is usually called stationarity.

Remark: $N\left(t\right)+N\left(s\right)$ does not $\sim P\left(\lambda \right(t+s\left)\right)$.

What is $N\left(t\right)+N\left(s\right)$? $=N\left(t\right)+N\left(s\right)\sim Poi\left(\lambda \right(t-s\left)\right)+2Poi\left(\lambda s\right)$

$2Poi\left(\lambda s\right)$ is not $Poi\left(2\lambda s\right)$. They might have different variance.

Remark: (1) and (2) define the prob distribution of the $Poi\left(\lambda \right)$ process uniquely.

Example 2.5.2 (Page 77)

Murder rate $540/year$. Assume # murders in the interval $[0,t]$ where $t=\text{proportion of a 360-day year}$ is a Poisson process $N\left(t\right)$.

Q1: What is prob of two or more murders with in 1 day $P\left(N\right(1day)\ge 2)$.

A1: $\lambda =\frac{540}{360}=1.5$

Q2: What is prob of “2 or more murders for each of 3 consecutive days”?

A2: $P_2=P_1^3={0.4422}^3=0.0865$

Q3: What is the prob of “No murder for 5 days”?

A3: $P_3=P\left(N\right(5days)=0)=P\left(Poi\right(\frac{15}{2})=0)=e^{-\frac{15}{2}}$

Q4: Rate during weekdays is 1.2/day, during weekends is 2.5/day. What is the prob of “10 or more murders in 1 week?

A4:

HW discussion (Problem 2.4.4 (a))

$X\sim NegBin(r_1,p)$, $Y\sim NegBin(r_2,p)$. $W=X+Y$. Find the distribution of $W$.

We know that $X=X_1+X_2+\cdots +X_{r_1}$, where the $X_i$’s are i.i.d. $Geom\left(p\right)$. Similarly we have $Y_i$s are i.i.d. $Geom\left(p\right)$.

Now if we assume $X$ and $Y$ are independent. Then all the $X_i$s and $Y_i$s are independent. Then $W=X_1+\cdots +X_{r_1}+Y_1+\cdots +Y_{r_2}$ are $r_1+r_2$ i.i.d. $Geom\left(p\right)$. Therefore by definition $W$ is $NegBin(r_1+r_2,p)$.

Preview to Chapter 3 and Brownian motion

Let $X_1,X_2,\dots ,X_n$ be a sequence of i.i.d. r.v.s with $E\left[X_i\right]=0$ and $Var\left[X_i\right]={\sigma }^2>0$.

Let $S_n={\overline{X}}_n=\frac{X_1+X_2+\dots +X_n}{n}$, then $E\left(S_n\right)=0$ and it turns out $S_n$ gets really small as $n\to \infty$.

as $n\to \infty$.

In fact we have the weak law of large numbers:

$\forall ϵ>0$, $P(|S_n|>ϵ)\to 0$ as $n\to \infty$.

Proof: By Chebyshev’s inequality:

What about dividing $X_1+X_2+\cdots +X_n$ by something much smaller than $n$?

Let $W_n=\frac{X_1+X_2+\cdots +X_n}{\sqrt{n}}$, then $E\left[W_n\right]=0$ when $n$ is large. $Var\left[W_n\right]={\sigma }^2$.

So we has a maybe limiting behavior? Yes. Distribution of $W_n$ tends to bell curve (Normal distribution with ${\sigma }^2$ variance).

Pick $t\in [0,1]$. Roughly speaking $m\approx nt$.

Let

we find, $E\left[W_n\right]=0$, $Var\left[W_n\right]=t{\sigma }^2$.

The distribution $W_n\left(t\right)$ converges to Bell curve with variance $=t{\sigma }^2$.

The whole entire collection of $\left\{W_n\right(t):t\in [0,1\left]\right\}$ converges to a stochastic process called Brownian motion $W$. It has these properties:

1. $E\left[W\right(t\left)\right]=0$;
2. $Var\left[W\right(t)-W(s\left)\right]={\sigma }^2(t-s)$;
3. $W\left(t\right)-W\left(s\right)$ and $W\left(s\right)$ are independent;
4. $W\left(t\right)-W\left(s\right)$ is Normal (bell curve).

Continuous Random Variables (Chapter 3)

Definition: A r.v. $X$ is said to be continuous with density $f$ is: $\forall a<b$,

Properties of densities:

1. $f\left(x\right)\ge 0,\forall x$
2. ${\int }_{-\infty }^{\infty }f\left(x\right)d=1$

Example 1

let $f\left(x\right)=c{e}^{-\lambda x}$, where $x\ge 0$, $c$ and $\lambda$ are positive constants. How should we define $f\left(x\right)$ for $x<0$?

Let’s say $f\left(x\right)=0,\forall x<0$. Do the integration.

Exponential distribution

Definition: The r.v. whose density is

$f\left(x\right)=0\forall x<0,\lambda e^{-\lambda x}\forall x\ge 0$ is called Exponential with parameter $\lambda$.

Notation: $X\sim Exp\left(\lambda \right)$

Cumulative distribution function (CDF)

Definition: The cumulative distribution function (CDF) of $X$ with density is defined as before:

• The fundamental theorem of calculus (Leibniz) says $F$ has a derivative, it is $f$.
• Also we see: $F\in [0,1]$
• $F$ is non-decreasing
• $\underset{x\to -\infty }{lim}F\left(x\right)=0$
• $\underset{x\to \infty }{lim}F\left(x\right)=1$

Example 2

If $X\sim Exp\left(\lambda \right)$, then we find $F\left(x\right)=0$ for $x\le 0$.

Note: the tail of $X$ is defined as $G\left(x\right)=P(X>x)=1-F\left(x\right)$.

For Exponential, $G\left(x\right)=e^{-\lambda }$.

Remark: If $X$ has density $f$ and $a\in R$, then $P(x=a)={\int }_a^{a}f\left(x\right)dx=0$.

Uniform distribution

The r.v. $X$ is said to be uniform between 2 fixed values $a$ and $b$ if its density is a constant $\frac{1}{b-a}$.

What about the CDF? It has 0 at $x=a$ and 1 at $x\ge b$.

Expectation and variance of continuous random variable (Chapter 3.3)

Definition: let $X$ have density $f$, then its expectation is ${\int }_{-\infty }^{\infty }xf\left(x\right)dx$

Example 3

$X\sim Exp\left(\lambda \right)$, $E\left[X\right]={\int }_{-\infty }^{\infty }xf\left(x\right)dx$.

Integration by parts:

Best choice: use a $dv$ whose $d$ is known, and if $u$ is a polynomial that’s good because $du$ has a degree one less than $u$. Choice: $dv=\lambda e^{-\lambda x}dx$, $v=-e^{-\lambda x}$, $u=x$, $du=dx$.

Exercise: $X\sim Unif[a,b]$, find $E\left(X\right)=\frac{a+b}{2}$.

Definition: the variance of $X$ with density $f$ is defined the same as before:

where $E\left[X^2\right]=\int x^{2}f\left(x\right)dx$.

Exercise: Verify that $E\left[X^2\right]=\frac{2}{{\lambda }^2}$ for $X\sim Exp\left(\lambda \right)$, and $Var\left[X\right]=\frac{1}{{\lambda }^2}$.

$Var\left[X\right]=\frac{(a-b{)}^2}{12}$ for $X\sim Unif(a,b)$.