Some Special Method

Exponential Distribution - General Inverse Method

We have already seen that if X~U[0,1] and l>0, then Y=-llog(X) ~ Exp(1/l)

This is actually a special case of a general method: let X be a continuous r.v. with cdf F. Let F-1 be the generalized inverse of F, that is F-1(y) = inf{x:F(x)≥y}
the generalized inverse is illustrated in the next figure: at a point x where F is strictly increasing it's just the regular inverse, but at a point where F is flat it gives the x where F starts to be flat:

so F(x1)=y1 and F-1(y1)=x1
but
F(x2)=F(x3)=F(x4)=y2 and F-1(y2)=x2

Now say we want to generate a r.v. X with cdf F. Let U~U[0,1], then X=F-1(U) ~ F because

The name of this method comes from a very useful theorem:

Theorem: Probability Integral Transform
Say X is a continuous rv with cdf F. Then Y=F(X)~U[0,1].

proof
FY(y) = P(Y≤y) = P(F(X)≤y) = P(X≤F-1(y)) = F(F-1(y)) = y
where we use the fact that if F is increasing, so is F-1

Example R check:

x=rnorm(1000,10,3)
hist(x,breaks=50)
y=pnorm(x,10,3)
hist(y,breaks=50)

Unfortunately the exponential is one of only a few examples of rv's we can generate with this method because it is one of the few r.v's with an explicit formula for the cdf. Here is another:

Example let U~U[0,1], a>0 and X=U1/a, then if 0≤u≤1

FX(x) = P(X≤x) = P(U1/a≤x) = P(U≤xa) =xa

so fX(x)=axa-1, or X~Beta(a,1)

Similarly 1-U1/b~Beta(1,b)

Knowing how to generate exponentials is important because it has a relationship with some of the other r.v.s we have discussed and this can be used to generate some of them. For example

Binomial Distribution

Say we want to generate X ~B(n,p). Now we know that if Y1,..,Yn are iid Ber(p) then Y1+..+Yn ~ B(n,p).
so let Ui~U[0,1], Yi = I(0,p)(Ui) and X = Y1+..+Yn, then X~B(n,p)

Normal Distribution (Box-Muller algorithm)

Say U1 and U2 are iid U[0,1] and set

then X and Y are independent standard normal r.v.s

the Jacobian of this transform is:

The problem with this algorithm is that it requires the computation of the sin and the cos functions. Here is a similar and much faster algorithm:
1) generate U1 and U2 are iid U[0,1]
2) set V1 = 2U1 -1, V2 = 2U2 -1 and S =V12 + V22
3) If S>1, return to step 1
otherwise set

then X and Y are iid standard normal. (This is called the polar method)