Random Variables

Example 1: We roll a fair die, X is the number shown on the die
Example 2: We roll a fair die, X is 1 if the die shows a six, 0 otherwise.
Example 3: We roll a a fair die until the the first "6", X is the number of rolls needed.
Example 4: We randomly pick a time between 10am and 12 am, X is the minutes that have passed since 10am.

There are two basic types of r.v.'s:
If X takes countably many values, X is called a discrete r.v.
If X takes uncountably many values, X is called a continuous r.v.

In Examples 1, 2 and 3 above X is discrete, in Example 4 X is continuous.

There are some technical difficulties when defining a r.v. on a sample space like , it turns out to be impossible to define it for every subset of without getting logical contradictions. The solution is to define a s-algebra on the sample space and then define X only on that s-algebra. We will ignore these technical difficulties.

Almost everything to do with r.v.'s has to be done twice, once for discrete and once for continuous r.v.'s. This separation is only artificial, it goes away once a more general definition of "integral" is used (Rieman-Stilties or Lebesgue)

(Commulative) Distribution Function

Example 1: say x=2.2, then P(X≤x) = P(X≤2.2) = P({1,2}) = 2/6 =1/3

Example 4: say x=67.5, then P(X≤67.5) = P(we chose a moment between 10am and 11h7.5min am) = 67.5/120 = 0.5625

Some features of cdf's:
1) cdf's are standard functions on
2) 0≤F(x)≤1 x
3) cdf's are non-decreasing
4) cdf's are right-continuous
5)

Example : find the cdf F of the random variable X in Example 3 above.
Solution: note X{1,2,3,...}
let A_i be the event "a six on the i^th roll", i=1,2,3, .... Then

and

so for k≤x<k+1 we have F(x)=1-(5/6)^k

Probability Mass Function (pmf) - Probability Density Function (pdf)

Example : the pmf of X in Example 3 is given by f(x) = 1/6*(5/6)^x-1 if x{1,2,..}, 0 otherwise.

Note that it follows from the definition and the axioms that for any pmf f we have

ExampleSay f(x)=c/x², x=1,2,3,... is a pmf. Find c

so c=6/p².

The function f is called the probability density function of the continuous random variable X iff

Again it follows from the definition and the axioms that for any pdf f we have

Example: Show that f(x)=lexp(-lx) if x>0, 0 otherwise defines a pdf, where l>0
Solution: clearly f(x)≥0 for all x.

This r.v. X is called an exponential r.v. with rate l.

Example Say f(x)=c/x², x>1 is a pdf. Find c.

Example Say f(x)=cxsin(px), 0≤x≤1 is a pdf. Find c.

Is this correct? Let's use R to check:
Define function with f=function(x) {x*sin(pi*x)}
get range of x values with x=seq(0,1,length=100)
Let's see what this looks like with plot(x,f(x),type="l")
Find the integral with integrate(f,0,1)
but 1/p=0.3183099, so we did ok.

Example Say f(x)=ce^-x2 is a pdf. Find c.
Unfortunately f does not have an anti-derivative, so this is tricky problem. We can still use R, though:
Define function with f=function(x) {exp(-x^2)}
to draw this, what range of x values should we use? Note that f(3)=0.0001234098 and f is symmetric, so we can use
x=seq(-3,3,length=100)
Let's see what this looks like with plot(x,f(x),type="l")
integrate(f,-3,3) gives us c=1/1.77

Random Vectors

Example 1 : we roll a fair die twice. Let X be the sum of the rolls and let Y be the absolute difference between the two roles. Then (X,Y) is a 2-dimensional random vector. The joint pmf of (X,Y) is given by:

X\Y	0	1	2	3	4	5
2	1	0	0	0	0	0
3	0	2	0	0	0	0
4	1	0	2	0	0	0
5	0	2	0	2	0	0
6	1	0	2	0	2	0
7	0	2	0	2	0	2
8	1	0	2	0	2	0
9	0	2	0	2	0	0
10	1	0	2	0	0	0
11	0	2	0	0	0	0
12	1	0	0	0	0	0

where every number is divided by 36.

all definitions are straightforward extensions of the one-dimensional case.

Example 1, cont for a discrete random vector we have the pmf f(x,y) = P(X=x,Y=y)
Say f(4,0) = P(X=4, Y=0) = P({(2,2)}) = 1/36 or f(7,1) = P(X=7,Y=1) = P({(3,4),(4,3)}) = 1/18

Example 2 Say f(x,y)=cxy is a pmf with x{1,2,3} and y{0,2}. Find c.
1=∑_x∑_y f(x,y) = f(1,0)+f(1,2)+f(2,0)+f(2,2)+f(3,0)+f(3,2) = 1·0+1·2+2·0+2·2+3·0+3·2 = 12, so c=1/12

Example 3 Say f(x,y)=cxy, 0≤x,y≤1 is a pdf. Find c.

so c=4.

Example 4 Say f(x,y)=cxy, 0≤x<y≤1 is a pdf. Find c.

so c=8.

Example 5 Say (X,Y) is a discrete rv with joint pmf f(x,y)=cp^x, x,y{0,1,..}, y≤x, and 0<p<1. Find c

Marginal Distributions

Example 1, cont we find f_X(2) = f(2,0) + f(2,1) + .. + f(2,5) = 1/36 or f_Y(3) = 6/36

Example 2, cont f_Y(0) = f(1,0) + f(2,0) + .. + f(3,0) = 0

Example 4, cont
Find f_Y(y)

Note that f_Y(y) is s proper pdf: f_Y(y)≥0 for all y[0,1] and

Conditional R.V.'s

Example 1, cont.: find f_X|Y=5(7|5) and f_Y|X=3(7|3)

Example 4, cont Find f_X|Y=y(x|y)
f_X|Y=y(x|y) = f(x,y)/f_Y(y) = 8xy/4y³ = 2x/y², 0≤x≤y. Here y is a fixed number!
Again, note that a conditional pdf is a proper pdf:

Note that a conditional pmf (pdf) requires a specification for a value of the random variable on which we condition, something like f_X|Y=y. An expression like f_X|Y is not defined!

Independence

Example 1, cont. we found f_X,Y(7,1) = 1/18 but f_X(7)*f_Y(1) = 1/6*10/36=5/108, so X and Y are not independent

Mostly the concept of independence is used in reverse: we assume X and Y are independent (based on good reason!) and then make use of the formula:
Say we use the computer to generate 10 independent exponential r.v's with rate l. What is the probability density function of this random vector?
We have f_Xi(x_i)=lexp(-lx_i) for i=1,2,..,10 so
f_(X1,..,X10)(x₁, .., x₁₀) = lexp(-lx₁) *..* lexp(-lx₁₀) = l¹⁰exp(-l(x₁+..+x₁₀))

Notation: we will use the notation X Y if X and Y are independent

Example

f_X|Y=2(1|2)=f(1,1)/f_Y(2) = (1/10)/(7/10) = 1/7
f_X|Y=2(2|2)=f(2,1)/f_Y(2) = (1/2)/(7/10) = 5/7
f_X|Y=2(3|2)=f(3,1)/f_Y(2) = (1/10)/(7/10) = 1/7
so

Example

Show that f is indeed a proper density

Find f_Y|X=x(y|x)

f_Y|X=x(y|x) = f(x,y)/f_X(x)

so

Show that f_Y|X=x(y|x) is also a proper density

Example

We have a discrete r.v X with pmf f_X(x)=0.5^x, x=1,2,.. , and a conditional rv Y with pmf f_Y|X=x(y|x)=xexp(-xy), y>0. We want F_Y(y)=P(Y≤y). Now first we have


a little bit of care: the geometric series ∑q^k only converges if |q|<1. Here y>0, so e^y>1 so 1/2e^y<0.5<1, we are save.

So, quite a calculation! Are we sure it's correct?

Test 1: for a cdf we have lim_x→∞F(y)=1. True for our result!

Test 2: use R:
x=rgeom(10000,0.5)+1
Y=rexp(10000,1/x)
y=1;length(Y[Y<y])/10000;2-2*exp(y)/(2*exp(y)-1)

or even better:

y=seq(0,4,length=100)
z1=2-2*exp(y)/(2*exp(y)-1)
z2=0*z1
for(i in 1:100) z2[i]=length(Y[Y<y[i]])/10000
plot(y,z1,type="l")
lines(y,z2,lty=2)

This type of model is called a hierarchical model, with one rv defined conditional on another. This way of describing a model is very useful in real live.

This is also an example for a mixture of a discrete and a continuous rv.

Law of Total Probability

Discrete-Discrete
Say X and Y are discrete rv's with pmf's f_X and f_Y, resp. Let B={X=x} and A_y={Y=y}. Then {A_y, all yS} form a partition and we have
f_X(x) = P(X=x) = P(B) = ∑_yP(B|A_y)P(A_y) = ∑_yf_X|Y=y(x|y)f_Y(y)

Discrete-Continuous
Say X is a discrete rv with pmf f_X and Y is a continuous rv with density f_Y
Here we need to be careful: for a discrete rv f_X(x)=P(X=x) makes sense, but for a continuous one we have
P(Y=y) = lim_h→0P(y≤Y≤y+h) = lim_h→0∫_y^y+h f_Y(t)dt = lim_h→0(F_Y(y+h)-F_Y(y)) = 0 for all y
first we condition on the discrete rv. now the event B={Y=y} does not work because P(B)=0 for all y. Let's instead consider the event B={Y≤y}:

For conditioning on the continuous rv we need to define a new discrete rv Y' with Y'=ih if ih≤Y<(i+1)h. Then

because this is a Riemann sum, so it converges to the corresponding integral.

Continuous-Continuous Actually, same as above, with the same proof: