Random Variables

Example 1: We roll a fair die, X is the number shown on the die
Example 2: We roll a fair die, X is 1 if the die shows a six, 0 otherwise.
Example 3: We roll a a fair die until the the first "6", X is the number of rolls needed.
Example 4: We randomly pick a time between 10am and 12 am, X is the minutes that have passed since 10am.

There are two basic types of r.v.'s:
If X takes countably many values, X is called a discrete r.v.
If X takes uncountably many values, X is called a continuous r.v.

In Examples 1, 2 and 3 above X is discrete, in Example 4 X is continuous.

There are some technical difficulties when defining a r.v. on a sample space like , it turns out to be impossible to define it for every subset of without getting logical contradictions. The solution is to define a σ-algebra on the sample space and then define X only on that σ-algebra. We will ignore these technical difficulties.

Almost everything to do with r.v.'s has to be done twice, once for discrete and once for continuous r.v.'s. This separation is only artificial, it goes away once a more general definition of "integral" is used (Rieman-Stilties or Lebesgue)

Notice that all the formulas from before are still valid for random variables because this is a direct extension of the probability of events

Example say we roll two fair dice and X is the sum of the dice. Then S={(1,1),..,(6,6)} and

P(X=3) = P({ωS : X(ω)=3}) = P({(1,2),(2,1)}) = 2/36

(Commulative) Distribution Function

Example 1: say x=2.2, then P(X≤x) = P(X≤2.2) = P({1,2}) = 2/6 =1/3

Example 4: say x=67.5, then P(X≤67.5) = P(we chose a moment between 10am and 11h7.5min am) = 67.5/120 = 0.5625

Some features of cdf's:
1) cdf's are standard functions on
2) 0≤F(x)≤1 x
3) cdf's are non-decreasing
4) cdf's are right-continuous
5)

Example : find the cdf F of the random variable X in example 3 above.
Solution: note X{1,2,3,...}
let A_i be the event "a six on the i^th roll", i=1,2,3, .... Then

and

so for k≤x<k+1 we have F(x)=1-(5/6)^k

Let's write a routine in R that draws this function. First we need to find out the were to draw it, after all F is defined on the positive half-line, but we need to stop somewhere. Lets say we go until F(k)=0.99, so 1-(5/6)^k=0.99, or
(5/6)^k=0.01, k·log(5/6)=log(0.01) or k=log(0.01)/log(5/6)=25.25, which we round up to 26 with the ceiling function. The rgraph is drawn in cdf.ex().

Probability Mass Function (pmf) - Probability Density Function (pdf)

Example : the pdf of X in Example 3 is given by f(x) = 1/6*(5/6)^x-1 if x{1,2,..}, 0 otherwise.

Note that it follows from the definition and the axioms that for any pmf f we have

The function f is called the probability density function of the continuous random variable X iff

Again it follows from the definition and the axioms that for any pdf f we have

Example: Show that f(x)=λexp(-λx) if x>0, 0 otherwise defines a pdf, where λ>0
Solution: clearly f(x)≥0 for all x.

This r.v. X is called an exponential r.v. with rate λ.

Random Vectors

Example : we roll a fair die twice. Let X be the sum of the rolls and let Y be the absolute difference between the two roles. Then (X,Y) is a 2-dimensional random vector. The joint pmf of (X,Y) is given by:

X\Y	0	1	2	3	4	5
2	1	0	0	0	0	0
3	0	2	0	0	0	0
4	1	0	2	0	0	0
5	0	2	0	2	0	0
6	1	0	2	0	2	0
7	0	2	0	2	0	2
8	1	0	2	0	2	0
9	0	2	0	2	0	0
10	1	0	2	0	0	0
11	0	2	0	0	0	0
12	1	0	0	0	0	0

where every number is divided by 36.

all definitions are straightforward extensions of the one-dimensional case.

Example : for a discrete random vector we have the pmf f(x,y) = P(X=x,Y=y)
Say f(4,0) = P(X=4, Y=0) = P({(2,2)}) = 1/36 or f(7,1) = P(X=7,Y=1) = P({(3,4),(4,3)}) = 1/18

Say (X,Y) is a discrete (continuous) r.v. with joint pmf (pdf) f. Then the marginal pmf (pdf) f_X is given by

For the example above we find f_X(2) = f(2,0) + f(2,1) + .. + f(2,5) = 1/36 or f_Y(3) = 6/36

Example Say f(x,y)=cxy, 0≤x<y≤1, find the constant c.

Find the marginal density of Y, f_Y(y)

Note that f_Y(y) is a proper pdf: f_Y(y)≥0 for all y[0,1] and

xy.ex() generates data from this distribution, draws the histogram of Y and adds this density.

Conditional R.V.'s

Example: for the discrete rv above find f_X|Y=5(7|5) and f_Y|X=7(3|7)

First the marginal pmf of Y is given by

y	0	1	2	3	4	5
fY(y)	6/36	10/36	8/36	6/36	4/36	2/36

so f_X|Y=5(7|5) = f(7,5)/f_Y(5) = (2/36)/(2/36) =1

f_X(x) = (x-1)/36 if x=2,..,7 and (13-x)/36 if x=8,..,12, so

f_Y|X=7(3|7) =f(7,3)/f_X(7) = (2/36)/(6/36) =1/3

Note

y	0	1	2	3	4	5
fY|X=6(y|6)	1/5	0	2/5	0	2/5	0

so Z=Y|X=6 is a proper discrete rv.

For continous r.v. everything works the same.

Example Say f(x,y)=8xy, 0≤x<y≤1, f ind f_X|Y=y(x|y)
f_X|Y=y(x|y) = f(x,y)/f_Y(y) = 8xy/4y³ = 2x/y², 0≤x≤y. Here y is a fixed number!
Again, note that a conditional pdf is a proper pdf:

xy.ex() generates data from this distribution. We want to check X|Y=y, but of course for a continuous rv and any fixed number y P(Y=y)=0. so we need to be a bit generous and use all observations from X with corrsponding Y values in [y-ε,y+ε] for some small ε, say 0.01. Then we draw the histgam of these x values and add the conditional density.

Note that a conditional pmf (pdf) requires a specification for a value of the random variable on which we condition, something like f_X|Y=y. An expression like f_X|Y is not defined!

Independence

Example : in the discrete example above we found f_X,Y(7,1) = 1/18 but f_X(7)f_Y(1) = 1/6·10/36=5/108, so X and Y are not independent

Example (X,Y) have a joint pdf f(x,y)=8xy, 0≤x<y≤1 Clearly X and Y are not independent because f_X(x)>0 for 0<x<1, f_Y(x)>0 for 0<y<1 but f(x,y)=0 if y<x

Mostly the concept of independence is used in reverse: we assume X and Y are independent (based on good reason!) and then make use of the formula:
Say we use the computer to generate 10 independent exponential r.v's with rate λ. What is the probability density function of this random vector?
We have f_Xi(x_i)=λexp(-λx_i) for i=1,2,..,10 so
f_(X1,..,X10)(x₁, .., x₁₀) = λexp(-λx₁) *..* λexp(-λx₁₀) = λ¹⁰exp(-λ(x₁+..+x₁₀))

Notation: we will use the notation X Y if X and Y are independent