Expectations

The expectation (or expected value) of a random variable g(X) is defined by

We use the notation Eg(X)

Example consider the discrete rv. X with pmf

then E[X] = 0·0.1+1·0.3+2·0.2+3·0.1+4·0.1+5·0.2 = 2.4

E[X²] = 0²·0.1+1²·0.3+2²·0.2+3²·0.1+4²·0.1+5²·0.2 = 8.6

E[e^X] = e⁰·0.1+e¹·0.3+e²·0.2+e³·0.1+e⁴·0.1+e⁵·0.2 = 39.5443

Example : we roll fair die until the first time we get a six. What is the expected number of rolls?
We saw that f(x) = 1/6*(5/6)^x-1 if x{1,2,..}. Here we just have g(x)=x, so

How do we compute this sum? Here is a "standard" trick:

and so we find

Example say X is a continuous rv with density f(x)=2x, 0<x<1. Then

Note that X~Beta(2,1), so it's easy to use R to check these results.

Example X is said to have a uniform [A,B] distribution if f(x)=1/(B-A) for A<x<B, 0 otherwise.
Find EX^k (this is called the k^th moment of X).

Some special expectations are the mean of X defined by μ=EX and the variance defined by σ²=V(X)=E(X-μ)². Related to the variance is the standard deviation σ, the square root of the variance.

Here are some formulas for expectations:

the last one is a useful formula for finding the variance and/or the standard deviation.

Example : find the mean and the standard deviation of a uniform [A,B] r.v.

and so σ=(B-A)/sqrt(12)

Example Find the mean and the standard deviation of an exponential rv with rate λ.

One way to "link" probabilities and expectations is via the indicator function I_A defined as

because with this we have for a continuous r.v. X with density f:

Expectations of Random Vectors

Example Let (X,Y) be a discrete random vector with f(x,y) = (1/2)^x+y, x≥1, y≥1. Find E[XY²]

Example Say f(x,y)=8xy, 0≤x<y≤1, find E[Y]

xy.ex() does this via simulation

Covariance and Correlation

Note cov(X,X) = V(X)

As with the variance we have a simpler formula for actual calculations: cov(X,Y) = E(XY) - (EX)(EY)

Example : take the example of the sum and absolute value of the difference of two rolls of a die. What is the covariance of X and Y? We have
μ_X = EX = 2·1/36 + 3·2/36 + ... + 12·1/36 = 7.0
μ_Y = EY = 0·6/36 + 1·12/36 + ... + 5·2/36 = 70/36
EXY = 0·2·1/36 + 1·2·0/36 + 2·2·0/36 +...+ 5·12·0/36 = 490/36
and so cov(X,Y) = EXY-EX·EY = 490/36 - 7.0·70/36 = 0

Note that we previously saw that X and Y are not independent, so we here have an example that a covariance of 0 does not imply independence! It does work the other way around, though:

Theorem: If X and Y are independent, then cov(X,Y) = 0 ( = cor(X,Y))

proof (in the case of X and Y continuous):

and so cov(X,Y) = EXY-EXEY = EXEY - EXEY = 0

Example Consider again the example from before: we have continuous rv's X and Y with joint density f(x,y)=8xy, 0≤x<y≤1. Find the covariance and the correlation of X and Y.
cov(X,Y)=E[XY]-E[X]E[Y]. We have seen before that E[Y]= 4/5
Now

and

and so cov(X,Y)=4/9-8/15·4/5 = 12/675

Also

xy.ex() finds estimates of these numbers via simulation

We saw above that E(X+Y) = EX + EY. How about V(X+Y)?

and if XY we have V(X+Y) = VX + VY

Conditional Expectation and Variance

Let E[X|Y] denote the function of the random variable Y whose value at Y=y is given by E[X|Y=y]. Note then Z=E[X|Y] is itself a random variable.

Example: An urn contains 2 white and 3 black balls. We pick two balls from the urn. Let X denote the number of white balls chosen. An additional ball is then drawn from the remaining three. Let Y equal 1 if the ball is white and 0 otherwise.
For example f(0,0) = P(X=0,Y=0) = 3/5·2/4·1/3 = 1/10.
The complete pmf is given by:

Y\X	0	1	2
0	1/10	2/5	1/10
1	1/5	1/5	0

The marginals are given by

x	0	1	2
P(X=x)	3/10	3/5	1/10

and

y	0	1
P(Y=y)	3/5	2/5

The conditional distribution of X|Y=0 is

x	0	1	2
P(X=x|Y=0)	1/6	2/3	1/6

and so E[X|Y=0] = 0·1/6+1·2/3+2·1/6 = 1.0

The conditional distribution of X|Y=1 is

x	0	1	2
P(X=x|Y=1)	1/2	1/2	0

and so E[X|Y=1] = 0·1/2+1·1/2+2·0 = 1/2

Finally the conditional r.v. Z = E[X|Y] has pmf

z	1	1/2
P(Z=z)	3/5	2/5

with this we can find E[Z] = E[E[X|Y]] = 1·3/5+1/2·2/5 = 4/5

Example we have continuous rv's X and Y with joint density f(x,y)=8xy, 0≤x<y≤1. We have found f_Y(y) = 4y³, 0<y<1, and f_X|Y=y(x|y) = 2x/y², 0≤x≤y. So

Throughout this calculation we treated y as a constant. Now, though, we can change our point of view and consider E[X|Y=y] = 2y/3 as a function of y:

g(y)=E[X|Y=y]=2y/3

What are the values of y? Well, they are the observations we might get from the rv. Y, so we can also write

g(Y)=E[X|Y=Y]=2Y/3

but Y is a rv, then so is 2Y/3, and we see that we can define a rv Z=g(Y)=E[X|Y]

What is the distibution of Z? Y is a continuous rv, therefore so is Z. Note that 0<Y<1 and so 0<Z<2/3. Now

P(Z<z)=P(2Y/3<z)=P(Y<3z/2)
and so if 0<z<2/3 we have
f_Z(z)=3/2f_Y(3z/2)

We have f_Z(z)≥0 and

Recall that the expression f_X|Y does not make sense. Now we see that on the other hand the expression E[X|Y] makes perfectly good sense!

There is a very useful formula for the expectation of conditional r.v.s: E[E[X|Y]] = E[X]
E[X] = 0*3/10 + 1*3/5 + 2*1/10 = 4/5

There is a simple explanation for this seemingly complicated formula!

Here is a corresponding formula for the variance:
V(X) = E[V(X|Y)] + V[E(X|Y)]

Example: let's say we have a continuous bivariate random vector with the joint pdf f(x,y) = c(x+2y) if 0<x<2 and 0<y<1, 0 otherwise.
Find C

Find the marginal distribution of X

Find the marginal distribution of Y

Find the conditional pdf of Y|X=x

Note: this is a proper pdf for any fixed value of x
Find E[Y|X=x]

Let Z=E[Y|X]. Find E[Z]