Probability 1: Fundamentals

There are three basic interpretations of probability:

• empirically through many repetitions of an experiment - relative frequency interpretation
• through reasoning about outcomes etc. - classical interpretation
• by using our intuition and experience - subjective interpretation

Example - coin tossing
what is the probability of getting "heads" when tossing a fair coin?

• relative frequency interpretation: take a coin and flip it!
the South African mathematician Jon Kerrich, while in a German POW camp during WWII tossed a coin 10000 times. Result 5067 heads, for a probability of 0.5067

• classical interpretation:
This experiment has two possible outcomes - heads and tails. Fair means they are equally likely, so p=P("heads")=P("tails")=0.5

• subjective interpretation: I think it's ½

An experiment is a well-defined procedure that produces a set of outcomes. For example , "roll a die"; "randomly select a card from a standard 52-card deck"; "flip a coin" and "pick any moment in time between 10am and 12 am" are experiments. A sample space is the set of outcomes from an experiment. Thus, for "flip a coin" the sample space is {H, T}, for "roll a die" the sample space is {1, 2, 3, 4, 5, 6} and for "pick any moment in time between 10am and 12 am" the sample space is [10, 12].
An event is a subset, say A, of a sample space S. For the experiment "roll a die", an event is "obtain a number less than 3". Here, the event is {1, 2}.
If all the outcomes of a sample space S are equally likely and if A is an event, then the probability of A is:

So, the probability of an event, say A, is the ratio of success to total. For example, flipping a coin what is the probability of a heads?  Here, the total number of outcomes is 2 and the number of ways to be successful is 1.  Thus, P(heads) = 1/2. As another example , consider randomly selecting a card from a standard 52-card deck: what is the probability of getting a king?  Here, the total number of outcomes is 52 and of these outcomes 4 would be successful. So, P(king) = 4/52.

Fundamentals

The definition above works well as long as S is finite but breaks down if S is infinite. Instead modern probability, like geometry, is built on a small set of basic rules called axioms, derived in the 1930's by Kolmogorov. They are:

if A1, ..., An are mutually exclusive

Example : Derive the formula above (for a finite sample space) from these axioms.

Solutions: say we have a sample space S={e1, ..., en} and an event A={ek1, ..., ekm}. Then:

Some useful formulas

Complement: P(A) = 1 - P(Ac)

Example : A fair coin is tossed 5 times. What is the probability of at least one "Heads"?
Sample Space S={(H,H,H,H,H),(H,H,H,H,T), ... , (T,T,T,T,T)}
S has 25 = 32 elements
P(at least one "Heads") = 1 - P("No Heads") = 1 - P({(T,T,T,T,T)}) = 1 - 1/36 = 31/32

Addition Formula: P(AB) = P(A)+P(B)-P(A∩B)
We roll two fair dice. What is the probability of a sum of 5 or 8, or highest number on either die is a 3?
Sample Space is above.
Event A = {(1,4), (2,3), (3,2), (4,1), (2,6), (3,5), (4,4), (5,3), (6,2)}, n(A) = 9
Event B = {(1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3)}, n(B) = 9
Event A∩B = {(2,3), (3,2)}, n(A∩B) = 2
P(AB) = P(A)+P(B)-P(A∩B) = 9/36+9/36-2/36 = 16/36 = 4/9