Poisson Process

The Poisson process N(t) is an example of a counting process, that is N(t) is the number of times something has happened up to time t.

Example: Consider an ATM machine and let N(t) be the number of customers served by the ATM machine at time t.

Because of the way it is defined every counting process has the following properties:
1) N(t)≥0
2) N(t) is an integer
3) If s<t then N(s)≤N(t)
4) If s<t then N(t)-N(s) is the number of events that have occured in the interval (s,t].

Definition

Let {N(t);t≥0} be a counting process. Then
a) {N(t);t≥0} is said to have independent increments if the number of events that occur in disjoint intervals are independent.
b) {N(t);t≥0} is said to have stationary increments if the distribution of the number of events that occur in any interval of time depend only on the length of the interval.

Example: The process of our ATM machine probably has independent increments but not stationary increments. Why?

The most important example of a counting process is the Poisson process. To define it we need the following notation, called Landau's o symbol:
a function f is said to be o(h) if limh→0f(h)/h = 0

Example: f(x)=x2 is o(h) but f(x)=x is not.

Definition

A counting process {N(t);t≥0} is said to be a Poisson process with rate λ>0 if
1) N(0)=0
2) N(t) has stationary and independent increments
3) P(N(h)=1) = λh + o(h)
3) P(N(h)≥2) = o(h)

Notice that this implies that during a short time period the probability of an event occuring is proportional to the length of the interval and the probability of a second (or even more) events occuring is very small.

Proposition

Let {N(t);t≥0} be a Poisson process, then N(t+s)-N(s) ~ Pois(t) for all s≥0

Proof Let pn(t) = P(N(t)=n). Then

where the last equation follows from (P(N(h)=0) = 1-P(N(h)≥1) = 1-P(N(h)=1) - P(N(h)≥2)
Now

The same basic idea works for the case pn(t) as well to finish the proof

Remark It is intuitively clear why the definition above should lead to the Poisson distribution. Take the interval (0,t] and subdivide it into k equal size intervals (0,t/k], (t/k, 2t/k) .. ((k-1)t/k,t]. The probability of 2 or more events in any one interval goes to 0 as k goes to ∞ because
P(2 or more events in any subinterval)
≤∑P(2 or more events in the kth subinterval)
= ko(t/k)
t×o(t/k)/(t/k)→0 as k→∞
Hence N(t) will (with probability going to 1) just equal the number of subintervals in which an event occurs. However, by independent and stationary increments this number will have a binomial distribution with parameters k and p=λt+o(t/k). Hence by the Poisson approximation to the binomial we see that N(t) will have a Poisson distribution with rate λt.

Example Suppose that N points are uniformly distributed over the interval (0,N) Let X be the number of points in (0,1) find the pmf of X if N is large.

Let's try this directly first:

and this get's ugly fast. Instead consider the following: Let N(t) be the points in (0,t), then for t small (relative to N) {(N(t),t≥0} will be a Poisson process with rate λ. Now

P(N(1)=0) = P(X=0) = e-1, so λ=1 and so

P(X=n) = P(N(1)=n) = e-1/n!

Definition

Let T1 be the time when the first event occurs, T2 the time from the first event until the second event etc. The sequence T1, T2, .. is called the sequence of interarrival (sojourn) times.

Proposition

Let {N(t);t≥0} be a Poisson process, and {Ti;i≥1} be the interarrival times. Then T1, T2, .. are iid Exp(λ)

Proof Note that {T1>t} is equivalent to {no events occured in (0,t]} and so

and we see that T1 ~ Exp(λ). But

because of independent and stationary increments. So we find that T2 ~ Exp(λ) and that T1 T2. By induction it is clear that the sequence {Tn,n=1,2,..} is an iid sequence of exponential r.v. with mean 1/λ.

Remark This result should not come as a surprise because the assumption of independent and stationay increments means that the process from any moment on is independent of all that occured before and also has the same distribution as the process started at 0. In other words the process is memoryless, and we have previously shown that any continuous rv on (0,∞) with the memoryless property has to have an exponential distribution.

Definition

Let Sn be the arrival time of the nth event. (This is also often called the waiting time until the nth event).

Proposition

Let {N(t);t≥0} be a Poisson process, and {Sn;n≥1} be the waiting times. Then Sn~ Γ(n,λ)

Proof Clearly Sn = ∑ni=1Ti, and so we find Sn ~ Γ(n,λ).

Example: Up to yesterday a store has 999,856 customers. They are planning to hold a little party when the 1,000,000th customer comes into the store. From experience they know that a customer arrives about every 4 minutes, and the store is open from 9am to 6pm. What is the probability that they will have the party today?
They will have the party if at least 144 customers come into the store today. Let's assume that the customers arrive according to a Poisson process with rate 4min (?) then we want the probability P(S144 < 9× 60). Now S144 ~Γ(144,4) and
P(S144 < 540) = 0.23.

Here is another proof of the last proposition. We use the fact that the nth event occurs at or before time t if and only if the number of events occuring by time t is at least n. So
N(t)≥n iff Sn≤t

This is a very useful equivalence, and much more general than just for the Poisson process, so here is an illustration:

With this we find

Example Say that on any given day hundreds of cars pass through a certain intersection. Any one of them has a (hopefully small) probability of having an accident at that intersection. Let X(t) be the number of accidents in the t days, then is X(t) a Poisson process?

There are two problems with the assumptions of the Poisson process here:
1) different days might have different numbers of cars going through (weekdays vs. weekends?) and the probability of having an accident is probably very different for different cars. The first problem might be handeled by considering a different time-scale (accidents per week?), the second problem actually is not a problem at all:

let Z1, Z2, .. be independent Bernoulli rv's with P(Zi=1)=pi. Let Sn=Z1+..+Zn. Then if λ=p1+..+pn we have

In the "classic" case where p1=..=pn=p=λ/n we have

and we see that this theorem not only gives us reason to think that the Poisson approximation works in the example above, it also provides a useful estimate of the error in the Poisson approximation to the Binomial.

Consider events occuring along the positive axis [0,∞) as shown here

Examples of such processes are the time points in X-ray emissions of radioactive material, instances of phone calls from a location, accidents at intersections, location of faults in lengths of cabels, arrival times of customers in a store etc.

Let N((a,b]) the number of events in the (time) interval (a,b] That is if t1<t2<t3<.. denote the times (locations ) of events, the N((a,b]) is the number of values ti for which a<ti≤b.

Now let's make the following postulates:

1) The number of events happening in disjoint intervals are independent

2) For any t,h>0 the distribution of N((t,t+h]) does not depend on t

3) There is a positive constant λ for which P{N((t,t+h)≥1}=λh+o(h)

4) There is a positive constant λ for which P{N((t,t+h)≥2}=o(h)

Theorem Any process satisfying 1-4 ia a Poisson process
proof first note that 2) implies that N((s,t]) has the same distribution as N((0,t-s]), so it is enough to find the distribution of N((0,t]) for all t>0.

let's divide the interval (0,t] into n subintervals of equal length h=t/n, and let Zi=1 if there is at least one event in ((i-1)t/n,it/n], 0 otherwise. Then Sn=Z1+..+Zn counts the total number of intervals that contain at least one event. Moreover by 3) pi=P(Zi=1)=λt/n+o(t/n). Now

We have seen previously that if U1,..,Un~U[0,1] and indep., then (U[1],..,U[n]) has joint density f(u[1],..,u[n])=n!. Clearly if U1,..,Un~U[0,t] and indep., then (U[1],..,U[n]) has joint density f(u[1],..,u[n])=n!/tn. Let Wi=U[i]. Now

Theorem let W1,W2,.. be the occurance times in a Poisson process with rate λ. Then

In other words, conditional on the total number of arrivals the arrival times have the same distribution as the order statistic of a uniform.

Example say {N(t),t≥0} is a Poisson process with rate λ. Find the mean time of the first event, given that N(1)=n, n≥1.

Example Customers arrive at a store according to a Poisson process of rate λ. Each customer pays $1 on arrival, and we want to evaluate the expected value of the total sum collected during (0,t] discounted back to time 0. If the the discount (inflation) rate is β, then this is given by

Now

Consider a Poisson process {(N(t),t≥0} with rate λ, and suppose ech time time an event occurs it is classified as either type I or II, with probabilities p and q=1-p, respectively, independent of anything else. Let N1(t) and N2(t) be the respective number of type i and II arrivals by time t, then

Proposition {N1(t),t≥0} and {N1(t),t≥0} are both Poisson process with resp. rate pλ and qλ. Furthermore the processes are independent.
proof

The proposition easily generalizes to r types.

Example Customers arrive at a store according to a Poisson process with rate of 2 per hour. Each customer is a "Buyer" with probability 0.3 or a "Window-Shopper" with probability q=0.7. What is the probability of at least 1 sale during a 2 hour period?

P(at least 1 sales) = P(N1(t)≥1) = 1-P(N1(t)=0) = 1-exp(-2*2*0.3) = 1-e-1.2 =0.7

Example (Coupon Collection Problem) There are m different coupons. Each time a person collects a coupon it is, independently of those previously obtained, of type j with probability pj. Let N denote the number of coupons one needs in order to have a complete collection of at least one of each type. Find E[N]

Let Nj be the number of coupons needed until we have one of type j, then N=max1≤j≤m Nj . It is easy to see that Nj~G(pj), but they are not independent and so finding the distibution of their maximum is very difficult

Let's assume that that coupons are collected according to a Poisson process with rate 1, and say an event is of type j if the coupon collected was of type j. If we let Nj(t) denote the number of type j coupons collected by time t, then it follows that {Nj(t),t≥0} are independent Poisson processes with rates pj. Let Xj denote the time of the first event of type j, and let X=max1≤j≤mXj be the time when we have all the coupons. Now the Xj are the waiting times of independent Poisson processes, so they have an exponential distributions and are independent, so

Now let Ti be the ith interarrival time, that is the time between finding the (i-1)st and the ith coupon. the X=∑Ti, but Ti~Exp(1), and they are independent, so E[X|N]=E[∑Ti|N]=NE[T1|N]=N, so

E[X]=E{E[X|N]}=E[N]

For example, say p1=..=pn=p=1/m, then


m 2 3 4 5 6 7 8 9 10
E[N] 3 5.5 8.3 11.4 14.7 18.2 21.7 25.5 29.3

What if m-1 have the same probability, but one is rarer, say only half of the probability of the others? So (wlog) 2p1==p2=..=pm=1, then pi=1/(m-1/2) for i≤2≤m and p1=1/[2(m-1/2)]


this integral has to be found numerically, using some numerical integration method. Then we get
m 2 3 4 5 6 7 8 9 10
E[N] 3.5 6.4 9.6 13.0 16.6 20.4 24.2 28.2 32.3

Proposition: If {Ni(t);t≥0} i=1,..,k represent the number of type i events occuring in (0,t] and if Pi(t) is the probability that an event occuring at time t is of type i, then

Example (HIV-Aids) one ofe the difficulties in tracking the number of HIV infected people is its long incubation time, that is an infected person does not show any symptoms for a number of years, but is capable of infecting others.
Let us suppose that individuals contract HIV according to a Poisson procss with unknown rate λ. Suppose that the incubation time until symptoms appear is a rv with cdf G, which is known, and suppose incubation times are independent. Let N1(t) be the number of individuals that have shown synptoms at time t, and let N2(t) be the number that have contracted HIV at time t but not yet shown synptoms. An individual that contracts HIV at time s will show symptoms at time t with probability G(t-s), so it follows from the above proposition that

say we know the number of individuals with system as time t is n1, then

for example if t=16 years, μ=10 years and n1=220,000, then n2=219,00