Problem 1 Let {X_n,n≥1} be a Markov chain with transition matrix P. Show that l=1.0 is an eigenvalue of P. Hint: is any matrix P the transition matrix of some Markov chain? What is the definition of "eigenvalue"?

Problem 2 Say {X_n,n≥1} is a Markov chain with states {1,2,3} and transition matrix

a) Find the eigenvalues and eigenvectors of this matrix and use them to calculate P(X₅=1|X₁=1)

b) Find the stationary distribution of this chain.

c) Write an R routine to simulate this Markov chain and use it to estimate the same probability P(X₅=1|X₁=1)