 
Summary: ISyE 3232
HW 7 Solutions
1. The sum of the initial probabilities of being in each state must be equal to one so
a = (.5, .25, .25). Similarly, the sum of each row in the transition matrix must be equal to
one so
P =
.2 .4 .4
0 .4 .6
.3 .1 .6
.
1.a. From the transition matrix P we see that P{X1 = 0X0 = 1} = 0.
1.b The row vector aP = (.175, .325, .500) describes the distribution of X1.
1.c The row vector aP15
= (.2093, .2326, .5581) describes the distribution of X15.
1.d. Yes, the Markov chain is irreducible. Clearly we can move from state 0 to state 1
and then from state 1 to state 2 and finally from state 2 back to state 0 so that all states
communicate with one another.
