 
Summary: Notes for Exam 1 in Math 605. Fall, 2011.
This exam will be closed book and closed notes. However, you will be allowed to bring
in one sheet (front and back) of notes to aid you in the exam. You will have an hour and
a half to complete the exam. There may be some simple proofs on the exam to ensure you
are aware of the basic concepts. However, the proofs will not be long and should be easy if
you understand the material.
Topics to be covered: chapter 3. More explicitly, here are some topics you should know,
and some questions you should be able to answer.
1. The basic idea, including the definition, of the Markov property (this is equation (3.3)
in the notes).
2. Consequences of the Markov property (for example, (3.4) and (3.5)).
3. What is the relevance of a transition matrix?
4. You must be able to turn a word problem (i.e. a description) into a stochastic process.
5. How do you simulate a discrete time Markov chain? If I give you a stationary distri
bution, a transition matrix and 3 or four uniform random variables, can you give me
the first 3 or four states of the Markov chain?
6. How do I find P{Xn = 7  x0 = 3}, and other such (higher order transition) values?
7. What are the ChapmanKolmogorov equations? What is there probabilistic interpre
tation?
8. All the definitions and results of section 3.4 need to be known. Further, there relevance
