Math 554 - Fall 2008

Instructor: Alexander Roitershtein
Office: 420 Carver Hall
Phone: 294 - 8135
Email: roiterst@iastate.edu
Office hours: MF 2:00pm--3:30pm, W 12:00pm--1:00pm, or by appointment.

Class schedule: MWF 1:10pm-2:00pm, Carver 0008
Textbook: J. R. Norris, Markov Chains, Cambridge University Press, 1997

Course descroption:
The topic of the course are Markov chains on discrete spaces in discrete and continuous time. In particular, we will discuss random walks on finite and infinite graphs, Poisson processes, birth and death processes, and their long-term behavior. In addition, we will introduce discrete-time martingales and study elements of the potential theory of Markov chains. We will also discuss applications of Markov chains to biology (branching processes and epedemics models) and queuing theory.
The course is essentially self-contained, only basic knowledge of elementary probability (random variables, conditional probability) is required.
We will cover the material corresponding to the content of Chapters 1, 2, 3, and Sections 4.1, 4.2, 5.1, 5.2 in the textbook. In most of the classes, but not always, I will follow the textbook pretty closely.

Syllabus:
1. Discrete-time Markov chains: classification and long-range behavior (Chapter 1 of the textbook)
2. Continuous-time Markov chains: construction and basic properties (Chapter 2 of the textbook)
3. Continuous-time Markov chains: classification and long-range behavior (Chapter 3 of the textbook)
4. Additional topics:
a) martingales (Section 4.1 of the textbook)
b) potential theory (Section 4.2 of the textbook)
5. Applications:
a) branching processes (Section 5.1 of the textbook)
b) models of epidemics and other applications to biology (Section 5.1 of the textbook)
c) queues and queueing networks (Section 5.2 of the textbook)

Grading policy:
Homework: 70% Final home exam: 30%.

Bonus: at the end of the grading process, I may move a few grades up by up to 5 points. These will be awarded based on some X factors: for example, homework, class room participation, or drastic improvement over the course of the semester.

Translation from the percentage to the grade will be done according to the following key:
A 91-100, A- 86-90, B+ 81-85, B 76-80, B- 71-75, C+ 66-70, C 61-65, C- 56-60, D 50-55, F 00-49.

Homework:
There are will be 4 homework assignments besides the final exam, each one usually lncluding 7-10 questions from the textbook. The homeworks will be posted here during the term.