MATH/STAT 491 - Stochastic Processes (Fall 2006)

FINAL EXAM INFORMATION:

[Extra extra office hours:] Tuesday, Dec.12th, 12:30-1:30pm in Padelford C301.

The exam will be held on Thursday, December 14th, 8:30-10:20am, in MEB 246. NOTE: this is a new room for us!

The final is worth 35% of your overall grade.

The exam is cumulative.

The exam will have a formula sheet attached (do not bring your own), as always. This is the new version, based on your suggestions.

You are allowed to bring a calculator.

The format of the exam will be similar to your tests. You will be asked to do 8 out of 10 [changed by popular demand] possible questions.

Extra office hours: Mon 11th and Wed 13th, 2-4pm, in Padelford C-301.

Announcements:

[Dec.9] Solutions to Assignment 4 have just been posted.

[Dec.4] The next (and last) practice problem set has been posted.

[Dec.4] The summary notes have just been updated. I will also give you the last practice problem set shortly, and I will make a new formula sheet for the final exam [suggestions welcome].

[Dec.1] Could you come up a with a stochastic model to describe this?

[Dec.1] "Typo" in today's lecture: P(B(1)-B(1/2)>-1)=P(Z/sqrt{2}>-1). My bad.

[Dec.1] Office hours: Next week, Wed and Thurs 3-4 in McCarty Hall Library; Dec.11th and Dec.13th 2-4 in Padelford C301.

Tisha tracked down the 1905 Einstein paper! Here is the link.

[Nov.28] The fourth (and last) assignment has now been posted.

Here are solutions to the second test.

Here are the solutions to the first test.

Here are the summary notes. Last update: Dec.4th.

Let me know if you see some typos.

In previous probability courses (most likely 394/395), you learned about random variables as useful tools for modelling the random phenomena in the world. You also looked at joint behaviour of two or more random variables. In this class we will study a particular extension of this: the behaviour of random variables as they evolve in time. This type of analysis is of great use in many fields: from finance (stock prices) to biology (spread of an epidemic). Time-permitting, we will look at many accessible examples of stochastic processes: random walks, Markov chains, MCMC, (compound) Poisson process, birth-death chains, Brownian motion, branching processes, queueing systems...

The official prerequisite of this course is at least a 2.0 in MATH/STAT 394 and 395. We will also make use of quite a bit of calculus and matrix algebra.

Instructor: Hanna Jankowski

E-mail: hanna[at]stat.washington.edu
Please include "[491]" (using square brackets) in the subjects of your e-mails; this'll help me separate if from my other, mostly junk, mail. Also, please send only plain text messages, no html.

Office Hours: Every Wednesday and Friday, beginning September 29th, 3-4 pm, in McCarty Hall Library.

If you cannot make it to my office hours, please schedule an appointment. All appointments will take place in my office, Padelford B-220.

Lectures will be held in BLM 414.

The main reference for this course will be the lectures. As an additional reference text I have chosen Probability and Random Processes, by Geoffrey Grimmett and David Stirzaker, 3rd Edition. However, this is not a required text.

There is a copy of this text available (soon) for short-term loan from the Mathematics Research Library in Padelford Hall C-306. I have also requested that the following texts be placed on reserve there.
Introduction to Probability Models by Sheldon Ross
An Introduction to Stochastic Modeling by Taylor and Karlin
Stochastic Modeling of Scientific Data by Peter Guttorp

Grading Scheme

There will be 4-6 assignments, roughly bi-weekly, two term tests and a final in this course. The assignments will be posted below, and will be worth 25% of your final grade. Note: I will sometimes pick some problems from the practice exercises and put them on your assignment - don't let this confuse you...

Assignment 1 Due: Friday, October 13th. With solutions.
Assignment 2. Due: Wednesday, October 25th. With solutions. Don't mock the typesetting - I didn't have time to fix it.
Assignment 3 . Due: Monday, November 13th. With solutions. [Typo corrected November 14th, 2pm]
Assignment 4 . Due: Friday, December 8th. With solutions [small typo for 4a fixed Dec.11].

Note: all assignments are due at the beginning of each lecture. There are no exceptions. I do not accept late assignments.

Term test 1: Friday, October 27th, 20%

Term test 2: Friday, November 17th, 20%

Final: December 14th, 8:30-10:20am, 35%

The tests will be held in Thomson 125.

Note: the exam will not be held in the same room as the lectures. This will be announced beforehand.

Bonus Work:
You will have the opportunity to earn bonus marks (up to a maximum of 5%) by presenting a problem, chosen by me, to the class. Interested students should let me know asap, as I have to limit the number of presentations per lecture to one. In lieu of a presentation, you may also choose to do a computer programming assignment. Again, please let me know asap if you're interested.

Course Policies etc.

The tests are all closed book tests. You will probably need a calculator - only non-programmable calculators are allowed.

There will be no make-up tests scheduled. In case of illness supported by medical documentation, your grade will be based on your other grades, pro-rated accordingly.

Course work will be handed back during lectures. If you disagree with the mark you have received, you should submit a request for regrading in writing to the instructor within three days of when the work was returned.

You are free to discuss your homework assignments with others, in fact, I would encourage you to do so. However, you must write up your own solutions. I cannot stress this last part enough: not only is the right thing to do, but it's also how you learn the material. The assignments are due at the beginning of the class on the due dates. I do not accept late assignments.

Attendance at all lectures is very important.

The best way to learn mathematics is to do mathematics. Make sure you keep up with the non-credit practice problems. Come see me if you have troubles. Understanding the lectures and working through the assignments and practice problems is the key to your success in this course.

If you have any concerns/suggestions etc., please let me know. This is what I'm here for.

Material Covered and Practice Problems:

In this section I will keep an ongoing list of the material we have covered, and the suggested practice questions attached to each lecture. It is imperative that you work through these on a timely basis.

We have covered the following sections/topics:

Week 0 :

Introduction: random walk!

Some review

Week 1:

More Review

Def'n of SP, Random Walk

Beginning discussion of Markov Chains (ref: any text with section on MC)

Week 2:

More MC, absorbing states (ref: 3.4 of Karlin and Taylor)

Finished up absorbing states, motivation for next section: classification of MC.

developed 3 classifications for individual states

Week 3:

classification of MC: irreducibility

limit theorems for MC (using classification)

stationary distributions and their implications

Week 4:

reversibility

review

Test!

Week 5:

MCMC - motivation and Metropolis-Hastings algorithm

MCMC - more theory and examples: MCMC , Ising .

Poisson process

Week 6:

Poisson and Birth Processes

Birth Processes con't.

Holiday

Week 7:

Last comments on Birth processes (end of Test 2 coverage), some review

Continuous-time Markov Chains (HJ away, MM subs)

Test 2! (HJ away, MM subs)

Week 8:

more CTMC

Holiday

Week 9:

making discrete-time MC continuous, birth-death processes

Brownian motion

Week 10:

Compound Poisson process & Martingales