MATH/STAT 491 - Stochastic Processes (Fall 2006)


  • [Extra extra office hours:] Tuesday, Dec.12th, 12:30-1:30pm in Padelford C301.

  • The exam will be held on Thursday, December 14th, 8:30-10:20am, in MEB 246. NOTE: this is a new room for us!
  • The final is worth 35% of your overall grade.
  • The exam is cumulative.
  • The exam will have a formula sheet attached (do not bring your own), as always. This is the new version, based on your suggestions.
  • You are allowed to bring a calculator.
  • The format of the exam will be similar to your tests. You will be asked to do 8 out of 10 [changed by popular demand] possible questions.
  • Extra office hours: Mon 11th and Wed 13th, 2-4pm, in Padelford C-301.


  • [Dec.9] Solutions to Assignment 4 have just been posted.
  • [Dec.4] The next (and last) practice problem set has been posted.
  • [Dec.4] The summary notes have just been updated. I will also give you the last practice problem set shortly, and I will make a new formula sheet for the final exam [suggestions welcome].
  • [Dec.1] Could you come up a with a stochastic model to describe this?
  • [Dec.1] "Typo" in today's lecture: P(B(1)-B(1/2)>-1)=P(Z/sqrt{2}>-1). My bad.
  • [Dec.1] Office hours: Next week, Wed and Thurs 3-4 in McCarty Hall Library; Dec.11th and Dec.13th 2-4 in Padelford C301.
  • Tisha tracked down the 1905 Einstein paper! Here is the link.
  • [Nov.28] The fourth (and last) assignment has now been posted.
  • Here are solutions to the second test.
  • Here are the solutions to the first test.

    Here are the summary notes. Last update: Dec.4th.

    Let me know if you see some typos.

    In previous probability courses (most likely 394/395), you learned about random variables as useful tools for modelling the random phenomena in the world. You also looked at joint behaviour of two or more random variables. In this class we will study a particular extension of this: the behaviour of random variables as they evolve in time. This type of analysis is of great use in many fields: from finance (stock prices) to biology (spread of an epidemic). Time-permitting, we will look at many accessible examples of stochastic processes: random walks, Markov chains, MCMC, (compound) Poisson process, birth-death chains, Brownian motion, branching processes, queueing systems...

    The official prerequisite of this course is at least a 2.0 in MATH/STAT 394 and 395. We will also make use of quite a bit of calculus and matrix algebra.

    Instructor: Hanna Jankowski

    E-mail: hanna[at]
    Please include "[491]" (using square brackets) in the subjects of your e-mails; this'll help me separate if from my other, mostly junk, mail. Also, please send only plain text messages, no html.

    Office Hours: Every Wednesday and Friday, beginning September 29th, 3-4 pm, in McCarty Hall Library.

    If you cannot make it to my office hours, please schedule an appointment. All appointments will take place in my office, Padelford B-220.

    Lectures will be held in BLM 414.

    The main reference for this course will be the lectures. As an additional reference text I have chosen Probability and Random Processes, by Geoffrey Grimmett and David Stirzaker, 3rd Edition. However, this is not a required text.

    There is a copy of this text available (soon) for short-term loan from the Mathematics Research Library in Padelford Hall C-306. I have also requested that the following texts be placed on reserve there.
    Introduction to Probability Models by Sheldon Ross
    An Introduction to Stochastic Modeling by Taylor and Karlin
    Stochastic Modeling of Scientific Data by Peter Guttorp

    Grading Scheme

    There will be 4-6 assignments, roughly bi-weekly, two term tests and a final in this course. The assignments will be posted below, and will be worth 25% of your final grade. Note: I will sometimes pick some problems from the practice exercises and put them on your assignment - don't let this confuse you...

    Assignment 1 Due: Friday, October 13th. With solutions.
    Assignment 2. Due: Wednesday, October 25th. With solutions. Don't mock the typesetting - I didn't have time to fix it.
    Assignment 3 . Due: Monday, November 13th. With solutions. [Typo corrected November 14th, 2pm]
    Assignment 4 . Due: Friday, December 8th. With solutions [small typo for 4a fixed Dec.11].

    Note: all assignments are due at the beginning of each lecture. There are no exceptions. I do not accept late assignments.

    Term test 1: Friday, October 27th, 20%

    Term test 2: Friday, November 17th, 20%

    Final: December 14th, 8:30-10:20am, 35%

    The tests will be held in Thomson 125.

    Note: the exam will not be held in the same room as the lectures. This will be announced beforehand.

    Bonus Work:
    You will have the opportunity to earn bonus marks (up to a maximum of 5%) by presenting a problem, chosen by me, to the class. Interested students should let me know asap, as I have to limit the number of presentations per lecture to one. In lieu of a presentation, you may also choose to do a computer programming assignment. Again, please let me know asap if you're interested.

    Course Policies etc.

    Material Covered and Practice Problems:

    In this section I will keep an ongoing list of the material we have covered, and the suggested practice questions attached to each lecture. It is imperative that you work through these on a timely basis.

    We have covered the following sections/topics:

    Week 0 :

  • Introduction: random walk!
  • Some review

    Week 1:

  • More Review
  • Def'n of SP, Random Walk
  • Beginning discussion of Markov Chains (ref: any text with section on MC)

    Week 2:

  • More MC, absorbing states (ref: 3.4 of Karlin and Taylor)
  • Finished up absorbing states, motivation for next section: classification of MC.
  • developed 3 classifications for individual states

    Week 3:

  • classification of MC: irreducibility
  • limit theorems for MC (using classification)
  • stationary distributions and their implications

    Week 4:

  • reversibility
  • review
  • Test!

    Week 5:

  • MCMC - motivation and Metropolis-Hastings algorithm
  • MCMC - more theory and examples: MCMC , Ising .
  • Poisson process

    Week 6:

  • Poisson and Birth Processes
  • Birth Processes con't.
  • Holiday

    Week 7:

  • Last comments on Birth processes (end of Test 2 coverage), some review
  • Continuous-time Markov Chains (HJ away, MM subs)
  • Test 2! (HJ away, MM subs)

    Week 8:

  • more CTMC
  • more CTMC
  • Holiday

    Week 9:

  • making discrete-time MC continuous, birth-death processes
  • Brownian motion
  • Brownian motion

    Week 10:

  • Compound Poisson process & Martingales
  • More on Martingales, overview of course
  • Review

    Practice (non-credit - do not hand these in) problems:
    Random Walk
    Markov Chains 1
    Markov Chains 2
    Markov Chains 3 , with some answers.
    MCMC and Poisson Process
    BM, CPP, and Martingales