# MATH 4430 3.0AF (Stochastic Processes) Assignments - Fall 2008

Access to assignment solutions has now been withdrawn. If you took this course, and need access, contact the instructor by e-mail.

### Practice problems

Given the long break caused by the strike, you may find it useful to work through some or all of the above practice problems.
• Section V, exercises 1.6, 1.7, 1.9
• Section V, problems 1.3, 1.9, 3.6
• Section VI, exercise 1.1 [do the cases n= 0, 1, 2 using (1.7) and (1.8), but also verify that the solution satisfies the Kolmogorov forward equations (1.2)]
• Section VI, exercise 2.1 [do the cases n= 1, 2, 3 using (2.2), but also verify that the solution satisfies the appropriate Kolmogorov forward equation. ]
• Section VI, exercise 6.2 [but find the infinitessimal matrix, not the transition matrix]
• Section VI, problems 1.3, 1.8, 2.1, 3.1, 6.4
Solutions

### Assignment 3 - modified

Originally due Wednesday November 5, 2008. Because of the Senate amnesty declared for students participating in the Nov 5 YFS fee protest, and then the strike that started Nov 6, this assignment will now be due the 2nd class after the strike ends. Solutions will not be posted till then.

The extra things I asked you to do in problems 1.1 and 1.2 came out garbled. I've now modified what I'm asking you to do in those problems, and only require you to do this for one of them, not both.

• Section IV, exercise 1.4
• Section IV, problems 1.1 and 1.2 (Hint - you may find that one of the transition matrices is doubly stochastic)
For one of these problems, also calculate the function h(i), the mean time it takes urn A to empty, starting with i balls in urn A. Use this to calculate the mean return time to 0 in two different ways, one using the stationary distribution and the other using the function h.
• Section IV, problem 2.4 (Just so the model is clear, if we start with a fresh component with lifetime k, and X(0)=0 this means that the component will last to the end of period 0 and then be replaced. In other words, k=1. Likewise X(0)=1 corresponds to k=2, etc.)
• Section IV, problem 2.8
• Section IV, problem 4.4: Also determine when the chain is null recurrent and when it is positive recurrent. Can it ever be transient? (Hint: show that pi_n is pi_0 times the sum of the a_k for k>n, and then use formula (5.2) of section I)
The following problems are not part of Assignment 3, but you should work on them after you're finished the assignment They were originally intended as preparation for the midterm.
• Section IV, problems 4.2 and 5.2; In both problems, describe the asymptotic behaviour of the n-step transition matrix.
Solutions

### Assignment 2

Due Tuesday October 14 (because the Monday is a holiday, and Friday is our exam)
Note that in the 'seeking zero' problem, if you can't find the general formula, then just work it out numerically for the case m=6. You will lose at most 1 mark.
• Section III, exercise 4.7
• Section III, problems 4.2, 4.4, 4.5, 4.7, 4.13
• Section IV, exercises 3.1, 3.2, 3.3, 3.4
Solutions

### Assignment 1

Due Monday September 29 (extended to October 3)
• Problems II.1.4, II.1.6, II.4.4, II.4.7, II.5.2 (read the definition of martingale)
• Exercises III.2.2, III.2.6
• Problems III.1.1, III.1.2, III.1.4, III.3.2
Solutions