A model for flipping a coin forever

In case you think the examples of models (eg, flipping a coin once, flipping a coin three times, etc.) I gave in class were trivial, here's a more complicated variation on them, the purpose of which is to model infinitely many flips of a fair coin.

Problem

A fair coin is flipped repeatedly, and a random variable N is defined as the number of flips needed to get the first head. Build a mathematical model for N.

The complication here is that we don't know in advance how many flips it will take, so we can't use ordered n-tuples of H's and T's for our sample points anymore, as no fixed value of n will work.

First solution: modelling via the distribution

If ALL we care about is the random variable N, then the "minimal" model we can build is to let our sample points range over the possible values of N. In this case, the possible values are 1, 2, 3, 4, etc. In other words, we could take Omega to be the natural numbers, and just define N(n)=n. Then building the model amounts to figuring out what numbers p_n we should choose for the probability of the event {n}. That is, we must figure out what the probability that N=n should be. Note that this approach will work for ANY discrete random variable - if we can figure out what its distribution should be, then we can always build a model for the random variable, by taking the sample points to consist of the possible values for the random variable, and then defining P via the target distribution.

Later, we'll have a name for the distribution of N - the geometric distribution. But for now we can proceed directly.

• P(N=1): The only way N can =1 is if the first flip is H. Prob = 1/2
• P(N=2): For N=2 we need the first flip to be T and the 2nd to be H. Prob = 1/2 x 1/2 = 1/4
• P(N=3): Now we need the first two flips to be T and the 3rd to be H. Prob = 1/2 x 1/2 x 1/2 = 1/8
• .........
• P(N=n): In general we need n-1 T's, followed by a H. Prob = (1/2)^(n-1) x 1/2 = 1/2^n.

Second solution: modelling the flips too:

The above is not totally satisfactory - for calculating things like the mean and variance of N, the model will work. But it leaves out lots of other things. We can't use it to figure out things like the conditional probability of N=5 given that the first flip is a T, as our first model has no way of expressing the latter possibility. All it can do is let us write down events involving N but not any of the individual flips. To model that as well we'd need to build a more suitable model, as it is awkward to change models every time one changes the question asked.

The following model can be used for ANY question about flipping a fair coin infinitely many times. Take Omega to be [0,1), and define P(B) to be the length of B, for any subset B of Omega.

That gives us our model, but how do we use it to talk about flips? For any sample point x (for convenience, I'm going to write x for a general sample point, rather than omega), write x in binary form. That is, let x_n be the nth digit in the standard binary representation of x. For example, in binary, .101 represents the number
1x(1/2) + 0x(1/4) +1x(1/8) = 5/8 Now interpret the event that the n'th flip is a head to be the set of x's such that x_n=1. I claim that the probability of this event is 1/2, for every n, and that these events are independent, which justifies such an interpretation.

I'll only check this for n=1 and n=2. Note that the binary expansions start

• .00 for x in [0,1/4)
• .01 for x in [1/4,1/2)
• .10 for x in [1/2,3/4)
• .11 for x in [3/4,1)
So the event that the first flip is a H is the union of the last two of these intervals, that is, [1/2,1). Obviously, it does have length (ie probability) 1/2. Similarly, the event that the second flip is a H is the union of the 2nd and 4th of the above intervals. Again the length of this set is 1/2. Finally, the even that BOTH the 1st and 2nd flips are H's is the fourth interval (corresponding to binary expansions starting .11). This has length 1/4 = 1/2x1/2, verifying independence.

Now that we can talk about flips mathematically, what about N? The event that N=n corresponds to the set of x's whose binary expansions start .000000000...001 where there are n-1 0's in the list. That is, we define N(x) to be

• 1 for x in [1/2,1)
• 2 for x in [1/4,1/2)
• 3 for x in [1/8,1/4)
• .....
• n for x in [(1/2)^n,(1/2)^(n-1))
This defines a legitimate random variable, which has the desired interpretation. If you want, you can check that the distribution of N, defined as in the second solution, agrees with that we worked out in the first solution.

What's the point of all this?

Mainly that the ideas going into the building of proper mathematical models can be more subtle than they appear at first, even for such simple sounding problems as flipping fair coins. But don't worry - building and using models like the second one above is not the kind of thing I expect you to do in the course. And, in this course anyway, we'll be able to steer clear of models like this when acutally carrying out our calculations.