> What do you consider as concepts? I am not willing to consider the
> representations of sample data as other than data-presentation. Nor
> do I see concepts there.
> The important concepts, as I see them, are those of probability and
> risk. Just about everything except data description is stated, more
> or less explicitly, in terms of probability. I do not consider
> learning how to compute probability as having much, if anything,
> to do with understanding probability.
> So what do YOU consider to be the concepts? I have stated what I
> consider them to be, and that the underlying problem is the balancing
> of risks. We need to understand the problem, not to produce solutions
> for other problems.
Fair enough; before we talk about empirical evidence concerning the
learning of concepts, we had better agree on what concepts we have
in mind; the proactive effects of "cookbook" learning may depend
on what concepts are taught later.
While I do agree with Rubin that balancing risk is *an* important
problem, and also that probability is central to the study of risk,
he and I disagree sharply about the centrality of the risk concept
in statistics. (I think we also disagree about how to teach a first
course on risk--I would favor a conventional case-based approach
with a purely informal treatment of probability--but that is perhaps
another discussion.)
I focus on statistics primarily as an aid to inductive inference,
rather than as a tool of decision making, and regard as central
the concepts of sampling distribution and (parametric) mathematical
model. A major goal of elementary teaching is to get students to
understand that many issues just can't be formulated very well without
something that amounts to a tractable mathematical model, and that
it is better to get that model explict than to hide it. What Rubin
dismisses as "data description" I view as an important part of teaching
about the mathematical description of data patterns. One of the reasons
that I like Chapters 1 and 2 of Moore & McCabe's book so well is that
these sections lend themselves to this task. Books that leave bivariate
description for a late chapter on linear regression are just not
acceptable to me. The mathematical models we use are of course
inherently probabilistic, and I wish we had time, within the
introductory course, to provide some really interesting examples
of probabilistic modelling (the section on genetics in the book
by Freedman, Pisani, Purves & Adhikari is a good shot at this);
but mostly I stick with two probabilistic ideas which students
can incorporate crudely into their thinking without much formalism:
the idea of a simple random sample and that of a measurement-error
distribution that is not too heavy-tailed. The notion of a model
parameter seems to me to be one of the most difficult to teach,
and the confusion of parameters with statistics--such as occurs
in the truly awful conclusion, "This experiment showed that there
is a statistically significant increase ..." seems very hard to
exorcise.
A second major goal is to get across the marvelously abstract concept
of sampling distribution, which I believe is the concept that distinguishes
statistics from other aspects of mathematical modelling. Obviously
the sampling distribution concept draws on probability, but what makes
it difficult is not probability, rather, it is the abstract idea of a set
of all samples and their probability distribution, conditional on a model.
I think it is because the ideas are so abstract that students may
sometimes benefit from previous exposure to cookbook statistics--abstract
ideas are grasped in relation to familiar inferential procedures,
and those procedures are illuminated in turn by the ideas. In teaching
a first course, I try to keep the actual procedures to be used--calculations
such as sample mean +- 1, 2, or 3 estimated standard errors--in front of
the student, while discussing the ideas of parametric model and sampling
distribution that underly the procedures. (Some students, naturally,
get lost in such a presentation.)
As for the distinction between inference and decision making,
which Professor Rubin apparently rejects, I can only say that my
own research experience and observations of research progress
suggest to me that research, both basic and applied, is driven
strongly by new ideas for collecting rich quantitative data
(hence, often, by new technology), and that interesting research
hypotheses are mostly invented to account for patterns discovered
by careful post-hoc examination of such quantitative data.
Naturally, such innovation is inevitably guided, and sometimes
blinded, by existing theory--there's no escape from that. Risk is
only a minor aspect of the research enterprise. To be sure, when one
tests new ideas for collecting data, or begins the careful exploration
of data already collected, one should be aware of the uncomfortably large
chance that nothing much is going to be discovered; and when one winds
down a project, with or without new insights, one should be aware of
the chance that something important has been overlooked. Still, in
my experience, dissatisfaction with existing methods and theories,
and the compelling push of a new idea are the decisive factors in
research decisions, with probabilities of success or failure
only introduced in very rough form. The metaphor of risk analysis
applies very poorly, because the diverse goals of research are not
reducible to any common currency or "utility," nor do I think it
is usually fruitful to attempt such a reduction.
I think that probabilists and statisticians may have been misled
by the mathematical results that are counterintuitive even for
the sophisticated analyst, and by many more that are counterintuitive
for the untrained analyst (e.g., the very weak dependence of sampling
variability on proportion of population sampled, when absolute
sample-size is help constant). From the fact that some consequences
of the probability axioms are very counterintuitive it does *not*
follow that the concept and basic properties of probability are
counterintuitive. In fact, basic probability concepts are very
easy to understand for most college students, until one introduces
mathematical notation to express them. One could give students
a good understanding of the concept of sampling distribution
without any formal discussion of probability. I am not sure
that one *should* do so--mathematical probability is both very
beautiful and very useful, and seems to me to be an important
part of a liberal arts education. But I do have trouble integrating
a formal introduction to probability with a coherent account of
statistical inference--the mathematical account seems to detract,
rather than add to an an understanding of other important concepts.
Despite my wide-ranging disagreements with Professor Rubin,
I find his stance very stimulating, because it makes me question
whether the standard introductory course, which seems oriented
toward statistics in the service of research, is really
appropriate for most of our audience. It could be argued that
a focus on risk analysis is more important than a research focus
for physicians, social workers, public health workers, engineers,
lawyers and actors (= future legislators), educators, generals
and admirals, business students, and a liberal arts audience.
How to teach such a course at the introductory level is another
question, of course. I would agree that "cookbook" statistics
might be at best irrelevant to such a course, but I don't think
I would teach measure and integration either! And I have to
wonder whether most statisticians have the detailed subject-matter
knowledge to teach a good case-based course in risk. Such
courses, even more than introductory statistics, might end up
being taught in profusion by specialists, a version for physicians,
a version for social workers, etc. It could be a real challenge
to develop a broad course of this kind.
In closing, I want to return to my original challenge to Professor Rubin:
to say whether any of his ideas on statistics education have an empirical
basis, either from uncontrolled and anecdotal classroom experience
(which is where most of my ideas come from) or from more formal
experimentation with sequencing of concepts in introductory teaching.
I don't think answers given by a priori analysis stand up very well
in the face of experience, in education any more than in other arenas.
Dave Krantz (dhk@columbia.edu)