[S] Bert Gunter's comments

Frank E Harrell Jr (fharrell@virginia.edu)
Wed, 11 Mar 1998 14:39:09 -0500

Bert made some excellent observations as usual. I'll just add a couple
of points that he stimulated me to think of-

1. I don't emphasize bias as much as the probability of getting close
to the "truth". There are many applications where I don't even
care about bias at all, where the variance dominates the mean
squared error.

2. External validation in the sense Bert described is certainly the
gold standard. Many researchers are unfortunately doing
split-sample validation, and it's easy to show that an "internal"
bootstrap validation using all the data provides a more precise
estimate of future model performance than does the (smaller)
hold-out sample. I have to be careful to say that "future" refers
to a similar stream of subjects using the same data collection
instruments, assessment of responses, etc.

Frank E Harrell Jr
Professor of Biostatistics and Statistics
Director, Division of Biostatistics and Epidemiology
Dept of Health Evaluation Sciences
University of Virginia School of Medicine

This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news