Re: [S] Why is AIC labeled Cp in step()?

Prof Brian Ripley (ripley@stats.ox.ac.uk)
Fri, 27 Feb 1998 08:08:21 +0000 (GMT)


Bill Venables wrote:
>
> Jim Robison-Cox writes:
> >
> > ...I realize that in normal theory linear models AIC and Cp
> > are equivalent measures, but I don't understand why the
> > step(), add1() and drop1() functions give a column labeled Cp
> > which is really AIC.
> >
> > Quoting from the _Guide to Stat & Math Analysis_ p 7-13:
> >
> > " The Cp statistic (actually what is shown is the AIC
> > statistic, the likelihood version of the Cp statistic - the
> > two are related by the equation AIC=sigmahat^2(Cp+n)) provides
> > a convenient criterion for determining whether a model is
> > improved by dropping a term."
> >
> > I also see in V&R MASS2 p 220-2 that their stepAIC() does
> > actually print AIC at the top of the column.
> >
> > My concern is that people see the column labeled Cp and think
> > that a good model will have values close to p (# of
> > parameters). Is there some reason that this column has been
> > mislabeled? Could we get it changed?
>
> This has long been a puzzle for me, too, especially when you
> realise that the original S developers (not S-PLUS) and Colin
> Mallows work in the same building in Murray Hill. It's an
> enigma. You know what my suggestion will be: use Brian's much
> slicker, tidier and more general stepAIC.
>

After some correspondence with Jim Robison-Cox I now understand
what he meant (but did not say). When you run step.lm with trace=T,
you get intermediate printout from drop1.lm and/or add1.lm. That
is labelled Cp (but is not output from step() itself), whereas
the output from step.lm is labelled AIC.

Reason: they are not the same thing! step.lm is a call to step.glm,
and for a non-gaussian(identity) GLM the `Cp' is based on a linearization
at that step and not at all comparable in general with the `AIC'.

It does not help that `Cp' is not C_p as defined by Mallows, nor C_p
as defined by most other references I consulted (RSS/sigma^2 + 2p, with
no -n) nor is `AIC' AIC as defined by Akaike, nor that the quote from the
S-PLUS 3.x manual is highly misleading nor that Dr Robison-Cox has
not noticed that the output shown in V&R MASS2 p 220-2 is actually from
step().

The biggest diffulty is perhaps that `AIC and Cp are equivalent measures'
only if sigma^2 is assumed known, and it is not in the software.

By the way, using Mallows' C_p approximately p only makes sense to me
if sigma^2 is known: it is then equivalent to s^2 \approx sigma^2
which is probably easier to use.

stepAIC was originally written to overcome problems with stepwise
fitting of GLMs, but does do a consistent job for linear models
with unknown sigma^2. Do note that even lm and its ancilliaries
do not do a consistent job if sigma^2 is known.

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu.  To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message:  unsubscribe s-news