[S] large object disorientation

Roger Koenker (roger@ysidro.econ.uiuc.edu)
Thu, 2 Jul 1998 17:45:21 -0500 (CDT)


Against my better judgement, I'm being inextrictably drawn into problems
of computing fits for linear models which involve X matrices which are
too big to handle comfortably as single objects in Splus. Of course,
it is clear that one can do old-fashioned things like accumulate moment
matrices by looping over submatrices stored as separate datasets, but
this seems a bit too much like reinventing SAS.

John Chambers about 5 years ago suggested here that there was some ongoing
research on a more modern approach to this sort of thing. I'm wondering
whether there is progress on this front which will appear in V 5.0
for Unix, or whether others have any experience which they would be
willing to share on this topic.

John's comment is available from the S-news archive:

http://lib.stat.cmu.edu/s-news/Burst/3862

The basic idea of using the method/class approach to recognize
disaggregated large objects, operate on them a piece at a time
and then assemble the results seems very attractive. Obviously,
it is going to be easier to to implement for some functions than
others, but it seems that it would be better to have a limited
functionality for large objects of this sort, than nothing at
all. In fact at the moment, I would be happy to have something
which managed to do basic indexing operations and some linear
algebra.

url: http://www.econ.uiuc.edu Roger Koenker
email roger@ysidro.econ.uiuc.edu Department of Economics
vox: 217-333-4558 University of Illinois
fax: 217-244-6678 Champaign, IL 61820

-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news