Re: [S] Problem reading large files.

Tae-Sung Shin (sts@statsoft.com)
Fri, 16 Oct 1998 10:05:27 -0500


I've faced the problem when I used small-memory workstation.
What I would do and did for a solution is just using another workstation
with large memory.

Tae-Sung Shin

At 09:16 PM 10/15/98 EDT, you wrote:
>
>Hi!
>I am running S-PLUS Version 3.4 Release 1 for Silicon Graphics Iris, IRIX
5.3.
>i have a large ascii file, over 100,000 lines long, one number on each line:
>0.766007
>0.916291
>0.924875
>0.924710
>0.914394
>0.042228 etc etc...
>When I try to read this in:
>my.vector _ scan("my.file") I get the error message:
>Error in scan....Cannot allocate 5242880 bytes: options("object.size")
>is 5000000: see options help file...
>
>So I did:
>options(object.size=10000000) and even much larger numbers,
>but s-plus keeps complaining; e.g.:
>
>Cannot allocate 167772160 bytes:
>options("object.size") is 100000000: see options help file
> and so on.
>
>If I use much higher than 10000000, it seems to jam our shared UNIX facility.
>
>100,000 is not that large, so it should be possible to read this in,
>convert it to a matrix, etc.
>
>Any suggestions?
>
>Thanks in advance,
>Andrey Feuerverger / Dept Statistics / Univ or Toronto
>andrey@utstat.toronto.edu
>
>
>-----------------------------------------------------------------------
>This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
>send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
>message: unsubscribe s-news
>
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news