[S] Memory problem

Kerrie Paige (kpaige@gte.net)
Thu, 14 May 1998 08:04:47 -0700


Hi All,

I'm relatively new to S-Plus and have just written my first good-sized
S-Plus function. All has been going quite well, with one exception: I keep
getting the dreaded 'unable to allocate dynamic memory' error. I've read
everything I can find about writing efficient programs and have checked the
S-new archives, but haven't been able to find anything that helps. I did see
one reference in the archives to 'S-Plus' infamous memory management
problems', so perhaps I'll have no choice but to go to C. However, if
anyone knows of anything I could do to fix this in S-Plus, I'd much
appreciate it as all of the code is already written and debugged in S-Plus.

The offending segment of code is listed below. Basically, it loops over
each observation in a data set and calls a function that increments two
matrices based on what it finds in that observation. The two matrices are
fairly good sized - (6x21x21x18) and (6x21x18) but I only need a *single*
copy of each of them throughout the entire program. Within the function,
the two matrices are combined into a list (called record) that is then
passed back to be incremented on the next pass.

state.t <- array(0, dim = c(n.age, n.state, n.state, n.per))
pop.t <- array(0, dim = c(n.age, n.state, n.per))
record <- list(state = state.t, pop= pop.t)
for(i in 1:nrow(data.set)) {
record <- create.ind.trans.record(data.set[i, ], record, const.list)}
record

This code uses nearly 100 MB of dynamic memory for a 100 observation set
(with 12 variables). Given that I need to run it on data sets of 4000+
observations, this a big problem. As far as I can tell, S-Plus is making
copies of the matrices on every pass. I know that calling remove is an
option for getting rid of objects you are no longer using, but it wasn't
obvious to me how I'd remove it given that I do still want an object called
'record'. Am I going to have to recode this in C?

Oh, and in case it helps, I am using S-Plus 4.0 for Windows on a 200 MHz
Pentium with 48 MB of RAM and 700MB+ available for swap space.
Interestingly, it doesn't seem to be the RAM that causes the limitation, but
rather the swap space. The program runs until it starts to need more than
700 MB and then crashes.

Thanks very much in advance!

Kerrie Paige

-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news