Dear SAS-L-ers,

krzysoporanku kicked off an interesting thread with his original message, which can be found in its entirety beneath the Sig line. His first paragraph was:

> I've a serious problem with large SAS 4GL source. We are trying >to submit one data step with about 1,000,000 source lines. All is >correct on Windows NT (2K) platform, but on HP UNIX 11 (SAS 8.2 32bit) >source generates error: > > > ERROR: Out of memory. >> FATAL: Error occurred in the code generation subsystem during program >load.

krzysoporanku, this is a fascinating problem, indeed! And you have already gotten great advice from some SAS-L heavyweights. (Heavyweights in the sense of SAS knowledge only--not the other kind)! But, I am thinking: if ever a DATA step cried out for pre-compilation, it would be this one! Why not separate out the SAS System's compilation of the DATA step from its execution of the DATA step? Perhaps that would resolve your memory issue. I mean doing the following:

1. Pre-compile your HUMONGOUS DATA step using the SAS Systems Stored Program Facility 2. Execute the now-compiled DATA step

Here is an example of the compilation:

filename storlib '/krzyso/poranku/complib';

data sas4gl.bigdata / pgm=storlib.hugeprog;

....1,000,000 (one-million!) freaking lines of SAS code....

run;

Here is an example of how you would run the pre-compiled SAS program:

data pgm=storlib.hugeprog; run;

This is soooo simple to do, that you could try it in a flash. And, divorcing the compile from the execution, just might free up enough memory to get the job done!

krzysoporanku, best of luck in your effort to get that amazingly huge SAS program to run on the Unix operating system!

I hope that this suggestion proves helpful now, and in the future!

Of course, all of these opinions and insights are my own, and do not reflect those of my organization or my associates. All SAS code and/or methodologies specified in this posting are for illustrative purposes only and no warranty is stated or implied as to their accuracy or applicability. People deciding to use information in this posting do so at their own risk.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Michael A. Raithel "The man who wrote the book on performance" E-mail: MichaelRaithel@westat.com Author: Tuning SAS Applications in the MVS Environment +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ It's not the size of the dog in the fight, but the size of the fight in the dog. - Archie Griffin +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

<<krzysoporanku's entire posting:>>

Hi All! I've a serious problem with large SAS 4GL source. We are trying to submit one data step with about 1,000,000 source lines. All is correct on Windows NT (2K) platform, but on HP UNIX 11 (SAS 8.2 32bit) source generates error:

> ERROR: Out of memory. > FATAL: Error occurred in the code generation subsystem during program load.

I can allocate about 1GB memory by defining an array:

> %let size=50; > data aaa; > array aaa {1024, 1024, &size} $9 _temporary_; > x=aaa(1,1,1); > output; > run; > > NOTE: The data set WORK.AAA has 1 observations and 1 variables. > NOTE: DATA statement used: > real time 8:45.45 > user cpu time 2.27 seconds > system cpu time 23.26 seconds > Memory 870479k

Unfortunately when I try to execute a large 4GL code I can allocate only about 100MB memory. SAS options (memsize, loadmemsize, maxmemquery, realmemsize) and Unix memory parameters has been set up to max values.

Any suggesions? Could someone test attached program on HP UNIX with SAS 8 (64Bit)?

Thank you very much for any help.

Krzysztof Pruszczak

/*********** begin memory test *******************/ /* this source shoud allocate about 200MB memory */

options fullstimer; %let count=2000000; filename test "%sysfunc(pathname(work))/_if_test.txt" ;

data _null_; file test lrecl=50; put 'data _null_ /pgm=work.src;'; do i=1 to &count; put ' if 1=0 then x=0;'; end; put 'run;'; run;

%inc test /nosource2;

filename test;

data pgm=work.src; run; /*********** end ******************/