Date: Thu, 6 Apr 2000 19:13:54 +1200
Reply-To: Don Stanley <don_stanley@XTRA.CO.NZ>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Don Stanley <don_stanley@XTRA.CO.NZ>
Organization: Xtra
Subject: Re: MVS -- U1319, No MKLES
Content-Type: text/plain; charset=us-ascii
Thanks Paul, and others who responded on this.
I had forgotten about MINSTG. Today we reproduced this problem in TEST, only
when using the 13 datasets, 1-12 are fine. I tried removing the MEMSIZE=0 and
using MINSTG instead.
Instead of a VM1319 the SASLOG suddenly became revealing. The message OUT OF
MEMORY appeared after the SET statement that was generated by a macro --setting
13 datasets and before the RUN, ie the log was
data <whatever> ;
set work.prod01 work.prod02 work.prod03 work.prod04 .. work.prod13 ;
OUT OF MEMORY
run ;
Interesting thing was, running under MEMSIZE=0 used very litlle memory.
So definately a memory problem, I can't see the code being very tweakable (maybe
a view instead of a datastep would help).
Just to test, we ran with memsize=32M and it failed, probably because the
physical region is 8M in our production systems. I did think the 32M memsize
would use disk like it was temporary memory, which is what SAS told me years
back memsize=0 does. So i'm leaving memsize=0 and our support guys should get
more sleep at the end of each month.
Cheers
Don
Paul Dorfman wrote:
> Don,
>
> It may be as simple as memory shortage, or a lot peskier problem having to
> do with a SAS bug.
>
> >From: Don Stanley <don_stanley@XTRA.CO.NZ>
> >
> >I've struck this error many times over the years.
>
> Don,
>
> Ditto. It may be as simple as memory shortage, or a lot peskier problem
> having to do with a SAS bug which SI does admit to.
>
> Sometimes I've found
> >my code needs tweaking, like when I had a massive format in a CNTLIN
> >dataset that wasn't sorted by the START variable. Sorting got round the
> >problem. SAS themselves document many usage notes and zaps relating to
> >this. As I understand it, its generally an indicator of a memory
> >shortage. But I've found no-one, not even at SAS, who can tell me what
> >MKLE's means.
>
> That sorting can actually decrease a format's memory usage is puzzling. Not
> so long ago, I was struggling with the same situation. First I took the lazy
> Susan approach and called SI. They sent me a zap for this bug. However, it
> was in December 1999, and the system folks had no desire to install
> _anything_ before after Y2K, so I ended up taking the industriuos Paul path,
> rewrote the entire thingie, and ran it in one-third the memory.
>
> >Can someone enlighten me?
>
> Not me. I asked the same question the SI MVS developer I was talking to
> regarding the aforementioned bug but never got to the point of deciphering
> the abbreviation. In my internal dictionary, it is mapped to the slot 'bad
> memory thing', somewhere next to S0C4.
>
> >Just as a point of interest, when I strike this problem, I almost always
> >use the MEMSIZE=0 invocation option as a first try at circumventing it.
> >I call this a 95% fix, the other 5% of the time I end up rewriting the
> >code. Today we put a new job into production. We have run it in test 20,
> >maybe 30 times. Bam. U1319 immediately. Only differences I can tell are
> >it runs under CONTROL-M (a scheduler) and it runs in a production
> >service class. The step that failed appears to be setting 13 datasets
> >(SET A B C ....), all with same structure, and all having a fairly small
> >number of variables. About 4000 records per dataset. I switched on
> >MEMSIZE=0 at invocation and its fine. I'd love to know what causes this
> >in the first place.
>
> In this particular case, you probably ran out of memory for buffers with 8M
> of real storage specified as default. MEMSIZE=0 of course fixed it by
> grabbing as much memory as the Big Iron Brother is willing to give you.
> However, MVS developers in SI insist adamantly that MEMSIZE=0 must never be
> used, and that REGION= and MEMSIZE= must be set to the same value you have
> to estimate beforehand.
>
> Another option I have found to be absolutely indispensable when memory is at
> stake is MINSTG ('minimize storage'; specified either at invocation or in
> the program). It causes a SAS step suspend all the real storage it has
> allocated upon the completion of the step. Otherwise, if the subsequent step
> requires more memory than the current one, memory usage keeps piling up.
> Watch this. OPTION MINSTG:
>
> 30 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.02 CPU seconds and 4561K.
>
> 31 data _null_; array a(100000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 3781K.
>
> 32 data _null_; array a(300000) _temporary_; run;
> NOTE: The DATA statement used 0.02 CPU seconds and 5341K.
>
> 33 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.02 CPU seconds and 4561K.
>
> As you see, each step allocates exactly as much memory as it needs. Now same
> thing with OPTION NOMINSTG:
>
> 42 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 4561K.
>
> 43 data _null_; array a(100000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 4561K.
>
> 44 data _null_; array a(300000) _temporary_; run;
> NOTE: The DATA statement used 0.02 CPU seconds and 7027K.
>
> 45 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 7027K.
>
> With 300000 elements in A, the step needs only 5341K, yet uses 7027 since
> the previous step's memory has not been released. MINSTG is actually
> documented in one of the v6 tech reports. They say that NOMINSTG provides
> some performance advantages by keeping the load modules in memory all the
> time, but I have not found any advantage to it, with the exception of having
> fun to rerun jobs bombed with the 'bad memory thing'.
>
> Kind regards,
> =====================
> Paul M. Dorfman
> Jacksonville, Fl
> =====================
>
> >Don
> >--
> >Don Stanley, B.SC, Dip O.R.S, MNZCS Director, Sysware Consulting
> >Group
> > Box 634, Wellington, NEW ZEALAND
> >
> > http://www.sysware.co.nz
> >EMAIL:: don_stanley@xtra.co.nz
> >http://www.geocities.com/don_stanley_nz/don_home.htm
> >Genealogy:: http://www.geocities.com/don_stanley_nz/family.htm
>
> ______________________________________________________
> Get Your Private, Free Email at http://www.hotmail.com
--
Don Stanley, B.SC, Dip O.R.S, MNZCS Director, Sysware Consulting Group
Box 634, Wellington, NEW ZEALAND
http://www.sysware.co.nz
EMAIL:: don_stanley@xtra.co.nz
http://www.geocities.com/don_stanley_nz/don_home.htm
Genealogy:: http://www.geocities.com/don_stanley_nz/family.htm
| 
