Date: Thu, 8 Jun 2006 08:53:23 -0700
Reply-To: "Pardee, Roy" <pardee.r@GHC.ORG>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: "Pardee, Roy" <pardee.r@GHC.ORG>
Subject: Re: Watch out for SAS keeping old data in memory while running
Content-Type: text/plain; charset="us-ascii"
On windows I'm using vbscript. I've got a fairly simple one that
batch-submits a single program, waits for sas to finish, and then trolls
the log looking for this regular expression:
"(error:|warning:|uninitialized)(?! (the .{4,15} product with which|your
system is scheduled))"
(The goal there is to flag any error or uninitialized var, and any
warning apart from the ones you get when your sas license is fixing to
expire.)
I'm using that to submit programs from ultraedit. The code is here, if
anybody's interested:
http://rafb.net/paste/results/yrbUc689.html
I could probably elaborate that for use w/>1 program--have it stop
processing subsequent programs upon the first error/warning.
If that'd be useful to anyone, shoot me a private e-mail & I'll gin
something up...
Cheers,
-Roy
-----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of Lou
Sent: Wednesday, June 07, 2006 8:01 PM
To: SAS-L@LISTSERV.UGA.EDU
Subject: Re: Watch out for SAS keeping old data in memory while running
""Pardee, Roy"" <pardee.r@GHC.ORG> wrote in message
news:8AD8F86B3312F24CB432CEDDA71889F201B5BCC3@ex06.GHCMASTER.GHC.ORG...
> Meh. If it was me I'd do up a script to act as the controller. That
> way you don't have to rely on the humans (too much) to clean up after
> themselves.
>
> But different strokes for different folks...
Interesting suggestion - on a windows platform, what's your scripting
language? The easiest thing I know to use is a .BAT file.
Right now, using a series of %include statements, if there's an ERROR in
some program in the chain, SAS goes into syntax check mode, nothing gets
processed, the output files are empty, and the run time can be
suspiciously short - on the order of minutes or less as opposed to hours
- depending on where the error occurred. With a batch file executing
each program independently, each program following the error may run and
produce output.
Total run time with an error may be comparable to total run time without
an error. Problem is that the output is likely to be incorrect. Or
sometimes the subsequent programs may have errors because something
they're dependent on isn't there or isn't correct. You end up with
several programmers trying to debug perfectly fine programs (that is,
code written per spec) because someone early on tried to divide by zero.
I suppose the solution would be to thoroughly test each program
individually to make sure no ERRORs are generated. That's supposed to
happen anyway.
But I've had programmers write stuff that will execute just fine - but
only if run from the enhanced editor. They think they've tested
thoroughly, but in reality they haven't. And even if they have,
production data that a program is seeing for the first time may have
quirks that weren't in the test data, stuff that is not supposed to be
there but somehow manages to sneak in.
A lot of programmers write all that cleanup code, only they place it at
the top of the program, to get rid of anything that may have been left
over from a program preveiously run during a session. If they're going
to write it anyway, put it at the bottom of the program - you may as
well clean up after yourself instead of after someone else.
Overall, the %include approach seems to work better in my experience.
The cost is that each program cleans up after itself.
>
> -----Original Message-----
> From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
> Lou
> Sent: Tuesday, June 06, 2006 7:14 PM
> To: SAS-L@LISTSERV.UGA.EDU
> Subject: Re: Watch out for SAS keeping old data in memory while
> running
>
> ""Pardee, Roy"" <pardee.r@GHC.ORG> wrote in message
>
news:8AD8F86B3312F24CB432CEDDA71889F201B5BC91@ex06.GHCMASTER.GHC.ORG...
> > Another way to go is to just always run in batch mode. You can do
> > >1 thing at a time then, and the log file documents exactly what
> happened.
>
> So, there are projects where a team of programmers can spend a few
> months writing a couple hundred programs, and when it comes time to
> deliver, they all have to be run, there are dependencies so things
> have to run in a specific order, and the whole thing might take
> several hours. We control that by writing a control program that
> basically consists of a series of %INCLUDE statements, and whether the
> thing is run in batch or interactively, it's all one SAS session.
> Heaven help you if something you've done screws up someone else's
piece of the job.
> You'll be the one there at midnight trying to fix things after
> everyone else has gone home at 5. It's not only a many of politeness,
> it's enlightened self interest - clean up after yourself.
