Date: Wed, 21 Jul 2010 11:19:05 +0000
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: "firstname.lastname@example.org" <goladin@GMAIL.COM>
Subject: Re: In search of a more efficient program
Content-Type: text/plain; charset="utf-8"
Reducing the length of numeric can cause trouble in terms of precision.
Sent from my Nokia phone
From: Andy Arnold
Sent: 21/07/2010 7:05:30 PM
Subject: In search of a more efficient program
I've inherited a large, complex SAS program. Most files are quite small;
however, some are extremely large and have become a problem.
The files in question have 12-24 fields that are mostly numeric with a few
short (1-4character) fields. The files that are killing me have 20M records
and one has 160M records.
The files don't use SAS compression because the records are too short to
make compression cost & time effective.
What are the trade-offs if I force numeric length to 4 or 2? Does SAS
always use a NUM 8 format internally? If I force short numeric field
lengths, will SAS have to convert them up to length=8 and back down again in
order to use the data?
What are the trade-offs if I use an index instead of a sort? The problem
sort takes an hour, uses a single numeric key with about 10 distinct values,
sorts 20M records that are about 100 bytes long. I've been successful with
indexing before; in that situation, I needed to sort the file in 15
Thanks for your input and advice.