Date:         Tue, 21 Oct 2003 16:24:04 -0700
Reply-To:     "Choate, Paul@DDS" <pchoate@DDS.CA.GOV>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "Choate, Paul@DDS" <pchoate@DDS.CA.GOV>
Subject:      Re: estimated sorting time
Comments: To: Zhonghe Li <zli@HSPH.HARVARD.EDU>

Elena-

Six hours sounds like a long time, but sort time depends heavily on things
like your I/O and CPU speed and the size of your records.  I usually get
good sort time estimates by running a few test sorts on increasingly large
subsets of the dataset (1k, 10k, 100k, 1,000k) and then projecting the time
based on empirical observation.

The free space on your drive is adequate; you normally need about 3 times
the original size.  A larger drive doesn't matter.

I/O is often the problem.  I've experience large reductions based on record
width, do you have large unnecessary character fields in your data?  Also,
depending on characteristics of your sort key you may have better results
using an index.

Here's some white papers from SAS
http://search.sas.com/suppquery.html?qt=sort+efficiency

hth

Paul Choate
DDS Data Extraction
(916) 654-2160

-----Original Message-----
From: Zhonghe Li [mailto:zli@HSPH.HARVARD.EDU]
Sent: Tuesday, October 21, 2003 3:33 PM
To: SAS-L@LISTSERV.UGA.EDU
Subject: estimated sorting time

Hi,

I am sort a 3.7 GB dataset on a computer with 16 free GB hard drive, and 1
GB of RAM.  So i run the tagsort.  It has been 6 hours already.

Can any one tell me how long it may take?

I have the choice of recreate a smaller dataset and run the sort - not
tagsort.  Then the question is how long it takes to run sort of a dataset
of 3 GB on a 20 GB free disk space?

Thank you for your help.

Elena