Date:         Mon, 18 Aug 2008 14:55:58 -0700
Reply-To:     Reeza <fkhurshed@HOTMAIL.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Reeza <fkhurshed@HOTMAIL.COM>
Organization: http://groups.google.com
Subject:      Re: File I/O error--large dataset problems?
Comments: To: sas-l@uga.edu
Content-Type: text/plain; charset=ISO-8859-1

On Aug 18, 10:50 am, "alan.ill...@gmail.com" <alan.ill...@gmail.com>
wrote:
> Hi,
>
> I'm analyzing data with about 235 million rows & 31 variables. It's
> about 55 GB.
>
> I have it stored on a 64-bit server with 1.6 TB free but SAS itself is
> running a different, local machine. The client has the drive maped.
> I'm starting at the outset with a very simple filter, designed to
> reduce the size of it to something more manageable:
>
> *lgwork is a library referring to the server;
>
> data lgwork.omadv;
>         set lgwork.om_orig;
>         if exdate - date <= 60 and year >= 2005;
> run;
>
> If I take out the 'if' statement and just copy the data through a data
> step, SAS handles it fine. The temporary .lck file keeps going up and
> information is copied. When I use this filter though, the .lck file
> sits at 1 kb and a temporary, empty dataset called lgwork.omadv is
> created.
>
> This procedure was run overnight in a batch process & I got a File I/O
> error in the morning.
>
> Does anyone know what the problem might be? I'm running SAS 9.1.3 SP 4
> on windows XP.
>
> Any help or advice you can offer is greatly appreciated! Thanks!

You may want to try a where condition as well, instead of if, I think
its faster. Not sure with the calculation included though.