Date:         Mon, 18 Aug 2008 15:58:00 -0400
Reply-To:     Paul Choate <Paul.Choate@DDS.CA.GOV>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Paul Choate <Paul.Choate@DDS.CA.GOV>
Subject:      Re: File I/O error--large dataset problems?

Hi Alan -

Your code looks okay to me.  You should set your obs=10000 or something
and try to debug from there. If you post your log people can help more.

options obs=10000;

debug your stuff - output to test files only!

options obs=max;

On Mon, 18 Aug 2008 09:50:50 -0700, alan.illing@gmail.com
<alan.illing@GMAIL.COM> wrote:

>Hi,
>
>I'm analyzing data with about 235 million rows & 31 variables. It's
>about 55 GB.
>
>I have it stored on a 64-bit server with 1.6 TB free but SAS itself is
>running a different, local machine. The client has the drive maped.
>I'm starting at the outset with a very simple filter, designed to
>reduce the size of it to something more manageable:
>
>
>*lgwork is a library referring to the server;
>
>data lgwork.omadv;
>        set lgwork.om_orig;
>        if exdate - date <= 60 and year >= 2005;
>run;
>
>
>If I take out the 'if' statement and just copy the data through a data
>step, SAS handles it fine. The temporary .lck file keeps going up and
>information is copied. When I use this filter though, the .lck file
>sits at 1 kb and a temporary, empty dataset called lgwork.omadv is
>created.
>
>
>This procedure was run overnight in a batch process & I got a File I/O
>error in the morning.
>
>
>Does anyone know what the problem might be? I'm running SAS 9.1.3 SP 4
>on windows XP.
>
>
>Any help or advice you can offer is greatly appreciated! Thanks!