Date: Mon, 18 Aug 2008 15:58:00 -0400
Reply-To: Paul Choate <Paul.Choate@DDS.CA.GOV>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Paul Choate <Paul.Choate@DDS.CA.GOV>
Subject: Re: File I/O error--large dataset problems?
Hi Alan -
Your code looks okay to me. You should set your obs=10000 or something
and try to debug from there. If you post your log people can help more.
debug your stuff - output to test files only!
On Mon, 18 Aug 2008 09:50:50 -0700, email@example.com
>I'm analyzing data with about 235 million rows & 31 variables. It's
>about 55 GB.
>I have it stored on a 64-bit server with 1.6 TB free but SAS itself is
>running a different, local machine. The client has the drive maped.
>I'm starting at the outset with a very simple filter, designed to
>reduce the size of it to something more manageable:
>*lgwork is a library referring to the server;
> set lgwork.om_orig;
> if exdate - date <= 60 and year >= 2005;
>If I take out the 'if' statement and just copy the data through a data
>step, SAS handles it fine. The temporary .lck file keeps going up and
>information is copied. When I use this filter though, the .lck file
>sits at 1 kb and a temporary, empty dataset called lgwork.omadv is
>This procedure was run overnight in a batch process & I got a File I/O
>error in the morning.
>Does anyone know what the problem might be? I'm running SAS 9.1.3 SP 4
>on windows XP.
>Any help or advice you can offer is greatly appreciated! Thanks!