LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous (more recent) messageNext (less recent) messagePrevious (more recent) in topicNext (less recent) in topicPrevious (more recent) by same authorNext (less recent) by same authorPrevious page (August 2008, week 3)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Mon, 18 Aug 2008 15:58:00 -0400
Reply-To:     Paul Choate <Paul.Choate@DDS.CA.GOV>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Paul Choate <Paul.Choate@DDS.CA.GOV>
Subject:      Re: File I/O error--large dataset problems?

Hi Alan -

Your code looks okay to me. You should set your obs=10000 or something and try to debug from there. If you post your log people can help more.

options obs=10000;

debug your stuff - output to test files only!

options obs=max;

On Mon, 18 Aug 2008 09:50:50 -0700, alan.illing@gmail.com <alan.illing@GMAIL.COM> wrote:

>Hi, > >I'm analyzing data with about 235 million rows & 31 variables. It's >about 55 GB. > >I have it stored on a 64-bit server with 1.6 TB free but SAS itself is >running a different, local machine. The client has the drive maped. >I'm starting at the outset with a very simple filter, designed to >reduce the size of it to something more manageable: > > >*lgwork is a library referring to the server; > >data lgwork.omadv; > set lgwork.om_orig; > if exdate - date <= 60 and year >= 2005; >run; > > >If I take out the 'if' statement and just copy the data through a data >step, SAS handles it fine. The temporary .lck file keeps going up and >information is copied. When I use this filter though, the .lck file >sits at 1 kb and a temporary, empty dataset called lgwork.omadv is >created. > > >This procedure was run overnight in a batch process & I got a File I/O >error in the morning. > > >Does anyone know what the problem might be? I'm running SAS 9.1.3 SP 4 >on windows XP. > > >Any help or advice you can offer is greatly appreciated! Thanks!


Back to: Top of message | Previous page | Main SAS-L page