On Tue, 12 Aug 2008 07:53:31 -0700, Xuhong <zhuxuhong2000@GMAIL.COM> wrote:
>I got I/O errors as below when I was using data step to find some
>qualified observations from a big encounter dataset.
>ERROR: An I/O error has occurred on file MED.FACI0803.DATA.
>NOTE: The data step has been abnormally terminated.
>NOTE: The SAS System stopped processing this step because of errors.
>NOTE: There were 41821734 observations read from the data set
> WHERE (firstdt>='01JAN2003'D and firstdt<='31DEC2007'D);
>WARNING: The data set WORK.HIV_TEMP11 may be incomplete. When this
>step was stopped there were
> 14639 observations and 2 variables.
>NOTE: DATA statement used (Total process time):
> real time 1:15:41.98
> cpu time 6:57.15
>Some details on server side:
> Work/Temp space --> G Drive / Capacity: 1TB / Free: 994GB
> dataset: --> O: Drive / Capacity: 1TB / Free: 390GB
>Are there any options to try to avoid the I/O error from my side (SAS
>programmer)? Also is there any way I can look at what is going on
>behind the scenes to see why the I/O error occurred?
>Any help and suggestions will be appreciated! Thanks very much!
I don't know how much help this will be towards the I/O problem, but those
times are striking. You may want to consider the efficiency of your code.
First, without knowing your purpose or data, can you subset the data or the
variables? For instance, if you are interested in only a certain number of
variables, KEEP only them:
data _null_ ;
set one ( KEEP = ID firstdt var1 var2 ) ;
Second, can you reduce the data:
data _null_ ;
set one ( where = ( region = 1 )) ;
If you write back to the list with a description of your data, your goals,
and some sample code (or the log), we might be able to offer more tailored