Date:         Thu, 12 Jul 2007 10:26:16 -0400
Reply-To:     "V. Bourcier" <veronique.bourcier@LAPOSTE.NET>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "V. Bourcier" <veronique.bourcier@LAPOSTE.NET>
Subject:      Re: Trying to read in a large .csv file
Content-Type: text/plain; charset=ISO-8859-1

1. Using the SAS Editor instead of Notepad to cope with large data sets:

Once I had an access file with several millions of rows. I saved it under
a .csv extension and opened it with the SAS Editor instead of Notepad which
could not cope with the data set size. Finally I could save the editor
document as a .txt file and do a data input.

2. Import the whole data set and then select the subset.

The whole data set needs to be read even if only a portion of the data set
is needed. Therefore, I would be tempted to say that only a selection on
the created SAS data set it possible. But I might be wrong.

data one (where=(x gt 2));
  input x;
  datalines;
2
3
;
run;

proc print data=one;
run;

Veronique




On Mon, 9 Jul 2007 10:24:44 -0400, ben.powell@CLA.CO.UK wrote:

>In windows use wordpad rather than notepad as its memory management seems
>to be better so you can view large text based file formats such as csv.
>
>It sounds like either a) the line formatting is irregular or b) sas hasn't
>been told correctly where to look for the end of line, such as because
>there is irregular number of variables per line for instance.
>
>Or else use a dumb import so you can view in SAS such as:
>
>data a;
>infile "blah.csv" dlm="¬" /*DUMMY DELIMITER.*/
>  dsd missover lrecl=5000;
>informat text $5000.;
>input text;
>if _N_<1000001;
>run;
>
>HTH.