Date:         Sun, 8 Jul 2007 23:07:36 -0400
Reply-To:     Arthur Tabachneck <art297@NETSCAPE.NET>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Arthur Tabachneck <art297@NETSCAPE.NET>
Subject:      Re: Trying to read in a large .csv file
Comments: To: Andrea Wainwright-Zimmerman <sassywench74va@GMAIL.COM>

Andrea,

I obviously have no idea if you are confronting a similar situation to the
one I had to solve the other day but, in the chance I'm not the only one
who is receiving some unique files, my solution may be of help.

To try and discover why SAS couldn't read the file, I downloaded a
shareware dos program called list.com.  If interested, it is available at:

http://www.buerg.com/downloads/listplus.zip

When you open a file with LIST, pressing alt-H shows the file entirely in
hexadecimal characters.  To shorten an otherwise long story, the file
started with the hexidecimal characters FF and FE, and then had null
characters (hex=00) imbeded between every other character.

I used SAS to rewrite the file, with the following code, and then proc
import worked as expected on the new file.

data _NULL;
  FILE "c:\want.txt"  RECFM=N ;
  infile 'c:\have.txt' RECFM=N ;
  input VAR1 $CHAR1. @;
  IF VAR1 GE '09'X  AND VAR1 LE '7A'X THEN do;
    PUT VAR1 $CHAR1. ;
  end;
run;

Hope that may help you as well,
Art
----------
On Sun, 8 Jul 2007 20:52:51 -0400, Andrea Zimmerman
<sassywench74va@GMAIL.COM> wrote:

>I know the file is huge, I tried to open it in Excel and it has so many
rows
>Excel can't handle it all.
>
>So I tried to import it into SAS, but SAS is only picking up 3232 rows!!!!
>
>I did better with Excel.
>
>Any ideas why?
>
>This is PC sas 9.1.
>
>Here's the code the import wizard generated:
>*
>
>PROC* *IMPORT* OUT= MSVLM.Freqs
>
>DATAFILE= "Z:\My Documents\CRM\VLM\MSVLM\MSVLM freqs\ID2 May 07
>data\msvlm_ID2_freq.csv"
>
>DBMS=CSV REPLACE;
>
>GETNAMES=YES;
>
>DATAROW=*2*;
>*
>
>RUN*;
>
>I really only need a small portion of the file.  Can I include if clauses
>here to import only the rows I need or should I import it all and then
keep
>the rows I need?
>
>TIA
>
>
>
>Andrea Wainwright-Zimmerman
>
>Capital One