Date:         Wed, 23 Jul 2008 17:33:13 -0400
Reply-To:     "Howard Schreier <hs AT dc-sug DOT org>"
              <schreier.junk.mail@GMAIL.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "Howard Schreier <hs AT dc-sug DOT org>"
              <schreier.junk.mail@GMAIL.COM>
Subject:      Re: reading 100K dbf files

On Wed, 23 Jul 2008 07:34:11 -0600, Alan Churchill <savian001@GMAIL.COM> wrote:

>Sent too fast. I forgot you said dbf.

Still, don't be quite so hasty in dismissing INFILE features. DBF files can
be read using a DATA step, if you know what you're doing (see Richard
Hockey's DBF2SAS macro,
http://www.devenezia.com/downloads/sas/macros/index.php?m=dbf2sas).

>
>Look at this paper:
>
>http://support.sas.com/techsup/technote/ts589a.pdf
>
>
>Alan
>
>Alan Churchill
>Savian
>www.savian.net
>
>
>-----Original Message-----
>From: Alan Churchill [mailto:savian001@gmail.com]
>Sent: Wednesday, July 23, 2008 7:26 AM
>To: 'Presley, Rodney (CDC/CCHP/NCBDDD)'; 'SAS-L@LISTSERV.UGA.EDU'
>Subject: RE: reading 100K dbf files
>
>Look at this option on the INFILE statement:
>
>FILEVAR=variable
>names a variable whose change in value causes the INFILE statement to close
>the current input file and open a new one. When the next INPUT statement
>executes, it reads from the new file that the FILEVAR= variable specifies.
>Like automatic variables, this variable is not written to the data set.
>
>
>Alan
>
>
>Alan Churchill
>Savian
>www.savian.net
>
>
>-----Original Message-----
>From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
>Presley, Rodney (CDC/CCHP/NCBDDD)
>Sent: Wednesday, July 23, 2008 6:57 AM
>To: SAS-L@LISTSERV.UGA.EDU
>Subject: reading 100K dbf files
>
>Dear SAS gurus,
>
>
>
>Our SQL data base for one of our projects is suspect.  I want to read in
>all of the data as originally received.  That data is stored in multiple
>DBF files.  All of the files are in one directory.  I have established a
>SAS libname to that directory and it is working.  The challenge is that
>there are over 100,000 files in the directory.  Each DBF file is one
>part of a patient record.
>
>
>
>The files follow a naming convention:
>
>CC#######.DBF
>
>Where 'CC' indicates which part of the patient record that file
>contains.
>
>###, the first three numerals indicate the clinic.
>
>####, the next four numerals indicate the patient.
>
>
>
>Clinic numerals and patient numerals do NOT go in consecutive order.
>
>
>
>I would like to read in all of the DBF files with the same leading two
>characters into one SAS data file.
>
>
>
>If you have ever encountered reading in this many files please let me
>know how you handled it.
>
>
>
>Thanks;,
>
>
>
>Rodney
>
>
>
>Rodney J. Presley, PhD
>Division of Hereditary Blood Disorders
>National Center on Birth Defects and Developmental Disabilities
>Centers for Disease Control and Prevention
>1600 Clifton Road, MS E 64
>Atlanta, GA 30333
>Work Phone: 404-498-6732
>Main Number: 404-498-6750
>Fax: 404-498-6799
>Email: RPresley@cdc.gov