Date:         Thu, 19 Mar 2009 13:36:54 -0500
Reply-To:     "./ ADD NAME=Data _null_;" <iebupdte@GMAIL.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "./ ADD NAME=Data _null_;" <iebupdte@GMAIL.COM>
Subject:      Re: Import many text files
Comments: To: Olaanaa <walaba@googlemail.com>
In-Reply-To:  <9aed6c71-a412-443e-af9c-27ff64670f1b@o11g2000yql.googlegroups.com>
Content-Type: text/plain; charset=ISO-8859-1

SAS provides a good set of tools to accomplish this task.  You can
read the output of a command "DIR" and use that information to open
files and read them with a powerful parser the INPUT statement.  I
would read all data into one SAS data set as that will facilitate data
summary later on.  You don't want a SAS data set for each txt file.

This example skips over the header and just reads the aerosol data.
You read the header as you see fit.

The first part of the program creates some sample data using the data
you posted, in the WORK directory.  You don't need to do that but you
do need to define a fileref that points to the directory where your
files are.  Then use that fileref in the data step further below where
you see pathname('WORK').

%let wpath = %sysfunc(pathname(WORK));
filename FT15F001 "&wpath\Man01_030408.txt";
parmcards4;
TrakPro Version 4.00 ASCII Data File
Model:,SidePak Aerosol Monitor
Model Number:,AM510
Serial Number:,10707008
Test ID:,002
Test Abbreviation:,
Start Date:,04/03/2008
Start Time:,12:44:15
Duration (dd:hh:mm:ss):,0:08:37:00
Time constant (seconds):,0
Log Interval (mm:ss):,01:00
Number of points:,517
Notes:,

Date,Time,Aerosol
dd/MM/yyyy,hh:mm:ss,mg/m^3
04/03/2008,12:45:15,0.004
04/03/2008,12:46:15,0.006
04/03/2008,12:47:15,0.003
04/03/2008,12:48:15,0.003
04/03/2008,12:49:15,0.002
04/03/2008,12:50:15,0.004
04/03/2008,12:51:15,0.010
04/03/2008,12:52:15,0.005
04/03/2008,12:53:15,0.016
04/03/2008,12:54:15,0.010
;;;;
   run;
filename FT15F001 "&wpath\Man01_100608.txt";
parmcards4;
TrakPro Version 4.00 ASCII Data File
Model:,SidePak Aerosol Monitor
Model Number:,AM510
Serial Number:,10707008
Test ID:,002
Test Abbreviation:,
Start Date:,04/03/2008
Start Time:,12:44:15
Duration (dd:hh:mm:ss):,0:08:37:00
Time constant (seconds):,0
Log Interval (mm:ss):,01:00
Number of points:,517
Notes:,

Date,Time,Aerosol
dd/MM/yyyy,hh:mm:ss,mg/m^3
04/03/2008,12:45:15,0.004
04/03/2008,12:46:15,0.006
04/03/2008,12:47:15,0.003
04/03/2008,12:48:15,0.003
04/03/2008,12:49:15,0.002
04/03/2008,12:50:15,0.004
04/03/2008,12:51:15,0.010
04/03/2008,12:52:15,0.005
04/03/2008,12:53:15,0.016
04/03/2008,12:54:15,0.010
;;;;
   run;

data work.Air;
   length command filevar filename $128 file $32;
   command = catx(' ','dir /S/B',quote(catx('\',pathname('WORK'),'M*.txt')));
   do while(1);
      infile dummy1 pipe filevar=command eof=eof;
      input;
      filevar=_infile_;
      infile dummy2 filevar=filevar filename=filename end=eof dsd;
      file = scan(filename,-1,'\');
      do while(not eof);
         input @'Date,Time,Aerosol' /;
         do until(eof);
            input (Date Time Aerosol)(:ddmmyy10. :time8. :F8.);
            output;
            end;
         end;
      end;
eof: stop;
   format date ddmmyy10. time time8.;
   run;

proc print;
   run;

On 3/19/09, Olaanaa <walaba@googlemail.com> wrote:
> Hi All,
>
> I have serial measurements of air pollution data that are collected
> from several places at various points in time. The data are comma
> delimited and logged every minute (sample shown below). Apart from the
> header which differs, all collection are of similar format having 3
> variables - Date, Time and Aerosol. Each file is named with place ID
> and date of collection.  Two such successive files for ID1 are named
> Man01_030408.txt and Man01_100608.txt as an example.   I have hundreds
> of such files and need to automate the import of these files into SAS
> and save the file with similar name to the text file. How can I employ
> this situation?
>
> Thanks for your input.
>
> Sample data follows.
>
> TrakPro Version 4.00 ASCII Data File
> Model:,SidePak Aerosol Monitor
> Model Number:,AM510
> Serial Number:,10707008
> Test ID:,002
> Test Abbreviation:,
> Start Date:,04/03/2008
> Start Time:,12:44:15
> Duration (dd:hh:mm:ss):,0:08:37:00
> Time constant (seconds):,0
> Log Interval (mm:ss):,01:00
> Number of points:,517
> Notes:,
>
> Date,Time,Aerosol
> dd/MM/yyyy,hh:mm:ss,mg/m^3
> 04/03/2008,12:45:15,0.004
> 04/03/2008,12:46:15,0.006
> 04/03/2008,12:47:15,0.003
> 04/03/2008,12:48:15,0.003
> 04/03/2008,12:49:15,0.002
> 04/03/2008,12:50:15,0.004
> 04/03/2008,12:51:15,0.010
> 04/03/2008,12:52:15,0.005
> 04/03/2008,12:53:15,0.016
> 04/03/2008,12:54:15,0.010
>