|
Hi Chang,
Nice example program. When the OP mentioned import it took it to mean
PROC IMPORT. So I modified your program to make a first record of
variable names and ran it with PROC IMPORT. I stopped it after about
10 minutes. I may try to let it run longer just to see if it can
finish.
Perhaps PROC IMPORT is overwhelmed by the number of variables.
%let seed = 123987;
filename ft44f001 temp lrecl=%eval(12 + 20000*13);
data _null_;
tab = '09'x;
/* file "20K.tab" lrecl=%eval(12 + 20000*13);*/
file ft44f001;
_n_ = 199;*99;
put 'id' @;
do j=1 to _n_;
put tab $1. 'j' j :8.-l +(-1)@;
end;
put tab $1. 'v';
do id = 1 to 10;
put id best12. @;
do j = 1 to _n_;
v = ranuni(&seed.);
put tab $1. v best12. @;
end;
v = 10*id; /* for checking */
put tab $1. v best12. @;
put;
end;
stop;
run;
data _null_;
infile ft44f001;
input;
list;
stop;
run;
proc import datafile=ft44f001 out=work.test dbms=tab replace;
run;
proc contents varnum;
run;
On Dec 6, 2007 10:46 AM, Chang Chung <chang_y_chung@hotmail.com> wrote:
> On Thu, 6 Dec 2007 08:17:31 -0800, cherub <cherub2life@YAHOO.COM> wrote:
>
> > I have a very urgent case, my data has more than 20,000 variables and
> tab-delimited,.
> > First variable is ID, My sas took one night and still is importing the
> first two obs.
> >
> > Does somebody have any experience about importing this kind of data, or
> how to spit the data by colomns with keep the id?
> ...
>
> hi, cherub,
> believe it or not, this is well-within the limits of sas. something else
> other than sas is at fault. For example, on my modest PC running windows
> vista and sas 9.1.3 sp4, sas can read a tab delimited file with 20,001
> variables and 10 observations much less than a second!
> cheers,
> chang
>
>
> %let pwd = %sysfunc(pathname(WORK));
> %put pwd=&pwd.;
> x cd &pwd.;
>
> /* create a tab-delimited text file with id and 20K numeric vars
> written with best12. format and 10 obs */
> %let seed = 123987;
> data _null_;
> tab = '09'x;
> file "20K.tab" lrecl=%eval(12 + 20000*13);
> do id = 1 to 10;
> put id best12. @@;
> do j = 1 to 19999;
> v = ranuni(&seed.);
> put tab $1. v best12. @@;
> end;
> v = 10*id; /* for checking */
> put tab $1. v best12. @@;
> put;
> end;
> stop;
> run;
>
> /* read back the text file */
> data one;
> /* windows max lrecl is 1M */
> infile "20K.tab" lrecl=1048576 dsd dlm='09'x;
> input id v1-v20000;
> run;
> /* on log
> NOTE: 10 records were read from the infile "20K.tab".
> The minimum record length was 260012.
> The maximum record length was 260012.
> NOTE: The data set WORK.ONE has 10 observations and 20001 variables.
> NOTE: DATA statement used (Total process time):
> real time 0.15 seconds
> cpu time 0.15 seconds
> */
>
> /* check */
> proc print data=one;
> var id v1 v19999 v20000;
> run;
> /* on lst
> Obs id v1 v19999 v20000
> 1 1 0.95414 0.73833 10
> 2 2 0.76909 0.65193 20
> 3 3 0.38431 0.49741 30
> 4 4 0.92854 0.64309 40
> 5 5 0.70494 0.95897 50
> 6 6 0.29815 0.02166 60
> 7 7 0.10953 0.94254 70
> 8 8 0.42944 0.81144 80
> 9 9 0.49239 0.16634 90
> 10 10 0.78268 0.84273 100
> */
>
|
