Date:         Thu, 13 Nov 1997 11:09:25 -0000
Reply-To:     Martin Trollope <MartinT@HOLLARD.CO.ZA>
Sender:       "SAS(r) Discussion" <SAS-L@UGA.CC.UGA.EDU>
From:         Martin Trollope <MartinT@HOLLARD.CO.ZA>
Subject:      Re: Sort with Experimental large file
Comments: To: Bernard Tremblay <bernard@CAPITALE.QC.CA>
Content-Type: text/plain; charset="us-ascii"

Hi Bernard,
 
I've had many problems with large sorts on MVS and on Windows. Eventually I found that splitting the large dataset into several smaller pieces , then sorting them individually and then interleaving them afterwards is considerably more efficient than trying to sort the whole lot a once.
 
Below is some sample code
 
Martin Trollope
martint@hollard.co.za
 
data dset1     dset2     dset3     dset4     dset5     dset6
     dset7     dset8     dset9     dset10     dset11     dset12
     dset13     dset14     dset15     dset16     dset17     dset18
     dset19     dset20;
set inpdset nobs=parm;
x = ceil(parm/20);
select;
  when (_n_<x) output dset1;
  when (_n_<x*2) output dset2;
  when (_n_<x*3) output dset3;
  when (_n_<x*4) output dset4;
  when (_n_<x*5) output dset5;
  when (_n_<x*6) output dset6;
  when (_n_<x*7) output dset7;
  when (_n_<x*8) output dset8;
  when (_n_<x*9) output dset9;
  when (_n_<x*10) output dset11;
  when (_n_<x*11) output dset10;
  when (_n_<x*12) output dset12;
  when (_n_<x*13) output dset13;
  when (_n_<x*14) output dset14;
  when (_n_<x*15) output dset15;
  when (_n_<x*16) output dset16;
  when (_n_<x*17) output dset17;
  when (_n_<x*18) output dset18;
  when (_n_<x*19) output dset19;
  otherwise output dset20;
end;
run;
 
%macro sorter;
  %do a = 1 %to 20;
    proc sort data=dset&a.;
      by var1 var2 var3;
    run;
  %end;
%mend sorter;
%sorter;
 
%sorter;
 
data addrlist;
 set dset1     dset2     dset3     dset4     dset5     dset6
     dset7     dset8     dset9     dset10     dset11     dset12
     dset13     dset14     dset15     dset16     dset17     dset18
     dset19     dset20;
by polno itemno priority;
if first.itemno;
run;
 
-----Original Message-----
From:   Bernard Tremblay [SMTP:bernard@CAPITALE.QC.CA]
Sent:   12 November 1997 12:11
To:     SAS-L@AKH-WIEN.AC.AT
Subject:        Sort with Experimental large file
 
Hi,
 
        I would like to use the large file implementation for sorting
big files.  Actually I can create large files without any problem.  But if
you want to sort them SAS will abend because it's missing space on work...
 
        What I did try is to specifiy the following in the config file:
 
                -partsize 512M
                -work (/work1,/disk2,/disk3,/disk4)
 
        Then I launch the following program:
 
                proc sort data=lib1.file1  out=lib2.file2;by var1;run;
 
        Even though lib1 and lib2 have sufficient space for the file,
SAS abends because it creates a work file on the first disk specified
in the work parameter of the config and that file grows too big for
this one disk to hold the file.  The work file can grow larger than
the maximum partsize specified in the config...
 
        I'm running sas 6.12.0020P103196
        on SunOS poseidon 5.4 Generic_Patch sun4d sparc.
 
        It looks like the sort program (the one from SAS) doesn't support
the large file implementation.
 
        Anybody can help ???
 
                        Bernard Tremblay
          \\\|///
        \\  - -  //
         (  @ @  )
+------oOOo-(_)-oOOo----------+---------------------------------+
|  Bernard Tremblay           |                                 |
|             La Capitale     | Tel:  (418) 646-2401            |
|                             | Fax:  (418) 646-5960            |
|                             | Int:  bernard@capitale.qc.ca    |
+-----------------------------+---------------------------------+
|             Imaginasys enr  | Res:  (418) 878-4447            |
|                             | Int:  bertrem@quebectel.com     |
+---------------Oooo----------+---------------------------------+
        oooO   (   )
       (   )    ) /
        \ (    (_/
         \_)