Ulrike:

If you write out a file with the same name as your source file, that means you need space equal to the source file plus the output file. Why? Because SAS won't delete the source file and rename the output file until the very end of the proc or data step. Until then, the output file will have a different, temporary name. (In Windows, open up an explorer window and watch the relevant folders as the program runs!)

In a DATA step merge, you're unlikely to use the WORK library to a significant extent.

In a SQL merge, you'll be sorting all the files you join, and will thus need something on the order of 3x what you include from the source file(s). In general, expect to use 3x the file size when sorting, and expect to use it in the WORK library.

Oh, and by the way: IDE/ATA storage space now costs on the order of $2/GB. If you're running Windows, have your boss spring for a couple of 60GB drives, one for WORK, one for keepers. And then design your programs to use multiple drives at once, minimizing the steps that involve reading from, and writing to, the same drive at once. If your source files are on one drive, and your output on another, you generally save time. If you're merging a big file to a small one, it may actually be economical to add a step to copy the smaller file to a drive other than the output drive.

Best,

Roger

Ulrike Grömping wrote: > > Dear all, > > I am confused about SAS' usage of storage: When do I need space in the > work library, and when do I need it in the permanent library ? > > I will have to handle some very large data sets, and have just started > with a roughly 800MB/3.5Mio observation dataset (PC). I have two > partitions on my computer, and had assigned the permanent library to the > partition with less free space (D:), assuming that I would need just > enough space for accomodating the data set. > > However, even though I could not find a physical file causing the > problem, a data step with a merge, that did not change the size of the > permanent data set, resulted drive D: running out of space. When I > reallocated the permanent library to the larger partition, all went > well. Does anybody know, how SAS handles space in the data step, i.e.: > > - When does it use the work library ? > - When does it use the permanent library instead? > - Where does it write onto the disk, if the disk gets fuller and fuller, > while I can't find the file that causes this ? > > Regards, Ulrike