on Mon, May 14, 2001 at 10:37:29PM +0100, Roland (roland.rashleigh-berry@VIRGIN.NET) wrote: > Karsten M. Self wrote: > > > > on Mon, May 14, 2001 at 08:26:41PM +0100, Roland (roland.rashleigh-berry@VIRGIN.NET) wrote: > > > > > Suppose you have a lot of SAS jobs running at the same time on Unix. > > > Let's say the long-running ones are set to a low CPU dispatching > > > priority and there is no problem with memory or disks being too busy > > > and the processors on average are running at 45%. Why on earth would > > > it run slow as a dog for all the SAS users? Any ideas? > > > > Qualify and quantify your statements. I strongly recommend: > > > > Mike Loukides, _System Performance Tuning_, O'Reilly & Associates, > > ©1990, Cambridge, MA. ISBN: 0-937175-60-9 > > > > Despite the age, fundamentals of performance tuning haven't changed > > significantly. > > I know. But where is the bottleneck? > > > What is "slow as a dog" and how are you assessing this? Are your jobs > > running more slowly when run in parallel than when run serially? How > > No. Me running an ordinary SAS job independently would take 5-10 times > longer than normal. And this for two other people in the room not > working on the same project.

Looks like you're getting better than linear scaling, though you're still being less than clear on what you're comparing between. 5-10 times slower processing (I assume between an otherwise unloaded system and a loaded one) when running 12-24 jobs is pretty good scalability.

> > are you asessing CPU, disk I/O, and memory contention? How have you > > configured your system, particularly the disk subsystems? Is network > > access to storage involved? > > Yes, network acces, but the performance problem was all down to jobs > running slow on one server, not in their communicating their results > down a netwerk.

Define "network access". What's critical is whether or not the SAS process itself is accessing data over the network, not whether or not you're accessing the server over the network. The bandwidth required for an editor or even an interactive SAS session is negligible compared to bandwidth required to shuttle data from remote storage to local CPU and back out again.

> > It would also help to specify the Unix vendor as system monitoring > > tools vary somewhat by platform. In general, top, ps, and a memory > > monitoring > > "top", "iostat" and "vmstat" used.

On Solaris. Complete answers please.

Post output of these commands.

> > utility (varies: free, mem, vmstat are common ones) will be your > > friends. Critical data include disk throughput and the number of jobs > > blocked waiting for I/O. > > Disks did not seem to be busy even.

Describe methods and rationale for this assessment, with program output.

> > Prioritizing jobs with nice will often fail spectacularly in producing > > any reasonable results when I/O contention is a bottleneck. 'nice' is > > more useful for rationalizing CPU-bound processes. > > According to "iostat", disk contention was not the problem.

Post output.

-- Karsten M. Self <kmself@ix.netcom.com> http://kmself.home.netcom.com/ What part of "Gestalt" don't you understand? There is no K5 cabal http://gestalt-system.sourceforge.net/ http://www.kuro5hin.org