Date:         Mon, 14 Apr 2003 16:18:25 -0700
Reply-To:     cassell.david@EPAMAIL.EPA.GOV
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "David L. Cassell" <cassell.david@EPAMAIL.EPA.GOV>
Subject:      Re: MME in Proc IML and in Proc Mixed
Content-type: text/plain; charset=iso-8859-1

Joël Rivest <jrivest@CDPQINC.QC.CA> wrote:
> I try to analyse a random coefficient model using Proc IML and for
this, I need to build the mixed model equations (MME) associated with
the problem. To build the MME I need to square a matrix that is 5800
rows by 1372 columns. That operation requires a lot of time with
Proc IML. However, when I perform the same analysis with Proc Mixed,
the solutions and the MME are produced within few seconds. I changed
the "worspace" and "symspace" option of Proc IML but it didn't improve
anything. My question is why Proc Mixed is so efficient in building the
MME (or IML so inefficient) and there is a way to improve the efficiency
of Proc IML for that purpose (building the MME)?

5800 by 1372?  That's kind of large.  Have you computed how much
space PROC IML needs in order to work with this?  Your machine may
be doing a lot of page-swapping in order to go through all the steps
you have incorporated.  PROC MIXED is all nice an compiled to do the
matrix manipulations as efficiently as possible, while PROC IML may
have to do a fair amount of interpreation of your IML code, and you
may not be specifying the most optimal methods of matrix manipulation.
Still, I'll guess that the size of your rectangular matrix is leading
to page-swapping, and the I/O is making the difference.

My recommendation is not to use PROC IML for tasks which are suited
to PROC MIXED.  And I find it somewhat disturbing that you have a
5800 x 1372 matrix in the first place.  Just how many coefficients
are you specifying?  Aren't you concerned about variable-selection
problems?

HTH,
David
--
David Cassell, CSC
Cassell.David@epa.gov
Senior computing specialist
mathematical statistician