Date:         Wed, 9 Jan 2008 14:33:56 -0500
Reply-To:     Peter Flom <peterflomconsulting@mindspring.com>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Peter Flom <peterflomconsulting@MINDSPRING.COM>
Subject:      Re: AIC mystery in MIXED
Comments: To: Ryan Utz <rutz@AL.UMCES.EDU>
Content-Type: text/plain; charset=UTF-8

I don't know if you can formally test this difference, but you can look at it by plotting the predicted variables from the two models against each other.  If the models predict very similar things, then whihc is 'better' is sort of beside the point.

Suppose you did come up with some statistical test.  Well, with a large enough sample, any difference will be significant.  With a small enough sample, even a very large difference will *not* be significant.

Evem when you *can* compare AIC values (or AICC, or BIC, or SBC, or whatever) it's often worthwhile considering if the more complex model is worth it, substantively

Peter

-----Original Message-----
>From: Ryan Utz <rutz@AL.UMCES.EDU>
>Sent: Jan 9, 2008 1:08 PM
>To: SAS-L@LISTSERV.UGA.EDU
>Subject: Re: AIC mystery in MIXED
>
>...the two comments I received on this issue were immensely helpful; thank
>you for improving my understanding of what's going on.
>
>But now a question arises: Is there a way to test a linear- versus power-fit
>using MIXED (or another procedure that produces AIC scores)?  I ask because
>related research papers on what I'm looking at report doing so, but if I
>have to keep my response variable constant, I'm not sure how I'd do so.  I
>would prefer to use MIXED because I'm really testing a more complicated
>(repeated-measures ANCOVA) model rather than the one I posted.  Any
>suggestions whatsoever (AIC or anything else) would be welcomed and appreciated.
>
>-Ryan
>
>
>>Hi all,
>>
>>I'm having issues using/interpreting AIC scores in proc MIXED.  I'm trying
>>to compare simple linear relationships with power function relationships
>>(both models have been shown to be consistently valid in related datasets).
>> When I go to interpret AIC (or AICc, etc) scores, however, power
>>relationships always emerge as the better model, even when it clearly isn't
>>the case.  As an example, I provided my actual data for an extremely simple
>>model at the bottom of this email (I'm testing much more complex models, but
>>the example below illustrates the problem).  To test the power relationship,
>>I've log-transformed both X and Y.  Running the code below shows that MIXED
>>suggests the power relationship is better (it has a lower AIC score), but if
>>you run a simple linear regression, clearly the non-transformed data (thus a
>>linear relationship) is superior.  This is true even when both models have
>>the exact same number of parameters.
>>
>>Is there something I'm doing wrong here, either in execution or
>>interpretation?  I'd like to use AIC scores to help choose a model, but
>>because of this issue I'm vary hesitant.
>>
>>Thanks ahead of time for any advice,
>>
>>Ryan Utz
>>University of Maryland Center for Environmental Science
>>
>>
>>data test;
>>input density length; cards;
>>0.099266504 82.8125
>>0.048193642 85.05405405
>>0.114893617 84.34210526
>>0.257685811 70.515625
>>0.044660194 86.92857143
>>0.244736842 76.37647059
>>0.020619946 89.5
>>0.058555133 93.6
>>0.125817923 84.08888889
>>
>>data test2; set test;
>>lndensity = log(density);
>>lnlength= log (length); run;
>>
>>title Linear Relationship;
>>proc mixed data=test2;
>>model length=density; run;
>>
>>title Power Relationship;
>>proc mixed data=test2;
>>model lnlength=lndensity; run;
>>
>>/*Simple regression for comparison*/
>>
>>Title Linear relationship-simple regression;
>>proc glm data=test2;
>>model length=density; run;
>>
>>Title Linear relationship-Power function;
>>proc glm data=test2;
>>model lnlength=lndensity; run;