Date: Tue, 14 Feb 2006 18:30:44 -0500
Reply-To: Jim Groeneveld <jim1stat@YAHOO.CO.UK>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Jim Groeneveld <jim1stat@YAHOO.CO.UK>
Subject: Re: comparison of levels
Content-Type: text/plain; charset=ISO-8859-1
Hi Kevin,
Opinions are divided regarding this subject. First of all I understand you
tested all four levels at once and found significance (somewhere at least).
Now you want to search for significance at each level. Well, if you hadn't
done the overall test you wouldn't have to perform any correction at all,
at least according to a specific approach.
The most conservative approach is the Bonferroni one, where you have to
correct for multiple testing by dividing the usual alpha (.05) by the
number of multiple tests to carry out. This reduces the chances on
significance drastically. In your case the resulting alpha, also for the
overall test, would be .01 (.05 / 5). Is the overall test still significant
at that alpha level?
There are quite some less conservative approaches, much the least
conservative one is the one proposed by Keppel. I don't have the references
at hand, but you can search for "Keppel partioning alpha". Keppel divides
the alpha by the total number of tests, and multiplies it by the number of
involved independent (orthogonal) tests.
In your case the four tests at each level are not independent of the
overall test, but independent of each other, so you might divide the total
alpha by 5 and multiply by 4, yielding a test-wise alpha, also for the
overall test!, of .04. A little bit more conservative would be to divide
the alpha equally over both the overall test and each of the sub tests
as .025. Is your overall test still significant at those alpha levels?
Regards - Jim.
P.S. Keppel, Geoffrey. (1991). Design and analysis: A researcher’s handbook
(3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
--
Jim Groeneveld, Netherlands
Statistician, SAS consultant
home.hccnet.nl/jim.groeneveld
[common disclaimer]
On Tue, 14 Feb 2006 17:52:34 -0500, Kevin Roland Viel <kviel@EMORY.EDU>
wrote:
> I performed a chi-squared test to determine whether the distribution of
>my outcome differed among four exclusive categories. The result was
>significant. I used the FREQ procedure.
>
> Now, for each level, I would like to determine if the distribution is
>different. If I correct for multiple testing, does it suffice to make
>four comparisons of the nature level 1 versus others, level 2 versus
>others, etc? What type of correction might I consider?
>
>Thanks,
>
>Kevin
>
>
>Kevin Viel
>Department of Epidemiology
>Rollins School of Public Health
>Emory University
>Atlanta, GA 30322
| 
