Date: Fri, 25 Feb 2000 11:47:23 -0600
Reply-To: Thompson Bill T Contr USAFSAM/FEC <Bill.Thompson@BROOKS.AF.MIL>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Thompson Bill T Contr USAFSAM/FEC <Bill.Thompson@BROOKS.AF.MIL>
Subject: Re: post hoc tests for unequal N
Content-Type: text/plain; charset="iso-8859-1"
Sorry, forgot to ask for help in understanding the use of the type1/type 2
error seriousness for the Waller-Duncan test. Is there a reference I can
read on this procedure? I have been unable to find anything in the v.9
manuals.
Thanks again,
Bill T.
-----Original Message-----
From: Nichols, David [mailto:nichols@SPSS.COM]
Sent: Thursday, February 24, 2000 5:09 PM
To: SPSSX-L@LISTSERV.UGA.EDU
Subject: Re: post hoc tests for unequal N
I don't know exactly what Statistica has implemented, but there's no reason
to expect Spjotvoll-Stoline results to match those from REGWF. The closest
thing that SPSS has to a Spjotvoll-Stoline approach would be a Tukey-Kramer,
which is what you get for the pairwise comparisons output with unequal Ns
when you specify Tukey. Since this is always as powerful as the
Spjotvoll-Stoline and maintains Type I error protection, I see no reason to
use a Spjotvoll-Stoline as opposed to a Tukey-Kramer.
All of the methods for homogeneous subsets in SPSS use the harmonic mean of
the sample sizes in order to maintain monotonicity of critical values and
logical rationale of the stepdown tests (these methods were designed for
equal sample sizes). They generally do not maintain Type I error protection
with unequal N and are therefore not recommended for that case (we actually
print a footnote indicating that Type I error levels are not guaranteed).
There are further problems with the REGWF approach. Even for equal sample
sizes, the critical values are not always monotonic, leading to situations
where applying the test correctly is difficult or impossible. Jason Hsu
discusses this in his Multiple Comparisons: Theory and Methods. He brought
this to my attention some time ago and we did some tweaking in response, but
it doesn't really solve the problem, so I'd be very hesitant to recommend
that method. Interestingly enough, I just tried his example in Release 8 of
SAS, and requesting the REGWF method on the MEANS statement in PROC GLM
produces the following: "NOTE: The REGWF option has been disabled and will
no longer be available. You can use the REGWQ option instead."
David Nichols
Principal Support Statistician and
Manager of Statistical Support
SPSS Inc.
> -----Original Message-----
> From: Thompson Bill T Contr USAFSAM/FEC
> [mailto:Bill.Thompson@BROOKS.AF.MIL]
> Sent: Wednesday, February 23, 2000 11:25 AM
> To: SPSSX-L@LISTSERV.UGA.EDU
> Subject: [SPSSX-L] post hoc tests for unequal N
>
>
> I ran a MANOVA with 13 DV's and 1 IV (Group with 5 levels).
> The overall
> MANOVA as significant at .00001. The five levels of Group have the
> following N's:
>
> Group N
> 1 148
> 2 5051
> 3 134
> 4 69
> 5 103
>
> Can someone tell me which Post Hoc test would be the most
> appropriate to use
> in SPSS. Statistica provides the Tukey HSD for unequal N's
> (Spjotvoll/Stoline test) but the results are significantly (excuse the
> choice of word) different from those of the REGW F performed
> in SPSS. My
> ignorance causes me some concern and I would appreciate any
> insight and
> recommendations on performing Post Hoc tests with unequal N'
> and possibly
> making sense of the REGW F output in SPSS.
>
> Thanks in advance for your assistance,
>
> Bill Thompson - Biostatistician
> Conceptual MindWorks Inc.
> School of Aerospace Medicine
> Aeromedical Consult Service - USAFSAM/FEC
> 2507 Kennedy Circle
> Brooks Air Force Base, Texas 78247 - 5117
> DSN: 240-6835 Comm: 536-6835
> Fax: DSN/Comm - 2817
> email: bill.thompson@brooks.af.mil
>
