Date:         Fri, 25 Feb 2000 11:59:04 -0600
Reply-To:     "Nichols, David" <nichols@SPSS.COM>
Sender:       "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From:         "Nichols, David" <nichols@SPSS.COM>
Subject:      Re: post hoc tests for unequal N

I don't see why you'd expect these to always give similar results. They're
different tests using different criteria. I wouldn't use either one with
unequal N.

David Nichols
Principal Support Statistician and
Manager of Statistical Support
SPSS Inc.

> -----Original Message-----
> From: Thompson Bill T Contr USAFSAM/FEC
> [mailto:Bill.Thompson@BROOKS.AF.MIL]
> Sent: Friday, February 25, 2000 11:35 AM
> To: SPSSX-L@LISTSERV.UGA.EDU
> Subject: Re: [SPSSX-L] post hoc tests for unequal N
>
>
> David, I just ran the MANOVA again and asked for both the Tukey b and
> Waller-Duncan post hoc tests.  Although for many of the variables the
> results are equivalent for many of the tests the results are quite
> different.  Again forgive my ingorance and can someone
> explain to me why the
> results are so different and which test results are the most
> appropriate to
> report.  I realize this may seem somewhat trivial to some,
> but in my world
> differences can me the difference between life and death,
> teating or not
> treating. Therefore, again, please forgive my ignorance and
> any help is
> greatly appreciated.
>
> Thanks in advance,
>
> Bill T.
>
> -----Original Message-----
> From: Nichols, David [mailto:nichols@SPSS.COM]
> Sent: Thursday, February 24, 2000 5:09 PM
> To: SPSSX-L@LISTSERV.UGA.EDU
> Subject: Re: post hoc tests for unequal N
>
>
> I don't know exactly what Statistica has implemented, but
> there's no reason
> to expect Spjotvoll-Stoline results to match those from
> REGWF. The closest
> thing that SPSS has to a Spjotvoll-Stoline approach would be
> a Tukey-Kramer,
> which is what you get for the pairwise comparisons output
> with unequal Ns
> when you specify Tukey. Since this is always as powerful as the
> Spjotvoll-Stoline and maintains Type I error protection, I
> see no reason to
> use a Spjotvoll-Stoline as opposed to a Tukey-Kramer.
>
> All of the methods for homogeneous subsets in SPSS use the
> harmonic mean of
> the sample sizes in order to maintain monotonicity of
> critical values and
> logical rationale of the stepdown tests (these methods were
> designed for
> equal sample sizes). They generally do not maintain Type I
> error protection
> with unequal N and are therefore not recommended for that
> case (we actually
> print a footnote indicating that Type I error levels are not
> guaranteed).
>
> There are further problems with the REGWF approach. Even for
> equal sample
> sizes, the critical values are not always monotonic, leading
> to situations
> where applying the test correctly is difficult or impossible.
> Jason Hsu
> discusses this in his Multiple Comparisons: Theory and
> Methods. He brought
> this to my attention some time ago and we did some tweaking
> in response, but
> it doesn't really solve the problem, so I'd be very hesitant
> to recommend
> that method. Interestingly enough, I just tried his example
> in Release 8 of
> SAS, and requesting the REGWF method on the MEANS statement
> in PROC GLM
> produces the following: "NOTE: The REGWF option has been
> disabled and will
> no longer be available.  You can use the REGWQ option instead."
>
> David Nichols
> Principal Support Statistician and
> Manager of Statistical Support
> SPSS Inc.
>
> > -----Original Message-----
> > From: Thompson Bill T Contr USAFSAM/FEC
> > [mailto:Bill.Thompson@BROOKS.AF.MIL]
> > Sent: Wednesday, February 23, 2000 11:25 AM
> > To: SPSSX-L@LISTSERV.UGA.EDU
> > Subject: [SPSSX-L] post hoc tests for unequal N
> >
> >
> > I ran a MANOVA with 13 DV's and 1 IV (Group with 5 levels).
> > The overall
> > MANOVA as significant at .00001.  The five levels of Group have the
> > following N's:
> >
> > Group   N
> > 1       148
> > 2       5051
> > 3       134
> > 4       69
> > 5       103
> >
> > Can someone tell me which Post Hoc test would be the most
> > appropriate to use
> > in SPSS.  Statistica provides the Tukey HSD for unequal N's
> > (Spjotvoll/Stoline test) but the results are significantly
> (excuse the
> > choice of word) different from those of the REGW F performed
> > in SPSS.  My
> > ignorance causes me some concern and I would appreciate any
> > insight and
> > recommendations on performing Post Hoc tests with unequal N'
> > and possibly
> > making sense of the REGW F output in SPSS.
> >
> > Thanks in advance for your assistance,
> >
> > Bill Thompson - Biostatistician
> > Conceptual MindWorks Inc.
> > School of Aerospace Medicine
> > Aeromedical Consult Service - USAFSAM/FEC
> > 2507 Kennedy Circle
> > Brooks Air Force Base, Texas 78247 - 5117
> > DSN: 240-6835  Comm: 536-6835
> > Fax: DSN/Comm - 2817
> > email: bill.thompson@brooks.af.mil
> >
>