Date:         Fri, 25 Feb 2000 12:06:13 -0600
Reply-To:     "Nichols, David" <nichols@SPSS.COM>
Sender:       "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From:         "Nichols, David" <nichols@SPSS.COM>
Subject:      Re: post hoc tests for unequal N

Waller, R. A., & Duncan, D. B. (1969). A Bayes rule for the symmetric
multiple comparison problem. Journal of the American Statistical
Association, 64, 1484-1499. Corrigenda, 67, 253-255.

Duncan, D. B. (1975), t tests and intervals for comparisons suggested by the
data. Biometrics, 31, 339-360.

Waller, R. A. and Kemp, K. E. (1975), Computations of Bayesian t-value for
multiple comparison. Journal of Statistical Computation and Simulation, 4,
169-172.

Incidentally, in digging these out of our internal algorithms document, I
notice that it explicitly says that the Waller-Duncan method isn't
applicable to the unequal N case. We dealt with this issue by using harmonic
means and warning people in order to not have to try to disallow the use of
certain methods in certain situations.

David Nichols
Principal Support Statistician and
Manager of Statistical Support
SPSS Inc.

> -----Original Message-----
> From: Thompson Bill T Contr USAFSAM/FEC
> [mailto:Bill.Thompson@BROOKS.AF.MIL]
> Sent: Friday, February 25, 2000 11:47 AM
> To: SPSSX-L@LISTSERV.UGA.EDU
> Subject: Re: [SPSSX-L] post hoc tests for unequal N
>
>
> Sorry, forgot to ask for help in understanding the use of the
> type1/type 2
> error seriousness for the Waller-Duncan test.  Is there a
> reference I can
> read on this procedure?  I have been unable to find anything
> in the v.9
> manuals.
>
> Thanks again,
>
> Bill T.
>
> -----Original Message-----
> From: Nichols, David [mailto:nichols@SPSS.COM]
> Sent: Thursday, February 24, 2000 5:09 PM
> To: SPSSX-L@LISTSERV.UGA.EDU
> Subject: Re: post hoc tests for unequal N
>
>
> I don't know exactly what Statistica has implemented, but
> there's no reason
> to expect Spjotvoll-Stoline results to match those from
> REGWF. The closest
> thing that SPSS has to a Spjotvoll-Stoline approach would be
> a Tukey-Kramer,
> which is what you get for the pairwise comparisons output
> with unequal Ns
> when you specify Tukey. Since this is always as powerful as the
> Spjotvoll-Stoline and maintains Type I error protection, I
> see no reason to
> use a Spjotvoll-Stoline as opposed to a Tukey-Kramer.
>
> All of the methods for homogeneous subsets in SPSS use the
> harmonic mean of
> the sample sizes in order to maintain monotonicity of
> critical values and
> logical rationale of the stepdown tests (these methods were
> designed for
> equal sample sizes). They generally do not maintain Type I
> error protection
> with unequal N and are therefore not recommended for that
> case (we actually
> print a footnote indicating that Type I error levels are not
> guaranteed).
>
> There are further problems with the REGWF approach. Even for
> equal sample
> sizes, the critical values are not always monotonic, leading
> to situations
> where applying the test correctly is difficult or impossible.
> Jason Hsu
> discusses this in his Multiple Comparisons: Theory and
> Methods. He brought
> this to my attention some time ago and we did some tweaking
> in response, but
> it doesn't really solve the problem, so I'd be very hesitant
> to recommend
> that method. Interestingly enough, I just tried his example
> in Release 8 of
> SAS, and requesting the REGWF method on the MEANS statement
> in PROC GLM
> produces the following: "NOTE: The REGWF option has been
> disabled and will
> no longer be available.  You can use the REGWQ option instead."
>
> David Nichols
> Principal Support Statistician and
> Manager of Statistical Support
> SPSS Inc.
>
> > -----Original Message-----
> > From: Thompson Bill T Contr USAFSAM/FEC
> > [mailto:Bill.Thompson@BROOKS.AF.MIL]
> > Sent: Wednesday, February 23, 2000 11:25 AM
> > To: SPSSX-L@LISTSERV.UGA.EDU
> > Subject: [SPSSX-L] post hoc tests for unequal N
> >
> >
> > I ran a MANOVA with 13 DV's and 1 IV (Group with 5 levels).
> > The overall
> > MANOVA as significant at .00001.  The five levels of Group have the
> > following N's:
> >
> > Group   N
> > 1       148
> > 2       5051
> > 3       134
> > 4       69
> > 5       103
> >
> > Can someone tell me which Post Hoc test would be the most
> > appropriate to use
> > in SPSS.  Statistica provides the Tukey HSD for unequal N's
> > (Spjotvoll/Stoline test) but the results are significantly
> (excuse the
> > choice of word) different from those of the REGW F performed
> > in SPSS.  My
> > ignorance causes me some concern and I would appreciate any
> > insight and
> > recommendations on performing Post Hoc tests with unequal N'
> > and possibly
> > making sense of the REGW F output in SPSS.
> >
> > Thanks in advance for your assistance,
> >
> > Bill Thompson - Biostatistician
> > Conceptual MindWorks Inc.
> > School of Aerospace Medicine
> > Aeromedical Consult Service - USAFSAM/FEC
> > 2507 Kennedy Circle
> > Brooks Air Force Base, Texas 78247 - 5117
> > DSN: 240-6835  Comm: 536-6835
> > Fax: DSN/Comm - 2817
> > email: bill.thompson@brooks.af.mil
> >
>