|
Frank,
I agree with you that frequency does not determine obsoleteness. I would
not say there are "no advantages" to the test. In fact, you cite one
advantage in your latest email -- that "it urged analysts to look at
calibration curves."
Also, both I and the authors view the contents of their paper differently
than you do. Hosmer and Lemeshow themselves, in their book Applied Logistic
Regression 2ed (2000), clearly recommend assessing the overall fit of a
logistic model with a combination of the tests to which I referred in my
email below.
Sincerely yours,
Mark J. Lamias
-----Original Message-----
From: Frank E Harrell Jr [mailto:fharrell@virginia.edu]
Sent: Tuesday, June 10, 2003 9:52 PM
To: Mark Lamias
Subject: Re: Hosmer-Lemeshow statistic: susceptible to scale?
On Tue, 10 Jun 2003 14:59:26 -0400
Mark Lamias <Mark.Lamias@grizzard.com> wrote:
> I wouldn't go quite so far as saying the statistic is obsolete. In fact,
it
> probably the most frequently used. Also, upon careful read of the article
> to which you refer, you will find that authors actually find that the fit
of
> a logistic regression model be assessed using several different tests
> together: the Hosmer-Lemeshow decile of risks test, Stukel's test, and
the
> Osius and Rojek normal approximation to the distribution of the Pearson's
> Chi-Squared statistic.
>
> Sincerely yours,
>
> Mark J. Lamias
> Statistical Consultant
Mark,
The fact that something is used frequently does not mean it's not obsolete.
By obsolete I mean there are technical disadvantages with no advantages.
And I don't read their paper the way you did. I see the new test as having
the higher power most often. If you see contradictions to my recollection
please let me know.
Frank
>
> -----Original Message-----
> From: Frank E Harrell Jr [mailto:fharrell@VIRGINIA.EDU]
> Sent: Tuesday, June 10, 2003 2:26 PM
> To: SAS-L@LISTSERV.UGA.EDU
> Subject: Re: Hosmer-Lemeshow statistic: susceptible to scale?
>
>
> On 9 Jun 2003 22:18:02 -0700
> diegov@zip.com.au (Diego V) wrote:
>
> > Sorry to cross post, but I need some help badly. I am currently
> > evaluating a predictive model that was built using logistic
> > regression, and using the HL test statistic to do this. I, however,
> > have found that the larger the sample size that I get, the smaller the
> > value of the statistic.
> >
> > Has anyone experienced a similar problem? TO me, it does not make
> > sense that the larger the sample, and given similar patterns in
> > observed and expected values, we should observe a significantly
> > different HL statistic; however that is exactly what I am getting.
> >
> > Any comments / suggestions on why could this be happening will be most
> > appreciated!!
> >
> > Diego Villaveces
> > email: diegov@zip.com.au
>
> The H-L test is largely obsolete, having been replaced by specific
directed
> tests of lack of fit and by the new test described in:
>
> @ARTICLE{hos97com,
> author = {Hosmer, D. W. and Hosmer, T. and {le Cessie}, S. and Lemeshow,
> S.},
> year = 1997,
> title = {A comparison of goodness-of-fit tests for the logistic
regression
> model},
> journal = Statistics in Medicine,
> volume = 16,
> pages = {965-980},
> annote = {goodness-of-fit for binary logistic model;difficulty with
> Hosmer-Lemeshow statistic being dependent on how groups are
> defined;sum of squares test;cumulative sum test;invalidity of
> naive
> test based on deviance;goodness-of-link function;simulation
> setup}
> }
>
> The new one degree of freedom test is implemented in R and S-Plus in the
> Design library's residuals.lrm function.
> ---
> Frank E Harrell Jr Prof. of Biostatistics & Statistics
> Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
> U. Virginia School of Medicine http://hesweb1.med.virginia.edu/biostat
---
Frank E Harrell Jr Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. Virginia School of Medicine http://hesweb1.med.virginia.edu/biostat
|
