Date: Thu, 29 Sep 2005 18:24:08 -0400
Reply-To: Sigurd Hermansen <HERMANS1@WESTAT.COM>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Sigurd Hermansen <HERMANS1@WESTAT.COM>
Subject: Re: Why a 95% confidence interval for c helps you.
Content-Type: text/plain; charset="us-ascii"
Nick:
In case you need a quick answer and not necessarily an authoritative
answer ...
Since (0 + 0.25%) and (1 - 0.975%) represent the tails of the
distribution of statistic c, I'd say that you have a 95% chance that
your model has a c statistic value between 0.82 and 0.99. If you are
look for a point estimate, you are missing the point of a confidence
interval. You will bring down the wrath of Cassell if you suppose that
0.905 gives you a better level of confidence than 0.82 to 0.99. We don't
want that, do we?
Consider this little analogy. You are trying to read a thermometer in
low light (limited degrees of freedom). All you can make out is that end
of the column of mercury likely ends somewhere between 82F and 99F. Say
you are 95% certain of that, but cannot actually see anything more than
that the column of mercury seems to extend close to the 82F mark and but
not much if any past the 99F mark. If you had to select a temperature,
you would likely take the midpoint between 82 and 99 to minimize the
maximum error. But why make a very likely wrong choice when the interval
expresses all that you know about the temperature.
Corollary: Don't bet on 90.5 on a roulette wheel.
Sig
-----Original Message-----
From: owner-sas-l@listserv.uga.edu [mailto:owner-sas-l@listserv.uga.edu]
On Behalf Of Nick .
Sent: Thursday, September 29, 2005 5:37 PM
To: sas-l@listserv.uga.edu
Subject: RE: Why a 95% confidence interval for c helps you.
Thanks Matthew,
I thought that we wanted the C-statistic from PROC LOGISTIC to be as
much above 0.5 (random) as possible. So, if my model has a c-value of
0.53 I amy not be too happy with it. But if it has a c-value of 0.65, I
am fine, I think. Now, you say, that if the 95% CI around c includes the
value 0.5, then the model may not be predictive when it comes to scoring
new populations. Am I right? From the code it was posted on this site
yesterday regarding the bootstrap c sttaistic, and the example that was
used there, I do not see 95% CI. What I see are the 2.5 percentile
(c-value 0.82, 50 percentile (c-value 0.92) and 97.5 percentile (c-value
0.99). How does this give me the 95% CI for c? For the example used in
the code Oliver posted, I think the c-statistic from logistic came out
to be something like 0.905. What is, then, the 95% CI around this 0.905
value? Thanks. NICK
----- Original Message -----
From: "Zack, Matthew M."
To: "Nick ."
Subject: RE: Why a 95% confidence interval for c helps you.
Date: Thu, 29 Sep 2005 15:49:02 -0400
If the 95% confidence interval for c [=the area under the ROC curve]
overlaps 0.50, then the test can be perceived as being no better than
random and thus of no use in discriminating those with the outcome from
those without the outcome.
Matthew Zack
-----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
Nick .
Sent: Thursday, September 29, 2005 3:14 PM
To: SAS-L@LISTSERV.UGA.EDU
Subject:
Thanks David. So, we do bootsrapping to know, say, at the 95% level that
the C-statistic is between, say, 0.65 and 0.71 as an example. How does
that help me any? I built a logistic model, I am happy with the results,
I get a c-statistic of 0.65, why do I now need to do more work to get a
CI on this? And why not do all the others to? I think proc logistic
gives us 3 or 4 such numbers not just c. Aside from some pesky boss, is
there a reason for doing this, is what the crux of my question is. NICK
----- Original Message -----
From: "David Neal"
To: "'Nick .'" , SAS-L@LISTSERV.UGA.EDU
Subject: RE: Bootstrapping a C statistic
Date: Thu, 29 Sep 2005 10:59:49 -0800
Nick,
In logistic regression, SAS only provides a single C statistic. If you
want a confidence interval, you will need to do something like
bootstrapping. David Neal -----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
Nick .
Sent: Thursday, September 29, 2005 10:48 AM
To: SAS-L@LISTSERV.UGA.EDU
Subject: Bootstrapping a C statistic
Hello all,
I was reading this topic about Bootstrapping a C statistic from proc
logistic modeling. My question is, why does one wish to do bootstraping
on this? I don't see the meaning of this post because I don't know what
it is we are trying to see here. NICK
--
___________________________________________________________
Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm
--
___________________________________________________________
Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm
--
___________________________________________________________
Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm
