```Date: Thu, 29 Sep 2005 18:24:08 -0400 Reply-To: Sigurd Hermansen Sender: "SAS(r) Discussion" From: Sigurd Hermansen Subject: Re: Why a 95% confidence interval for c helps you. Comments: To: "Nick ." Content-Type: text/plain; charset="us-ascii" Nick: In case you need a quick answer and not necessarily an authoritative answer ... Since (0 + 0.25%) and (1 - 0.975%) represent the tails of the distribution of statistic c, I'd say that you have a 95% chance that your model has a c statistic value between 0.82 and 0.99. If you are look for a point estimate, you are missing the point of a confidence interval. You will bring down the wrath of Cassell if you suppose that 0.905 gives you a better level of confidence than 0.82 to 0.99. We don't want that, do we? Consider this little analogy. You are trying to read a thermometer in low light (limited degrees of freedom). All you can make out is that end of the column of mercury likely ends somewhere between 82F and 99F. Say you are 95% certain of that, but cannot actually see anything more than that the column of mercury seems to extend close to the 82F mark and but not much if any past the 99F mark. If you had to select a temperature, you would likely take the midpoint between 82 and 99 to minimize the maximum error. But why make a very likely wrong choice when the interval expresses all that you know about the temperature. Corollary: Don't bet on 90.5 on a roulette wheel. Sig -----Original Message----- From: owner-sas-l@listserv.uga.edu [mailto:owner-sas-l@listserv.uga.edu] On Behalf Of Nick . Sent: Thursday, September 29, 2005 5:37 PM To: sas-l@listserv.uga.edu Subject: RE: Why a 95% confidence interval for c helps you. Thanks Matthew, I thought that we wanted the C-statistic from PROC LOGISTIC to be as much above 0.5 (random) as possible. So, if my model has a c-value of 0.53 I amy not be too happy with it. But if it has a c-value of 0.65, I am fine, I think. Now, you say, that if the 95% CI around c includes the value 0.5, then the model may not be predictive when it comes to scoring new populations. Am I right? From the code it was posted on this site yesterday regarding the bootstrap c sttaistic, and the example that was used there, I do not see 95% CI. What I see are the 2.5 percentile (c-value 0.82, 50 percentile (c-value 0.92) and 97.5 percentile (c-value 0.99). How does this give me the 95% CI for c? For the example used in the code Oliver posted, I think the c-statistic from logistic came out to be something like 0.905. What is, then, the 95% CI around this 0.905 value? Thanks. NICK ----- Original Message ----- From: "Zack, Matthew M." To: "Nick ." Subject: RE: Why a 95% confidence interval for c helps you. Date: Thu, 29 Sep 2005 15:49:02 -0400 If the 95% confidence interval for c [=the area under the ROC curve] overlaps 0.50, then the test can be perceived as being no better than random and thus of no use in discriminating those with the outcome from those without the outcome. Matthew Zack -----Original Message----- From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of Nick . Sent: Thursday, September 29, 2005 3:14 PM To: SAS-L@LISTSERV.UGA.EDU Subject: Thanks David. So, we do bootsrapping to know, say, at the 95% level that the C-statistic is between, say, 0.65 and 0.71 as an example. How does that help me any? I built a logistic model, I am happy with the results, I get a c-statistic of 0.65, why do I now need to do more work to get a CI on this? And why not do all the others to? I think proc logistic gives us 3 or 4 such numbers not just c. Aside from some pesky boss, is there a reason for doing this, is what the crux of my question is. NICK ----- Original Message ----- From: "David Neal" To: "'Nick .'" , SAS-L@LISTSERV.UGA.EDU Subject: RE: Bootstrapping a C statistic Date: Thu, 29 Sep 2005 10:59:49 -0800 Nick, In logistic regression, SAS only provides a single C statistic. If you want a confidence interval, you will need to do something like bootstrapping. David Neal -----Original Message----- From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of Nick . Sent: Thursday, September 29, 2005 10:48 AM To: SAS-L@LISTSERV.UGA.EDU Subject: Bootstrapping a C statistic Hello all, I was reading this topic about Bootstrapping a C statistic from proc logistic modeling. My question is, why does one wish to do bootstraping on this? I don't see the meaning of this post because I don't know what it is we are trying to see here. NICK -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm ```

Back to: Top of message | Previous page | Main SAS-L page