Date: Fri, 12 Aug 2011 11:46:37 -0400
Reply-To: "Viel, Kevin" <kviel@SJHA.ORG>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: "Viel, Kevin" <kviel@SJHA.ORG>
Subject: Re: Fundamental Stat Question
In-Reply-To: <29FC5B5A8EF58D49A772E9055C4F9254049D9C86@JNJUSRYGMS01.na.jnj.com>
Content-Type: text/plain; charset="us-ascii"
Certainly. Approaching the analyses with the simplest model is very wise.
Every model is wrong. They are models. I illustrate what a model is by telling my students that a square can be a model for a circle; a sphere can be a model for a cube; and a stick figure can be a model for a human.
Sometimes, replication is the only support.
It is quite frustrating to sit at the edge, like a 0.03 or 0.07, and wonder, especially about the model or sampling.
HTH,
Kevin
Kevin Viel, PhD
Senior Research Statistician
Patient Safety & Quality
International College of Robotic Surgery
Saint Joseph's Translational Research Institute
Saint Joseph's Hospital
5671 Peachtree Dunwoody Road, NE, Suite 330
Atlanta, GA 30342
(678) 843-6076: Direct Phone
(678) 843-6153: Facsimile
(404) 558-1364: Mobile
kviel@sjha.org
NOTICE: This e-mail message and all attachments transmitted with it may contain legally privileged and confidential information intended solely for the use of the addressee. In addition, this correspondence may contain private patient information protected under the federal privacy rule, 45 C.F.R. Parts 160 and 164, and applicable state law. Unauthorized use or disclosure of this information is strictly prohibited. If the reader of this message is not the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail or at the telephone number above and delete the original message and all copies and backups thereof. Thank you
> -----Original Message-----
> From: Uddin, Sharif [ATRMUS] [mailto:SUddin@its.jnj.com]
> Sent: Friday, August 12, 2011 11:33 AM
> To: Viel, Kevin
> Subject: RE: Fundamental Stat Question
>
> Kevin,
>
> The 0.03 and 0.07 were just examples - not the main point.
> The debate that I started is to choose between simple and modeling
> approach -
> It seems to me - after all the theories and philosophy, we are kind of
> back to square one.
>
> Even if you fit the best model you can - there may be a better model -
> nobody knows what the perfect model is.
>
> - SU
>
> -----Original Message-----
> From: SAS(r) Discussion [mailto:SAS-L@listserv.vt.edu] On Behalf Of
> Viel, Kevin
> Sent: Friday, August 12, 2011 9:51 AM
> To: SAS-L@LISTSERV.VT.EDU
> Subject: Re: Fundamental Stat Question
>
> > b. What's wrong, if any, if we just go with point-wise analysis
> > since this looks much better?
>
> This is not the reason to choose one technique over another. The choice
> should be based on, among other things, what model is most appropriate.
>
> In the end, you may have to deliver "bad" news. I don't see a p-value
> hovering around 0.07 as much more negative than one hovering around 0.03
> as inspiring. Its just a cut-off by convention. As such, I would want
> to see replication at these levels. I know that many people will take
> the 0.03 and celebrate, but an analyst will still ponder over it: if a
> few values were changed, would the p-value increase? If so, could a few
> values been mismeasured or incorrectly recorded? I entered research on
> the bottom of the hierarchy and, in end of the beginning of my career,
> still obtain and enter my own data, so I know that a value is not always
> exact.
>
> If a company goes headlong into further testing and developing of the
> product and it later turns out that the p-value was not so inspiring for
> the reason that the effect was not true or so strong, then they might
> have appreciated not seeing results that "look much better".
>
> I have seen sponsors happy to see "disappointing" results.
>
> I am pretty much a student, so I won't offer anything more than Steve
> said, whose posts, among several other SAS-L statistical luminaries, I
> always read and re-read, then read again.
>
>
> HTH,
>
> Kevin
>
>
>
> Kevin Viel, PhD
> Senior Research Statistician
> Patient Safety & Quality
> International College of Robotic Surgery
> Saint Joseph's Translational Research Institute
>
> Saint Joseph's Hospital
> 5671 Peachtree Dunwoody Road, NE, Suite 330
> Atlanta, GA 30342
>
> (678) 843-6076: Direct Phone
> (678) 843-6153: Facsimile
> (404) 558-1364: Mobile
> kviel@sjha.org
>
>
> > -----Original Message-----
> > From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
> > Uddin, Sharif [ATRMUS]
> > Sent: Wednesday, August 10, 2011 3:13 PM
> > To: SAS-L@LISTSERV.UGA.EDU
> > Subject: Fundamental Stat Question
> >
> > Welcoming insight from the experts on this:
> >
> >
> >
> > Let's say I am running a clinical trial with two treatment A and B and
> > collecting data on some measure at Baseline, Month 3, Month 6, and
> Month
> > 12. This is balanced design with no missing data.
> >
> >
> >
> > I want to see if there is a difference of treatment effect at Month
> 12:
> >
> >
> >
> > Approaches:
> >
> >
> >
> > 1. Point wise t-test at Month 12. Let's say at 12 month, we have
> > the difference in score and Std Err: 16.04 and 4.3262 respectively
> with
> > t and p-value: 1.78 and 0.0162.
> >
> >
> >
> > 2. Fit a Linear model with treatment and visit as factor with
> > treatment*visit interaction using proc glm (we know it is wrong
> because
> > scores are related for the same subject across the visit, hence
> > violation of fundamental assumption, but we are doing it anyway, just
> to
> > see what happened) - we get Estimate of the difference and Std Err:
> > 16.04 and 8.7784 respectively with t and p-value as 1.827 and 0.0710
> >
> >
> >
> > 3. Fit a sophisticated mixed model with proc mixed and repeated
> > statement and find the right co-variance structure etc. and we get -
> > Estimate of the difference in score and Std Err: 16.04 and 9.0009
> with
> > t and p-value: 1.78 and 0.0785 respectively.
> >
> >
> >
> >
> >
> > These are the questions:
> >
> > a. Why should I bother to use modeling - sophisticated or not?
> >
> > b. What's wrong, if any, if we just go with point-wise analysis
> > since this looks much better?
> >
> >
> >
> > Thanks,
> >
> >
> >
> > SU
> >
> >
> Confidentiality Notice:
> This e-mail, including any attachments is the
> property of Catholic Health East and is intended
> for the sole use of the intended recipient(s).
> It may contain information that is privileged and
> confidential. Any unauthorized review, use,
> disclosure, or distribution is prohibited. If you are
> not the intended recipient, please delete this message, and
> reply to the sender regarding the error in a separate email.
Confidentiality Notice:
This e-mail, including any attachments is the
property of Catholic Health East and is intended
for the sole use of the intended recipient(s).
It may contain information that is privileged and
confidential. Any unauthorized review, use,
disclosure, or distribution is prohibited. If you are
not the intended recipient, please delete this message, and
reply to the sender regarding the error in a separate email.
| 
