|Date: ||Sun, 30 Apr 2006 14:18:56 -0700|
|Reply-To: ||David L Cassell <davidlcassell@MSN.COM>|
|Sender: ||"SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>|
|From: ||David L Cassell <davidlcassell@MSN.COM>|
|Subject: ||Re: How to get two-sided p value of permutation test|
|Content-Type: ||text/plain; format=flowed|
>I have two questions about permutation tests (Fisher's randomization
>I used permutation test to analyze the difference between two unbalanced
>groups. One group has 3 observations, the other has 5. The procedure is as
>1. calculate the sum of group 1 as observed value--obs;
>2. recompute the sum of group 1 for each permutation (totally 56
>3. Of these 56 combinations, I calculate the number N of group 1 sum
>than or equal to obs(original sum).
>4. get one sided p-value using: N/56.
>My questions are:
>1.Is it the right procedure for unbalanced comparison?
>2.How to get two-sided p-value? especially when one-sided p-value>0.5?
 No. The 'unbalanced' part is not the problem. The problems
are the sample size and the method used.
There's no reason why 'unbalanced' means you cannot use ordinary statistical
approaches. But there's no way around the fact that you have ridiculously
groups. You are not going to get any sort of power (probability of finding
significant difference when there really is one) that you can live with.
Now why do you have 8 observations? How did the study work? How did you
categorize into two groups? A permutation test makes assumptions about
the nature of the data and the meta-data. Your data may not meet those
specs. I would like you to write back to SAS-L and explain more about your
study. If you cannot afford more than 8 observations, then perhaps you
should give up on the 'statistical inference' part of the study.
And finally, I do not see that you are using an appropriate approach. What
are you trying to test? Why is the sum of group 1 meaningful? How can
the mere sum be attributed as meaning something about your study if you
do not compare against group 2, and you do not think about the variability
involved within and across groups? It seems from way over here that your
method -before you evne get to a randomization test - is flawed.
 I don't think there's any point in talking about 1-sided and 2-sided
until you get part #1 fixed first.
I think you would benefit from writing back to SAS-L and discussing your
problem in a lot more depth. What are you data? Where do they come
from? What do they mean? What do you want to achieve with these
data? What is your hypothesis that you want to test? (Write the null
and alternative hypotheses as explicitly as possible, putting them into
mathematical formulas if you can.) What will you do once you make a
'significant difference'/'no significant difference' evaluation?
David L. Cassell
3115 NW Norwood Pl.
Corvallis OR 97330
Express yourself instantly with MSN Messenger! Download today - it's FREE!