```Date: Mon, 7 Nov 2005 16:30:45 +0100 Reply-To: Marta García-Granero Sender: "SPSSX(r) Discussion" From: Marta García-Granero Organization: Asesoría Bioestadística Subject: Re: Samplesize? In-Reply-To: <10049.1131374952@www56.gmx.net> Content-Type: text/plain; charset=ISO-8859-15 Hello Karl, KK> Thank you again for your comprehensive and well explained answer. You're wellcome. >> Anyway, I see another complication in your design: it is a factorial >> design (Factor A: system; factor B: task). Do you foresee an >> interaction between system and task? (like: system A is faster for >> task 1, but slower for task 2). In that case, you should consider the >> sample size required to detect that interaction (*), and the fact that >> the statistical analysis for system A vs system B would be more >> complicated than a simple repeated measures ANOVA. KK> My hypothesis is that system A serves the user better than system B. System KK> A uses an approach I have developed, system B does implements an existing KK> approach which serves as a baseline. I measure the performance along several KK> measures. First the time users need to perform a task, second the average KK> user ratings (these ratings are given by the user on results the system KK> returns to the user during the task), and third the amount of user actions KK> necessary to complete the task. I use three measures since they provide a KK> more comprehensive view about the effect of the new approach. KK> I am mainly interested in the difference between these two systems. I see KK> the tasks as some realistic problem for a potential user of such a system. KK> I therefore think I have two IVs in this case, system and task. However, I KK> am not particularly interested in finding out about the nature of each task KK> in relation to the system. Even if you are not interested in the nature of each task, it's another IV in your model. You have 4 groups, therefore you have 3 df that can be further split into: - 1 df: "System" main effect - 1 df: "Task" main effect - 1 df: Interaction between task and system Lack of interaction is needed to be able to talk about a general "system" main effect (is system A better than system B for any task?). If interaction is present, then you can only answer questions related to "simple effects": (a) for Task 1: is system A better than system B? (b) for Task 2: is system A better than system B? The presence of interaction might be due to the "system" effect being restricted to only one task, or to the "system" effect being much stronger for one task than for the other, or even for reverse effects (while system A is better than system B for task 1, for task 2 the effect is reversed and system B is better than system A...) KK> It is more about that the task represents the KK> general use of the two systems. I know that I have the danger, that if the KK> user tasks do not fulfill that, I might get bad results because of that and KK> not because of the system that was used. I think this is what you refer to KK> as interaction. However, I am more interested in the general outcome and the KK> average result rather than a factorial division between system and task. I KK> have counterbalanced the order in which people perform these tasks according KK> to the following table: KK> 1st Task 2nd Task 3rd Task 4th Task KK> ------------------------------------------------------------------------ KK> User 1 | Task A@SystemA Task A@ SystemB Task B@ SystemA Task B@ SystemB KK> User 2 | Task A@ SystemB Task A@ SystemA Task B@ SystemB Task B@ SystemA KK> User 3 | Task B@ SystemA Task B@ SystemB Task A@ SystemA Task A@ SystemB KK> User 4 | Task B@ SystemB Task B@ SystemA Task A@ SystemB Task A@ SystemA KK> ------------------------------------------------------------------------ KK> However, to come back to your suggestion, I would have the data to do both, KK> a factorial 2x2 full factorial ANOVA between results from 2 tasks on 2 KK> systems (within subject) or a simple, collapsed view on two systems and KK> their average performance on the tasks provided. The second, simpler KK> analysis would be a confounded version of the factorial analysis. I would be KK> served well with the simpler analysis if there would a strong effect (which KK> I expect to happen). KK> What do you think about that? Unless you have sound "a priori" reasons to believe that an interaction effect is not to be expected, then your model should take it into account. Do you know that statistical joke that says that if 2 people find a chicken, one of them eats it while the other watches, then statistics says that everyone ate half a chicken? An average of heterogeneous effects is not really representative, in my opinion. -- Best regards, Marta mailto:biostatistics@terra.es ```

Back to: Top of message | Previous page | Main SPSSX-L page