Date:         Fri, 4 May 2001 15:43:39 +0100
Reply-To:     John Whittington <John.W@MEDISCIENCE.CO.UK>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         John Whittington <John.W@MEDISCIENCE.CO.UK>
Subject:      Re: OT:  Aggregate preference measurement, disagreement index
Comments: To: kmself@IX.NETCOM.COM
In-Reply-To:  <E14sDYB-0004DS-00@relay1.netnames.net>
Content-Type: text/plain; charset="us-ascii"

At 17:50 24/04/01 -0700, Karsten M. Self wrote (in part):

>....This works relatively well, but it fails to take into account
>variance in votes indicating possible differences of opinion,
>controversiality, or other disagreement among moderators.  Assuming six
>moderations, all other things equal, I'd like to see:
>(1, 1, 1, 5, 5, 5) rated higher than (3, 3, 3, 3, 3, 3), though they
>both have the same mean.

Karsten, I haven't noticed any replies to this post of yours, so just a few
thoughts.....

The problem there is that you would have to create a whole set of
(essentially arbitrary) rules as regards what you wanted to be rated higher
than what.  The example you give is straightforward enough - but where, for
example, would you want, say, (2,2,2,4,4,4) to stand in relation to, say,
(1,1,4,4,4,4)? etc. etc. etc.

You could, I suppose, just generate arbitrary derived indices which had
roughly the effect you wanted.  For example, if you divided your mean by
(range +1) or (SD+1) [the +1 to avoid division by zero problems], you'd get
a figure which would be largest when there was least variability of the
scores - but whether third parties would consider such a manipulated score
as having any meaning is a different matter.

>I'd also like to indicate a stronger sense of certainty for more ratings
>than fewer.  (5) is less significant than (5, 5, 5, 5, 5), and may be
>less significant than (5, 5, 5, 5, 4).

Again, you would have to (arbitrarily) decide upon your rules.  I suppose
the most dramatic extreme would be to use the sum of all available scores,
rather than the mean - which would give figures of 5, 25 and 24 for your
three examples, respectively.  If you also divided by (range+1), as per
above, they would then become 5, 25 and 12 respectively - but that is
probably too 'violent' an adjustment for your needs - and, again, the
figures might well not be considered meaniful by third parties.

Kind Regards,


John

----------------------------------------------------------------
Dr John Whittington,       Voice:    +44 (0) 1296 730225
Mediscience Services       Fax:      +44 (0) 1296 738893
Twyford Manor, Twyford,    E-mail:   John.W@mediscience.co.uk
Buckingham  MK18 4EL, UK             mediscience@compuserve.com
----------------------------------------------------------------