```Date: Thu, 20 Jun 1996 15:00:00 EST Reply-To: "Alderton, David" Sender: "SAS(r) Discussion" From: "Alderton, David" Subject: Re: Scoring Standardized Exams > Rob Morrison wrote: > > > > What is done in the statistical community to deal with this? Having 46 > missing > > answers seems different from 46 wrong answers yet it seems they should both > > get 0/46. Traditional test scores are a function of the number correct and the number wrong, blanks are ignored. Psychometricians (the people who study the mathematical properties of tests) have long used (since the 1930s) the following scoring methodology: Score = #correct - #wrong*(1/[k-1]), with k=# forced choice alternatives Take a 10 question 5 alternative test. If a person BLINDLY guesses, she or he should get 2 questions correct, i.e., 1/5*10 (a 1/5 chance on each item). This person's score will be zero using the above formula, since: 0 = 2 - 8*1/[5-1] = 2 - 8 * 1/4 = 2 - 8 * .25 = 2 - 2 This scoring can obviously lead to negative scores, however, most large norm based test scores are transformed to another metric (some function of a cumulative probability metric) so that negative scores are either 1st percentile, 0 "points", or some positive minimum value. > In the Norwegian Mathematical Olympiad - http://www.math.uio.no/~abelk - we > give multiple choice questions in the first two rounds. Here, there are five > alternatives, one of which is correct. We score the answers: > > Wrong: 0 points > Correct: 5 points > Blank: 1 point Einar Andreas Rodland described the scoring of the Norwegian Mathematical Olympiad which is simply a variant of the traditional correction of guessing formulation above. Take a 10 item 5 alternative test again. If a person leaves everything blank she or he gets 10 points. If a person blindly guesses on all 10 items, she or he will get 2 items correct at 5 points each and thus get a score of 10 points which is equivalent to the person who left them all blank, which is the goal; however, getting 10 points for doing nothing is as odd as getting negative points for guessing badly. The key theoretical formulation is that you want someone who knows nothing but blindly guesses to get the same score as someone who knows nothing and "honestly" answers by not responding! Two additional points related to this. For those of use who have taken large normative tests that use these corrections for guessing we have all been "told" about it in the directions when instructed NOT to guess unless you can eliminate one or more alternatives. Wonder why? Take a 5 alternative question and look at what happens to the guessing probability based on the number of alternatives you can eliminate... Correct by Eliminate Guessing 0 1/5=.20 -- you break even 1 1/4=.25 -- you win on average 2 1/3=.33 3 1/2=.50 4 1/1=1.0 (know the right answer even if you don't!) Last point. Empirical studies show that the traditional correction for guessing penalty (#wrong * 1/[k-1]) is too conservative ... so GUESS! Regards, David. David L. Alderton, Ph.D. Centers for Disease Control and Prevention Division of HIV/AIDS Surveillance Branch Atlanta, GA -- DLA5@cidHIV1.em.CDC.gov ```

Back to: Top of message | Previous page | Main SAS-L page