A member of a professional group that I belong to sent me the following. His university was trying to determine some facts about testing. They wanted to know, among other things, whether there was more of a chance that students would get the right answer on a multiple choice test or on a straight question and answer test. They also wanted to know about the efficacy of having "None of the above" or "All of the above" as answer choices.
They gave multiple tests to students registered in spring term classes. One test they gave to 11 classes had "None of the above" as the correct answer to all the questions. The percentage of students choosing this answer? 39% A different test they gave had "All of the above" as the answer to all of the questions. The percentage of students choosing this answer? 14% On a different test they omitted the "all" and "none" choices, but all the correct answers were the first choice in the list of possible answers. Percentage of students getting the right answer? 29%.
When students were told that the test they were going to take was not going to be multiple choice but a question and written answer format they did better, given the same subject matter. The percentage of those answering the questions correctly? 83%. One reason the instructors believe for the higher marks on the written test is that students know they won't have a crutch during the exam so they actually study for the test. On a multiple choice test they figure they can play the percentages and get some questions correct just by guessing.
We've always believed that if students had the right answer in front of them, on a multiple choice test, they would do better on the test. Clearly there are other factors involved.
Instructors in this university are now being advised to make their tests, whenever possible, tests where students have to write in the answer rather than pick from a list.
Just as a little aside. One of the instructors maintained that students taking multiple choice tests don't always read the questions closely. To prove his point he included the following question on a biology exam:
Question: Sex was invented in
a. 1389 bce
b. 1157 ce
c. 1392 ce
d. 1501 ce
e. None of the above
The percentage of students who picked "None of the above"? 28%. No one left the question unanswered.
12 comments:
My multiple choice tests used to only have two answers to choose from. I was giving a current events quiz to junior high students and decided to throw in a free question that everyone could get right. The question was who was elected president in 1868. The two answer choices were Ulysses S. Grant and George W. Bush.31% picked Bush. When I asked the class why pick Bush the answers really gave me a headache. One student said that it wasn't today's president Bush he meant--it was his father. Another said that he had never heard of Grant so it had to be Bush. Another said that he had misread the question and thought I asked 1968 not 1868.
Back when I was in school a teacher gave us a multiple choice test about all the bridges in New York City. I can't forget the question he asked about one of them. It was "Which is the correct spelling of the following Bridge(Kosciousko Bridge)and he spelled it five different ways. Needless to say, only about 20% of the class got the right answer.
I had an economics teacher once who made guessing on multiple choice tests dangerous. He would subtract points for the wrong answer, not just add points for the right answer. And each question had one of the answers that was so impossible to be right that he would subtract double the points if you chose it. Was safer just to leave out a question then to guess.
In the old days some of the actuarial exams were multiple choice. You'd get a complicated mathematical question involving time series or numerical analysis, and these would be the choices:
a. 5.11
b. 5.12
c. 5.13
d. 5.14
e. 5.15
And yes, there was a 1/4 point deduction for getting it wrong, so guessing would work against you.
It might be interesting to see how the study was controlled. For example, were the questions for the multiple choice and open answer questions the same? Or do the teachers give different types of questions on multiple choice exams?
The result you report is provocative, but without more detail it is hard to evaluate.
Mike,
I emailed the professor and he answered that yes, the written questions and the multiple choice questions were the same, but with this proviso: they were given to different sections. This makes it a bit harder to correlate the results because 1)different sections may have covered the material differently and 2)different instructors may have placed emphasis in different places. Giving the same test to the same section would be of no use, since there would be residual memory of one test that would interfere with the other test.
I never test my students with multiple choice tests. It is much harder for me, but I know that what you mentioned is true. They actually study when they know there is no crutch. Unless they don't which happens too and when that happens, they do lousy on the test and everyone kind of knows that they're messed up this time. MC allows them to either not study or to study a little and still somehow do ok. Even the good students may rely on the choices and not study as much if they have a busy night. Down with the MC.
Your post made my day with the "Sex was invented in" question. :)
I think that multiple choice tests at the college level are actually harder than non multiple choice tests. Being a current grad student, I know a bit about this. In my case, with multiple choice tests it's not that I've studied less, it's that sometimes there might be more than one answer that seems right, and then deciding between them is hard. (This is especially true when you have the questions that are a. b. c. d. and then you have e. as something like "a and c" and f "a, b, and c but not d" etc. And you might know that c is a correct answer but you could still get the question wrong.) I'd much rather be given terms to define or essays to write. Then I can just show that I know the material.
There's also the whole issue of the teacher's side. A multiple choice exam is far easier to correct than any other, allowing the teacher to check a class's exams with efficiency, even using a computer or having another person correct it -- without him /her being proficient in the material.
If a teacher has a more efficient way of correcting exams, s/he can spend more time on other professional tasks, for example, curriculum development and research, or have more time for family personal hobbies, etc. Since most teachers, especially schoolteachers, are constantly overworked, this is an important consideration.
Then the question becomes: When we consider the good of both teacher and student, which type of exam offers more benefits?
(BTW, I'm not necessarily advocating multiple choice here. I'm a proponent of written answers, especially essays, or in the case of math-oriented material, showing your work from start to finish). An exception would be a professional competency exam in math or the exact sciences (such as your example) when the bottom line matters, and not the reasoning that got you to it).
Enjoyed the post.
ALN
The other benefit to multiple choice examinations is fairness. There can be no bias in grading a multiple choice exam, while there can be lots with other kinds of exams.
Mark
Mark,
You are mostly but not 100% correct on the fairness. Every time the SAT administers the test at least one question ends up getting "thrown out" because it is proven that given the way the question was worded there were two possible answers, or that none of the answers was actually correct.
You are mostly but not 100% correct on the fairness. Every time the SAT administers the test at least one question ends up getting "thrown out" because it is proven that given the way the question was worded there were two possible answers, or that none of the answers was actually correct.
I should have defined what I meant by "fairness". I meant fairness from one pupil to another, not fairness of the exam overall.
Mark
Post a Comment