Three Option is More Reliable for Multiple-Choice Questions Than Five Option — Here is Why
How many options should be there in a Multiple-Choice Question? Five four or three options? 3 option is better than 5 option for multiple choice questions in terms of validity and reliability.
Here is why: https://youtu.be/Zx9HA1ZhXlw
Hi and hola para amigos
Which one is better for a test that consists of multiple-choice questions, so-called MCQs? With three options or with five options per question?
If you are a student, you could think that it is better with three options because you can guess the single best option easier. If you are a teacher, you avoid using MCQs with three options and so you try to add two more options to make it less guessable. Everybody might think that the students and teachers are right because guessing among three options is easier than among five options, so MCQs with five options are better, but sorry, you are wrong. The evidence showed a million times in the past that three options is better than five options for MCQs.
The flamingo is excited to reveal why MCQs with three options is better for the reliability of the tests.
Have you ever witnessed a three-option multiple-choice question in an exam? You likely not, since it is rarely used in health professions education. The thing that we got used to seeing is MCQs with five options. But the evidence that the researchers accumulated in decades shows that three option is better. What do I mean by saying “better”? Three option produces higher reliability. So how?
Let’s jump into the research.
The authors of the article that I will present now, carried out research using three option MCQs and five option MCQs. Three option MCQs were five option MCQs previously but the authors deleted two options that are most far away from being true for each one of the questions. After the implementation of the tests with the randomized groups, they analyzed the data.
They found that three option MCQs have similar levels of reliability and item statistics. Furthermore, the students who participated in the three option MCQ exam did not perform better, both of the groups got similar points.
The difference was the response time. The three option group spent 16% less time. It reduces testing time so it allows us to add more questions into the test, a hundred and four questions instead of ninety questions in this example.
Why this is so important? If you can use more questions in the same amount of testing time, the sampling would be wider, then the structure that you cover gets bigger. It means you can create more valid tests without increasing testing time.
Perfect but what if these results are only for this particular research? How about the results of the other research on this topic? No worry. Several previous research came up with similar results consistently that show superiority of three option MCQs to five option MCQs. There is nearly a century of research supporting the use of three option ones.
Ok, but why we don’t use it? Why we are stuck on five option MCQs?
Well, it can be explained in various ways but there are some prominent reasons. The first one is the strong orthodoxy. Changing habits and traditions are difficult. The second one would be the belief that three option MCQs artificially increase the grades of the students because it is easier to guess the true option compared to five option MCQs. We witnessed this baseless belief in this research by seeing that the question writers, called subject matter experts, determined higher cut-off scores for the questions for three option MCQs. But as you see in the results, the overall score that the students got from three option test is not higher than five option one. It is not easier than five option MCQs. So the strong tradition and unfounded beliefs of item writers could lead them to stick on five option MCQs even if the evidence is obviously in favor of three option MCQs. The result of this habit is that the presence of non-functional options that the students rarely or never select. These useless options do nothing but increasing the testing time. We must get rid of our baseless beliefs and we should follow the evidence.
By the way, it reminded me of the biggest myth in medical education: Edgar Dale’s Learning Pyramid. You should avoid using the learning pyramid as well. If you want to know why it is a myth, you can watch it here: https://www.youtube.com/watch?v=X5l_ayRr9Fc
I left the link of the article that I mentioned in the video that three option is better. You can check it at the description below if you want to learn more: https://youtu.be/Zx9HA1ZhXlw
See you and adios para amigos.
And also, don’t forget the flamingo.
