Are open-book exams dangerous?

The physicians are allowed to access all kind of information source during clinical practice in case of forgetting some specific information. The real clinical practice allows, even encourages, them to access. However, we do not allow medical students to access information resources.

But why?

Why do we still use closed-book examination instead of open-book ones? Is it because the students would get artificially higher scores in open-book exams?

Even if our common sense prevents us to use open-book exams due to this reason, a recent randomized controlled study found really important results contrary to our common sense. Open-book exams may not be as dangerous as we think.

The flamingo presents the results of the study: https://youtu.be/escIONCQCnM

A group of researchers from New Zealand designed a study to find out how open-book exams affect students, their scores, and the cut-off scores for pass/fail decisions determined by experts.

The exams consisted of 80 multiple-choice questions (MCQs). Each MCQ was based on a case scenario and focused on diagnosis, management, etiology or underlying mechanisms.

Year-5 medical students in a six-year medical school participated in the study. Students were randomised into either open-resource or closed conditions, and into either question pool for the first half of the exam. After a short break, students then took the exam questions from the other pool under the other conditions.

The students in the open-resource condition arm were allowed to access books, files or online resources. Potential resources could have included the student learning management system, online library, databases and resources. It was not allowed to communicate with people.

Ok, we captured the study design. How about the results? Let’s start with the cut-off scores for pass/fail determined by experts. The cut-off scores in open-resource exam was significantly higher than closed-resource one. It seems the experts have thought that open-book provides advantage to the students so the pass mark should be higher. However, the mean scores of the students were not different between open-resource and closed-resource conditions. So it led pass rate to be lower. 74% for closed-resource, 31% for open resource. It was not fair. Apart from that, the time spent in open-resource condition was significantly higher than closed-resource condition.

The most important thing would be that the standard setters thought open-resource condition would make the examination easier but it has minimally affected student performance. So, worrying about the open-book exams would be too easy may be unfounded. This unfounded belief could lead experts to estimate the pass-marks higher than the true value. Their belief could stem from the questions that are created only to assess factual recall. This kind of factual recall exam definitely be easier for open-book condition. So, we can conclude that if your questions assess lower levels of cognitive skills, then the best option is still closed-book exams. But the exam has an expected quality to assess higher levels, no need to worry about open-book exams.

Well, another thing is that open-resource exams could be prone to assess more important skills. The candidates who know what they need to look up, where to look it up, and how to analyse the robustness of the information and then synthesise that information may achieve higher scores since they would spend less time to finish. Those who doesn’t have these skills will go through problems such as being not able to finish the exam on time. Higher amount of time spent in open-resource condition would be a sign for that.

All in all, it seems that open-book exams deserve to be used instead of closed-book ones.

By the way, you can consider to take a look at the videos that is relevant to assessment. How many options should be there in a multiple-choice question? If you think the answer is five, you should watch this video to be surprised. The other one is that “Does webcam based proctoring prevent cheating in online exams?”. Researchers from UK compared the results of online and traditional exams. There are interesting results.

I will leave the links of these videos at the description of the video: https://youtu.be/escIONCQCnM

If you want to read more about the article that I presented in the current video, you can find the link of that at the description.

See you and adios para amigos.

And also, don’t forget the flamingo.