ChatGPT for Generating Cases to Teach & Learn: Results of a Randomized Controlled Experiment

Today, we’ll look into a pioneering study about the use of ChatGPT in medical education as it has been conducted in the first months of ChatGPT. Our research question is: Can ChatGPT generate clinical cases for teaching comparable to those written by human experts?

In our medical school, there is a training to teach evidence-based medicine. In the training, medical students receive ill-defined clinical vignettes that direct them to search medical literature, make critical appraisal and identify the best evidence, and apply this knowledge to solve the case. This process aims to improve their literature search, critical thinking and clinical reasoning skills, which is essential for their future clinical practice. They also present their journey in the ocean of knowledge as a five-minute video record that has been designed to improve their digital and presentation skills.

It is a fruitful training for our students but it is always effortful for the faculty to write new cases each year for each student because the number of medical students in a year is around four hundred. After ChatGPT has been released at the end of November 2022, we aimed to benefit from ChatGPT’s skills in generating clinical cases. In January 2023, we used this prompt to generate clinical cases for this training. You will see, [in the video], an example of clinical vignette written by a human expert, the prompt for that, and the case generated by ChatGPT.

Following a review process carried out by human experts, we randomly provided the clinical vignettes to students. The intervention group received ChatGPT-generated cases without any changes, while the control group received human-written cases focusing the same clinical problems as those in the ChatGPT-generated cases.

The assessment was carried out through a survey where students evaluated various aspects of the cases. Criteria included understandability, relevance, the inclusion of critical clinical reasoning elements, and the overall educational value. We found that there is no significant difference between the groups. The effect sizes were small.

It can be interpreted as ChatGPT is able to generate clinical vignettes comparable to those written by experts. Even if this evaluation was based on self-perceived value of the cases in the eyes of students, it shows the potential of ChatGPT in generating clinical vignettes that can be valuable and useful for teaching purposes.

In another study, we used ChatGPT to generate case-based multiple-choice questions and it showed promising results. We also have a custom GPT to generate multiple-choice questions, and a very concise video on how to use it. Do not miss the opportunity to ease your tasks with these sources.

See you and adios para amigos.

And also, don’t forget the flamingo.

Follow the Flamingo on X (Twitter): https://twitter.com/MedEdFlamingo