Twelve Tips for Introducing Very Short Answer Questions (VSAQs) into Your Curriculum

The use of single best answer questions in medical education has raised concerns. Clinical uncertainty is inherent in medicine, and there is often no single best answer. Patients don’t typically come into the clinic and ask, “Out of these five diagnoses, which one do you think is most likely?”.

Assessing students with artificial situations where patients present a list of diagnoses is inauthentic. It is crucial for assessment methods to reflect the realities of clinical practice and prepare students accordingly. Additionally, studies have shown that students perform better on single best answer questions compared to very short answer questions. However, this may be due to cueing and answer recognition behaviors, raising concerns about the validity of single best answer questions assessments.

Very short answer questions have gained strong support as an effective assessment tool. They involve a clinical scenario and a free-text answer, typically limited to one to five words. Constructed-response questions like Very short answer questions align with the constructivist theory of learning, promoting higher-order cognitive processes. Very short answer questions demonstrate higher reliability, discrimination, and authenticity compared to single best answer questions. Despite testing the same knowledge, students tend to score higher in single best answer questions. Very short answer questions have greater validity in assessing the ability to arrive at a correct answer without cueing or guessing. They also encourage more authentic clinical reasoning strategies, such as analytical reasoning and acknowledging uncertainty.

This video presents twelve tips provided by researchers from the UK for implementing very short answer questions in medical curricula.

Tip 1

Identify areas where very short answer questions are better suited than single best answer questions for assessing applied knowledge. Some areas, such as arterial blood gas analysis and prescribing, are not well assessed by single best answer questions due to the difficulty of creating plausible distractors or the potential for easy recognition of the correct answer. Very short answer questions allow students to generate their own answers and have been effective in assessing prescribing skills, detecting errors, and providing valuable feedback for improvement.

Tip 2

When introducing a new assessment, it is important to consider the required resources. Converting Single best answer questions into very short answer questions by removing answer options is a straightforward approach. However, single best answer questions that involve choosing the best option from a list may not work well as very short answer questions.

Tip 3

Use authentic scenarios when constructing very short answer questions. Unlike single best answer questions, very short answer questions can utilize authentic clinical cases without the need for contrived alterations to create distractors. Very short answer questions are particularly effective in assessing clinical reasoning in uncertain situations, such as diagnosis, investigation, or management. The constructed response format of very short answer questions allows for the acknowledgement and management of clinical uncertainty. However, it is important to carefully write Very short answer questions to ensure that the uncertainty stems from the clinical problem rather than the question construction itself.

Tip 4

When creating very short answer questions, it is important to have specific and clear lead-ins that test a particular area of knowledge, avoiding ambiguity about what the question is asking. A vague lead-in can generate uncertainty about the type of answer expected. For example, “what is the diagnosis?,” may not be specific enough. Specific lead-ins help ensure that any uncertainty in the very short answer question is related to the clinical problem and not the question construction. In proteinuria, there can be ambiguity about whether a general diagnosis, which is nephrotic syndrome, or the specific underlying cause, which is amyloidosis, is being asked for. To provide clarity, a more specific lead-in question could be used, such as “What diagnosis is most likely to be confirmed with renal biopsy?”

Tip 5

Very short answer questions should have a defined range of correct answers, considering that there may be multiple appropriate responses in clinical management scenarios. Unlike single best answer questions, very short answer questions allow for a broader range of acceptable answers. However, to streamline the marking process, it is important to guide students towards a specific range of correct answers using the clinical vignette and lead-in. This ensures feasibility and efficiency in marking. Having shorter answers and using dropdown menus for certain aspects of the response can help standardize the marking process. The flexibility of very short answer questions allows for consideration of viable alternative answers that were not initially considered.

Tip 6

Consider your marking strategy for very short answer questions. Online examination management software allows for semi-automated marking, grouping identical responses and using automated matching algorithms. Machine marking compares student answers to preapproved acceptable answers, with identical responses automatically marked as correct. Non-exact matches are reviewed by examiners to determine correctness. Factors to consider include the tolerance for misspellings, allowance of partial credit, accuracy of human marking, and post-hoc review processes to manage question and candidate performance.

Tip 7

Give standard setters access to acceptable answers for very short answer questions. Providing answers helps standard setters establish a valid standard and appropriate pass mark. Studies show that access to acceptable answers leads to lower pass marks for very short answer question papers compared to standard setting without answers. Standard setters gain a better understanding of very short answer question difficulty and appreciate the increased challenge compared to single best answer questions. Detailed training should also be provided to standard setters to ensure familiarity with very short answer question format and the standard setting method used (for example, Angoff or Ebel).

Tip 8

Prepare your students for very short answer questions by making the assessment process transparent and familiarizing them with the method. Important areas to cover include explaining what a very short answer question is and its added value, guiding students on how to approach questions analytically, providing instructions on answering within word limits and specificity requirements, and familiarizing them with any software or technology used. Offering practice questions and providing feedback helps students become comfortable with very short answer questions, especially before summative assessments.

Tip 9

Utilize very short answer question answers to provide rich feedback to students. Very short answer question responses can highlight errors in judgment that may not be identified in Single best answer questions. The detailed feedback allows for specific guidance on correct reasoning and helps identify potential errors in clinical practice. Feedback can be provided individually through the exam platform or as class-wide feedback addressing common errors.

Tip 10

Very short answer question answers provide valuable feedback to teachers by identifying cognitive processes, student errors, and misconceptions. This feedback helps improve understanding of students’ clinical knowledge, areas of difficulty, and cognitive approaches. Teachers can use this feedback to shape and enhance clinical teaching, address blind spots in the curriculum, and promote analytical reasoning. Very short answer questions offer insights that are not apparent in single best answer questions, highlighting significant cognitive errors and providing opportunities for improvement.

Tip 11

Student responses to Very short answer questions can be used as plausible distractors for constructing single best answer questions. By analyzing students’ real distractor responses, educators can identify plausible incorrect options that can enhance the quality of single best answer questions. The quality of incorrect options in Single best answer questions affects the accuracy and difficulty of the question, and utilizing students’ very short answer question responses can improve the overall quality of single best answer questions.

Tip 12

Very short answer questions can be used to enhance assessment for learning. Evidence shows that very short answer questions are perceived by students as a better representation of real-world clinical practice and improve their preparation for clinical practice compared to single best answer questions. When used in formative team-based learning sessions, very short answer questions promote group discussions and activate prior knowledge, enriching the learning experience. Very short answer questions allow for a wider range of attempts and multiple plausible answers, stimulating in-depth discussions among students. Using Very short answer questions in a formative setting helps identify and address misconceptions early, benefiting student learning across different teaching modalities.

Considering these 12 tips, you may conduct a more authentic assessment of applied knowledge with improved reliability, discrimination, and authenticity. Therefore, the tips may lead to enhance students’ preparation for clinical practice and their ability to provide high-quality patient care.

If you want to read more, you can find the link to the article at the description below the video: https://www.youtube.com/watch?v=EJ-Pm5LjfLQ

See you and adios para amigos.

And also… Don’t forget the flamingo.

Follow the Flamingo on Twitter: https://twitter.com/MedEdFlamingo