Does direct observation influence the quality of workplace-based assessment documentation?
Competency-based medical education makes observation crucial in workplace-based assessment. Observation has two types: Direct and indirect. We need to observe trainees directly or indirectly to document evidence on their competence levels. Which one does provide more reliable data? If you think direct observation is more valuable than indirect observation in terms of reliability, you may be wrong.
A study on this topic conducted in Canada will get you surprised!
The flamingos is ready to present this study.
Hi and hola para amigos!
First of all, if you don’t know what competency-based medical education and entrustable professional activities are, I recommend you to click the cards above at the right corner to watch these beginner-level videos.
Let’s get back to the observation case. Direct observation is valuable. It fosters supervisor–trainee trust and promotes effective feedback. Despite various benefits, it is challenging: The amount of time required to perform is high, and it may decrease learner autonomy. However, EPA assessment needs to get quality data to document entrustment decisions. At this point, indirect observation comes into play. However, it is not known whether the quality of EPA assessment is affected by the observation type. Does indirect observation decrease the quality?
Well, in an emergency department that uses EPA-based assessment in Canada, this has been studied. Seventeen trainees’ EPA assessments were evaluated. There were more than one thousand documented EPAs but many of them were excluded because of some reasons such as not knowing what the observation type is. Finally, 244 documented EPAs have been evaluated using the instrument called The Quality of Assessment for Learning. It is a short instrument “designed to evaluate the quality of written comments associated with a single clinical performance score”. Four blinded raters independently scored the quality using this instrument.
Let’s jump into the results. There was no significant difference between the mean quality scores in the direct and indirect observation groups. It seems indirect observation is as valuable as direct observation.
This doesn’t mean that let’s get rid of direct observations and let’s only observe indirectly. Both types of observations provide data from different aspects of performance. Competence committees need both. This study notices us to value indirect observation as much as we value direct observation. However, it is worth to note that this study has been conducted in a single institution within a single training program, it may not be generalizable to other programs.
If you want to read more, you can find the link to the article at the description below this video: https://youtu.be/r7FhcV_2i14
See you and adios para amigos.
And also, don’t forget the flamingo.
Follow the Flamingo on Twitter: https://twitter.com/MedEdFlamingo
