Abstract
Background
Videotaped recordings of simulation-based performance may allow learners the opportunity
to review, evaluate, and reflect upon their own performance.
Objectives
To determine the accuracy of resident performance self-assessment after a simulation-based
encounter; compare low- and high-scoring residents’ abilities to evaluate their performance;
and determine if video-assisted performance review improves self-assessment accuracy.
Methods
Emergency Medicine residents participated in a videotaped simulation-based assessment.
Residents evaluated their performance immediately after completing simulated cases,
and after reviewing the session’s video. Self-ratings were compared to the faculty
observers, and scores were divided based on the median.
Results
Seventeen residents participated, providing 270 self-ratings before, and 269 after,
video review. Before video review, residents accurately graded their performance in
73.7% of the items. High- and low-scoring residents accurately self-assessed 83.9%
and 62.2% of items, respectively. The odds of a high scorer accurately rating their
own performance were 3.2 times that of a low scorer (95% confidence interval [CI]
1.9–5.2, p
<0.001). After video review, resident self-assessments were accurate for 73.6% of the
items. High scorers were accurate in their post-video self-assessment in 83.3% of
the items, vs. 62.4% for low scorers. After video review, the odds of a high scorer
accurately self-rating their performance were 3.0 times that of a low scorer (95%
CI 2.1–4.1, p
<0.001).
Conclusions
Residents’ abilities to self-assess vary, and performance quality may influence self-assessment.
Video review did not significantly increase self-assessment accuracy. Improving self-assessment
skills may assist residents in identifying practice gaps, thereby allowing them to
focus their energy toward filling that gap.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Emergency MedicineAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
Accreditation Council for Graduate Medical Education (ACGME). ACGME Outcomes Project. Available at: http://www.acgme.org/outcome/comp/compMin.asp. Accessed January 27, 2009.
American Board of Medical Specialties. MOC competencies and criteria. Available at: http://www.abms.org/Maintenance_of_Certification/MOC_competencies.aspx. Accessed December 2, 2010.
- Accuracy of physician self-assessment compared with observed measures of competence: a systematic review.JAMA. 2006; 296: 1094-1102
- Flawed self-assessment: implications for health, education, and the workplace.Psychol Sci Public Interest. 2004; 5: 69-106
- Psychometric characteristics of simulation-based assessment in anaesthesia and accuracy of self-assessed scores.Anaesthesia. 2005; 60: 245-250
- Self-assessment in simulation-based surgical skills training.Am J Surg. 2003; 185: 319-322
- First-year medical students’ assessment of their own communication skills: a video-based, open-ended approach.Patient Educ Couns. 2007; 68: 161-166
- "I’ll never play professional football" and other fallacies of self-assessment.J Contin Educ Health Prof. 2008; 28: 14-19
- Patient assessment and management examination: lack of correlation between faculty assessment and resident self-assessment.Am J Surg. 2008; 195: 16-19
- How can I know what I don’t know? Poor self assessment in a well-defined domain.Adv Health Sci Educ Theory Pract. 2004; 9: 211-224
- A longitudinal study of self-assessment accuracy.Med Educ. 2003; 37: 645-649
- Resident self-assessment of operative performance.Am J Surg. 2003; 185: 521-524
- Value of debriefing during simulated crisis management: oral versus video-assisted oral feedback.Anesthesiology. 2006; 105: 279-285
- Difficulties in recognizing one’s own incompetence: novice physicians who are unskilled and unaware of it.Acad Med. 2001; 76: S87-S89
Article info
Publication history
Published online: August 11, 2011
Accepted:
May 25,
2011
Received in revised form:
December 17,
2010
Received:
August 31,
2010
Identification
Copyright
© 2011 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
