GME Evaluation Task Force RecommendationJuly 1, 2008
Surgical Skills, Procedural Skills, Specific Communication Skills
Unlike the shared general competencies, Patient Care objectives are defined by each RRC. Thus, no specific tool can be recommended across GME programs. Instead, we offer principles to guide the development or selection of tools. Checklists define the discrete tasks that comprise the overall skill being assessed. Scales evaluate procedural skills that transcend individual tasks, such as knowledge of anatomy, instrument handling, flow of operation, etc. Richard Reznick and colleagues at the University of Toronto developed scales for the Observed Structured Assessment of Technical Skills, or O-SATS (1,2).
Focused assessments for skills are used primarily to assess patient care by determining if a psychomotor skill has been acquired. It is possible to combine some communication items if this is the only time a resident interacts with patients, but it is not the primary use of this assessment. It is also possible to develop a focused assessment of specific communication skill tasks, such as an informed consent discussion or specific counseling following a practice guideline.
Skill checklists primarily have content validity. The items for a specific checklist may have come from the literature where someone has decided the checklist has content validity. Alternatively, if designing a checklist within the program, having those who are “expert” in the skill review and approve the checklist would serve as a level of validity. Reliability exists in several dimensions: the ability of different assessors to come to the same decision (inter-rater reliability) and the internal consistency of the checklist items (do they “fit” together). However, if checklists are used to identify when someone has mastered the skill, the internal consistency is not relevant (since everyone should get 100% eventually). Therefore, the best reliability evidence would be consensus of faculty concerning the decision that the resident is competent in that skill.
A systematic approach is recommended to maximize the use of the focused assessments and facilitate data management. A sample workflow document for focused assessment of surgical skills follows.
1. Winckel CP, Reznick RK, Cohen R, Taylor B. Reliability and construct validity of a structured technical skills assessment form. Am J Surg1994;167(4):423-7.
2. Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative "bench station" examination. Am J Surg 1997;173(3):226-30.
- GME Curriculum Committee
- Curriculum Resources
- ACGME Competencies & Assessment
- GME Evaluation and Assessment Recommendations - 7/1/08 - New
- Learning & Career Development
- Learning Portfolios
Graduate Medical Education
500 Parnassus Avenue, MU 250 East
San Francisco, CA 94143-0474