GME Evaluation Task Force RecommendationJuly 1, 2008
PROFESSIONALISM, INTERPERSONAL AND COMMUNICATION SKILLS
360 evaluations include observations by a variety of individuals from the multiple contexts where professionalism and communication skills are demonstrated. Ratings by self, peers, colleagues and faculty are generally collected using different tools than ratings by patients. The 360 method captures information on most of the competencies listed by the ACGME under “Professionalism” and “Interpersonal & Communication Skills and is highly recommended for every program’ assessment system.
Reliability and Validity:
The optimal number and frequency 360 assessments is uncertain. A feasible minimum to achieve inter-rater reliability = .80 may be 5 non-clinical and 6 clinical raters each on 2 occasions (1).
Preliminary evidence of construct validity shows modest growth in 360 scores comparing senior vs. junior residents with a magnitude similar to the growth in other competencies including critical self-reflection skills (2).
Content validity exists to the extent that the survey items actually assess the professionalism and communication skills they are intended to measure. The 9-item evaluation includes communication (patients/families, nursing/allied staff), respect (patients, nursing/allied staff), compassion, reliability, honesty/integrity, responsibility, and advocacy.
N.B. Faculty need to complete the 360 evaluation separately from their global ratings at the end of the rotation so the data can be summarized and reported accurately.
Format: 9 items sample aspects of professionalism and communication skills. Each item is scored on 9-point scales with 1-3 = unsatisfactory, 4-6 = meets expectations, and 7-9=excellent.
Uses of the Data
Summarizing the data: Score averages, ranges, and comparative data for the PGY year are provided as part of the data report residents review before their semi-annual meetings. The other data sources for the 360 assessment include the relevant items (e.g., respect) from the residents’ clinical educator evaluations and patient surveys. These are also summarized as means and ranges compared with averages for the PGY year.
Formative Uses: Assessment results support development of professionalism and communication skills by individual residents, identify trends in performance across PGY groups and spur possible improvements to the curriculum in these two competencies
Summative Decisions and Remediation: Scores in the unsatisfactory (scores 1-3) would trigger remediation and a low satisfactory score (4) would trigger suggestions for improvement
A systematic approach is recommended to maximize the use of the assessments and facilitate data management. An example follows in the Appendix.
1. Murphy DJ, Bruce DA, Mercer SW, Eva KW. The reliability of workplace-based assessment in postgraduate medical education and training: a national evaluation in general practice in the United Kingdom. Adv in Health Sci Educ 2008 DOI 10.1007/s10459-008-9104-8.
2. Learman LA, Autry AM, O'Sullivan P. Reliability and validity of reflection exercises for obstetrics and gynecology residents. Am J Obstet Gynecol 2008;198(4):461.e1-8; discussion 461.e8-10.
- GME Curriculum Committee
- Curriculum Resources
- ACGME Competencies & Assessment
- GME Evaluation and Assessment Recommendations - 7/1/08 - New
- Learning & Career Development
- Learning Portfolios
Graduate Medical Education
500 Parnassus Avenue, MU 250 East
San Francisco, CA 94143-0474