GME Evaluation Task Force RecommendationJuly 1, 2008
Despite their common usage and familiarity, global assessments are considered weak evidence of learning. If used along with an evaluation tool of greater validity, global assessments may be used to evaluate all 6 ACGME competencies.
Many UCSF programs use a short global assessment tool that includes the minimum language version of the 6 competencies and a 9 point scale. This format is preferred to a longer assessment form listing the full language version of each competency in multiple items. The 9-point scale is preferred to shorter scales because it allows raters to identify differences among residents who meet or exceed expectations. An additional item assessing overall performance is also recommended. A sample global assessment is provided below.
Global assessments are subject to recall bias and halo effects which threaten validity and imprecision which threatens reliability. Given these limitations the least burdensome possible tool should be used. Greater numbers of evaluators may improve inter-rater reliability but will not make up for the subjective nature of these assessments.
- Timing: after each clinical rotation; for continuity clinics at least twice annually.
- Who Performs: only individuals who directly supervised the resident during the specific timeframe being assessed.
- Format: Each competency is represented by one item rated on a 9-point scale anchored with descriptive language on either end and divided into thirds (1-3: unsatisfactory / does not meet expectations, 4-6: satisfactory / meets expectations, 7-9: outstanding / exceeds expectations). An additional item assesses overall competency. Written comments are provided listing strengths and opportunities for improvement. Programs should review progress and share feedback with residents midway and after each rotation. Mid-rotation feedback is especially important for residents who appear not to be meeting criteria for satisfactory performance.
- Scoring Criteria and Training: The distinction between satisfactory and unsatisfactory performance is an important one. If guidelines are not available for making this distinction, standard-setting can be used to improve accuracy. Standard-setting is especially important for global assessments because raters tend to fall into stricter vs. lenient categories.
- Documentation: Score summaries are generated automatically by the evaluation management system and can be accessed by residents after each rotation. Overall progress should be reviewed in writing at least twice annually.
Uses of the Data
- Comparing global evaluation scores to performance of peers or one’s own prior performance can help identify trends.
- Summative Decisions: Global evaluation scores persistently falling below expectations (3 or less on the 9 point scale) could delay or prevent a resident from advancing or graduating. Generally, however, such decisions would be based on overall assessments of progress incorporating other mearues.
- Remediation Threshold: Programs should communicate what performance on global assessments would trigger remediation. For example, residents and faculty would be informed in advance that scores < 4 are unsatisfactory and a score of 4 is marginal.
- GME Curriculum Committee
- Curriculum Resources
- ACGME Competencies & Assessment
- GME Evaluation and Assessment Recommendations - 7/1/08 - New
- Learning & Career Development
- Learning Portfolios
Graduate Medical Education
500 Parnassus Avenue, MU 250 East
San Francisco, CA 94143-0474