Skip Navigation

GME Evaluation Task Force Recommendation

Back to GME Evaluation
and Assessment Portal
July 1, 2008

PRACTIC BASED LEARNING



PBLI: CRITICAL APPRAISAL SKILLS
Overview

Critical appraisal skills are essential for finding, evaluating and translating evidence from the literature into clinical practice.

Core Measure for UCSF GME

The Critical Appraisal Exercise is to be used to evaluate residents’ ability to answer a focused clinical question by searching the medical literature, summarizing their findings, evaluate the limitations of the evidence, and discuss how their practice will change based on the evidence gathered. It is intended to be used in journal clubs, other critical appraisal presentations, and after ad hoc searches of the literature to guide clinical management.

Templates for critically appraising articles of specific types can be found at Oxford’s Centre for Evidence-based Medicine: http://www.cebm.net/index.aspx?o=1157 However, these worksheets are directed toward evaluating the quality of a study and not the critical appraisal abilities of the learner.

Reliability and Validity

The number of evaluators and assessment opportunities to assure adequate reliability are unknown. Content validity can be established to the extent that key components of critical appraisal are included in the checklist rating form.

Administration
Timing: Depends on importance of developing critical appraisal skills. Programs should assign a certain number that need to be completed satisfactorily in a specified time frame.
Who Performs: Designated experts in critical appraisal. Generally, evaluators would be faculty although experts in library science could evaluate the learners’ search strategies. Learners can also assess their own performance and compare it to the faculty rating.
Format: an exercise with instructions completed by the resident followed by a checklist of 6 components completed by the faculty member
Scoring Criteria and Training: No faculty training is required. The checklist components are judged as Yes/No.
Documentation: At minimum, twice annually as part of semi-annual review meetings.

Uses of the Evaluation Data
Formative Feedback: concurrent or same day written evaluation and debriefing
Tracking Resident Learning: individualized learning plans discussed with a mentor and skill development over time
Assess Program Effectiveness: aggregating data across all residents or by PGY group
Summative Uses: depending on how important critical appraisal skills are to the training program, poor performance could trigger remediation and affect promotion or progress decisions



PBLI: Critical Appraisal Skills
PBLI: Critical Care Appraisal Skills
Core Measure for UCSF GME

 

E*Value Forms
(pending)



Back to GME Evaluation and Assessment Portal

Curriculum Home