GME Evaluation Task Force RecommendationJuly 1, 2008
CONFIDENTIAL RESIDENT EVALUATION OF FACULTY
Observation is the primary method by which clinical educators are rated. The SOM Clinical Educator Teaching Evaluation Form was developed at UCSF as a global assessment conducted at the end of a clinical rotation to assess the quality of medical student teaching by residents and faculty. This form consists of 19 items: 11 items on a 5-point Likert-type scale, 4 are narrative/open ended items, and 4 items are triggered only if low scores are received on certain critical items on the form. However, experience and internal studies of the form indicate it can be shortened without losing any reliability. We recommend this shorter form as our core measure of clinical teaching.Reliability and Validity
There are many instruments developed to measure clinical teaching effectiveness. Most of these instruments do tend to measure a global teaching effectiveness score, interpersonal, and clinical teaching/pedagogic practices dimensions, and have high internal consistency. Each item in the SOM Clinical Educator form includes detailed anchors illustrating each point on the 5-point scale (1=poor, 5=excellent). Due to the internal consistency of these forms, a shorter item set has adequate reliability and content validity. Our proposed core measure includes: conveyance of information, teaching enthusiasm, direction and feedback, promotion of critical thinking, treat me with respect, treat others with respect, and overall teaching effectiveness. Research has recommended that scales be tailored to learner (medical student and resident) and setting (e.g. outpatient vs. inpatient); hence, additional items may be included but should be similar in format to the other items and include clear anchors.Administration by GME programs
Frequency: It is recommended that faculty teaching be evaluated after a designated number of interactions between faculty and residents. The number of interactions is dependent on the length of the rotation and should be designated accordingly. Immediately after a clinical rotation ends would be optimal, and the minimum frequency would be twice annually.
- Who Performs: Residents.
- Scoring Criteria and Training: It is recommended that the form be publicly visible and that evaluators know the scoring/rating in advance so that they know what they are rating about their instructors. Faculty should also be aware of the criteria by which their teaching will be judged. There is no training associated with the use of this evaluation.
- Documentation: Annually as part of the program review meeting and otherwise as indicated by faculty mentorship meetings, promotion & advancement meetings, etc.
Use of Data
Each department and division will establish its own practices for review of faculty teaching evaluations. However, the program director is responsible for deciding which faculty members will teach the residents in the program. The ACGME requires an annual program review meeting at which time the aggregate data on faculty teaching is reviewed and compared with the program’s internally defined standards. Action plans are required if standards are not met.Optional Items
The GME Evaluation TF recommends that all GME programs use the SOM form as a core set to facilitate benchmarking for individual programs and the School. Review of the other evaluation tools revealed potentially useful items that programs may choose to add to the basic form. We have included these as “Item Bank Recommendations.” Remember ultimately what matters the most is the overall teaching effectiveness and comments.Item Bank Recommendations:
- During this time I personally interacted with or observed the faculty and base this evaluation on (very concrete item based on hours of contact)
- Refers resident to pertinent references for further reading
- Reviews exam findings with resident
- Discusses differential diagnosis and work-up with resident
- Reviews treatment options with resident
- Provides follow-up to the residents on interesting cases
- Takes time to stress proper surgical technique
- Discusses rational for surgical judgment
- Please rate how well this attending emphasized problem solving (i.e. thought process leading to decisions
- Monitored my stress level (it requires a grammatically different scale than what they have included – item is not a must but definitely an interesting idea to monitor burnout)
- Neurological Surgery had each of their questions categorized by ACGME competencies. This was nice – easy to track later.
- The IM Cardiology UC Consult form was nicely tailored to the specialty and type of education.
- The LEAH Fellowship form was nice and brief although I would recommend a five point scale and spell out teaching effectiveness.
The UCSF Academy of Medical Educators uses Teaching Observation (TOP) Forms to provide structured feedback by a trained peer who has observed a faculty member doing a lecture or facilitating a small group. The forms include no scales and are not scored. Their principal use is to facilitate and tailor feedback. The number of faculty receiving structured feedback on their teaching can be used as a measure of faculty development for the Annual Program Review.
- GME Curriculum Committee
- Curriculum Resources
- ACGME Competencies & Assessment
- GME Evaluation and Assessment Recommendations - 7/1/08 - New
- Learning & Career Development
- Learning Portfolios
Graduate Medical Education
500 Parnassus Avenue, MU 250 East
San Francisco, CA 94143-0474