Skip Navigation

GME Evaluation Task Force Recommendation

Back to GME Evaluation
and Assessment Portal
July 1, 2008

PROFESSIONALISM, INTERPERSONAL AND COMMUNICATION SKILLS




Patient Care Surveys: Professionalism

Reliability and Validity:

The optimal number of patient surveys is uncertain. As initially used in an ABIM continuing professional development context, 20 surveys were recommended. A feasibility study with Canadian internal medicine residents showed 12 outpatient surveys to be associated with poor reliability (0.56) (1).

Data using the Consultation and Relational Empathy (CARE) survey, which is used for physician accreditation in Scotland, suggest that more than 40 patients may be necessary for good inter-rater reliability (.80) with 25 patients providing adequate reliability (.70) (2).

Content validity exists to the extent that a measure actually assesses the communication and professionalism skills it is intended to measure. The ABIM’s 10 items sample multiple aspects of doctor-patient communication (greeting, listening, establishing rapport, explaining, inviting participation in decision-making) and professionalism (truthfulness, respect, sensitivity to linguistic barriers). The physician characteristics evaluated using the CARE survey are similar to those assessed using the ABIM survey. Although the CARE survey has some advantages in how it describes the characteristics being evaluated, we recommend the ABIM survey because it is so widely used and studied in the United States.

Administration
Timing: Patient satisfaction surveys may be obtained regularly as a quality measure. Otherwise, administration twice annually is the minimum recommended for assessment of competency.
Who Performs: Patients under the direct care of the resident.
Format: The ABIM Patient Survey includes 10 complex items. Both sample similar aspects of communication and professionalism skills.
Scoring Criteria and Training: The ABIM survey uses scales ranging from 1-5 (poor, fair, good, very good, and excellent), uses simple language, and relies upon the subjective experience of individual patients. Patients receive a general orientation but no specific instructions regarding the scoring criteria. Although a 9-point scale is used for the 360’s Health Care Team and Self-Evaluations, the patient surveys use only 5 points because finer distinctions are challenging for patients. This 5- vs. 9-point scale difference is important to note when scores are summarized and discussed with residents.
Documentation: Summaries with comparative data are made available for review at the semi-annual meetings.

Uses of the Data
  • Summarizing the data: Score averages, ranges, and comparative data for the PGY year are provided as part of the data report residents review before their semi-annual meetings. The other data sources for the 360 assessment include the relevant items (e.g., respect) from the residents’ clinical educator evaluations and patient surveys. These are also summarized as means and ranges compared with averages for the PGY year.
  • Formative Uses: There are many useful ways to use the assessment results to support development of professionalism and communication skills by individual residents, identify trends in performance across PGY groups and spur possible improvements to the curriculum.
  • Summative Decisions and Remediation: It is important to set criteria that would trigger a plan of improvement to explain the criteria to the residents before they are assessed.

  • Workflow Procedures

    A systematic approach is recommended to maximize the use of the assessments and facilitate data management. An example follows in the Appendix.

    References

    1. Tamblyn R, Benaroya S, Snell L, McLeod P, Schnarch B, Abrahamowicz M. The feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents. J Gen Intern Med 1994;9(3):146-52.

    2. Murphy DJ, Bruce DA, Mercer SW, Eva KW. The reliability of workplace-based assessment in postgraduate medical education and training: a national evaluation in general practice in the United Kingdom. Adv in Health Sci Educ 2008 DOI 10.1007/s10459-008-9104-8.


    Patient Care Assessment Forms
    Patient Care Surveys - Professionalism, Communication and Interpersonal Skills
     

    E*Value Forms
    (pending)



    Back to GME Evaluation and Assessment Portal

    Curriculum Home