A PATIENT SURVEY TO ASSESS RESIDENT PERFORMANCE

1
A PATIENT SURVEY TO ASSESS RESIDENT PERFORMANCE

• Paul Jeppsen
• Deb Simpson, PhD
• James Robinson, MD
• Department of Ophthalmology and
• Office of Educational Services

2
Need/Problem Statement

• Ophthalmology residency at MCW sought reliable
  and simple method for patients to evaluate
  residents consistent with ACGME Competencies
• There is no readily available patient survey
  designed to evaluate ophthalmology residents.

3
Background

• Reviewed literature on patient evaluation of
  residents performance
• Limited reports
• Patients effectively assess humanistic qualities
  of physicians1
• 20-50 patient surveys per resident are needed to
  yield stable ratings2

1. Matthews DA, Feinstein AR. A New Instrument
for Patients Ratings of Physician Performance in
the Hospital Setting. J Gen Intern Med.
1989414-22. 2. Swing SR. Assessing the ACGME
General Competencies General Considerations and
Assessment Methods. Acad Emer Med. 2002 Vol 9,
Number 111278-1288.
4
Methods I Survey Development

• Interviews
• Semi-structured interviews of patients in an
  outpatient ophtho clinic
• Identified attributes a good ophthalmologist
• Content analysis used to identify themes
• Survey Item
• Items drafted consistent with themes
• Reading level analysis (Grade 6)
• A priori classification of items into
  competencies
• Survey form compliant with low-vision reading
  recommendations

5
Methods II Pilot Analysis

• Piloting
• Pilot 1 22 ophtho clinic patients to
• Identify lt 10 consistently understood items I
• Identify an appropriate rating scale
• Pilot 2 27 ophtho clinic patients
• 10 selected items with in the same outpatient
  clinic
• Statistical analysis
• Reliability analysis
• Principal component analysis using the Varimax
  rotation to determine if a priori analysis was
  confirmed
• Statistical analysis SPSS Windows Version 10.1

6
(No Transcript)
7
SURVEY ITEMSMeans Standard Deviation
Mean
SD
8
Factor Analysis
9
Results

• 27 surveys in preliminary analysis
• Overall Reliability 0.66
• Adjusted Reliability 0.72 (after removing one
  question Answering your questions directly and
  completely)
• 5 different Ophtho competencies evaluated (4
  ACGME)
• A priori classification not confirmed by factor
  analysis

10
Discussion

• Strengths
• Questions derived from ophtho patients
• Items drafted using patients own words
• Short, easy to use survey (reading/vision)
• Strong reliability
• Limitations
• Non-confirmatory Factor Analysis
• Minimal statistical association with ACGME
  competencies
• Small sample size?
• Feasibility - surveys needed per resident

11
Next Steps

• Pilot 3 9 questions
• Implement in multiple ophtho residency clinics
• Transfer survey questions to scan form
• Incorporate patient survey results into 2/year
  resident review process