GME Outcome Measures Data Analysis That Makes A Difference

1
GME Outcome MeasuresData Analysis That Makes A
Difference

• Diane M. Hartmann, MD
• Senior Associate Dean for Graduate Medical
  Education
• University of Rochester Medical Center
• Rochester, NY

2
TimelinesPhase 3 - 7/06-6/11

• Program Focus
• Use resident performance data as basis for
  improvement
• Begin to use external measure (patient surveys,
  clinical quality measure) to verify resident and
  program performance

• Accreditation Focus
• Evidence that programs are making data driven
  improvements
• Review the external program performance
  indicators and input from GMEC to make sure your
  program is achieving its educational objectives

3
GME Outcome Measures

• Global measures
• Institutional and program educational outcomes
• Clinical outcomes

4
Global Measure Uses

• Annual report
• Hospital administrative decision making
• Marketing (community value and philanthropy)

5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Mayo Clinic, Florida

• Research
• Teaching
• Patient Service
• General
• Basic Requirements
• Bonus Round

25 25 20 30 negative points positive
points
Acad Med 2009 84 212-210
9
Mayo Clinic, Florida
Acad Med 2009 84 212-219
10
Acad Med 2009 84 212-219
11
Additional Global Data

• Number of programs
• Number and type of trainees (LCME, DO, IMG)
• Underrepresented in medicine trainees
• Institutional match statistics
• Finance data (cap numbers)
• Graduate career types and geographic locations
• Program/Institutional Awards or special status
• GME initiatives

12
2008/2009 GME Trainees
14.4
4.2

81.4
13
Main Match Statistics
14
GME FTEs and the CAP
607
600
582
565
555
523
504
The 1996 DME base year cap for the 2006
cost report (SMH) 437.68 10.70, resident
appeal, (HH) 57.11 505.49
15
Accreditation Cycle Length70 Programs
74.3 of our programs have 4-5 year cycle lengths
42
10
13
5
3 Programs Not Included Maternal-Fetal Medicine,
Urogynecology, Vascular Surgery Residency
16
Retention of Residents and Fellows in Rochester
17
of respondents noted
39
64
88
16
40
61
78
20
37
53
73
23
27
57
65
From June 2005 - 2008 exit survey data
17
Primary Care Resident Retention
18
Institutional and Program Educational Outcomes
19
ACGME Program Directors Guide to the Common
Program Requirements

• V. Evaluation
• C. Program Evaluation and Improvement
• Common Program Requirement

• Program Evaluation and Improvement
• The program must document formal, systematic
  evaluation of the curriculum at least annually.
  The program must monitor and track each of the
  following areas
• Resident performance
• Faculty development
• Graduate performance, including performance of
  program graduates on the certification
  examination and,
• Program quality. Specifically
• Residents and faculty must have the opportunity
  to evaluate the program confidentially and in
  writing at least annually, and
• The program must use the results of residents
  assessments of the program together with other
  program evaluation results to improve the
  program.
• If deficiencies are found, the program should
  prepare a written plan of action to document
  initiatives to improve performance in the areas
  listed in section V.C.1. The action plan should
  be reviewed and approved by the teaching faculty
  and documented in meeting minutes.

20
GME Outcome MeasuresAAMC GRA Infolist
AnswersNovember 2007, October 2008

• Items listed on University of Rochester template
• Plus

Reviews ACGME citation resolution Internal
citation resolution Resources Clinical
volume Trainee / Faculty ratio PD financial
support Duty Hour Compliance
21
GME Outcome Measures
Contd

• Resident Attrition/probation
• Career choice and practice location
• Graduate survey
• Job market
• Faculty Development activities
• Academic productivity
• GMEC attendance/participation
• Evaluation Faculty, resident, program
• Completion rates
• Inservice exam scores
• Type and evidence of competency based

22
GME Outcome Data Sources

• Local collection tools
• Commercial Data Systems
• (EValue, New Innovations, etc.)
• AAMC GRA survey
• NRMP, ERAS, Charting Outcomes in the Match
• Specialty boards
• ACGME

23
Commercial Data Systems

• Procedure logging
• Evaluation summaries
• Compliance reports
• Work hour tracking
• Internal review citations

24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28

29
(No Transcript)
30
(No Transcript)
31
ACGME Data

• Data Resource Book
• Number of programs
• Average cycle length
• Program director hours
• Program director turnover
• Resident numbers and characteristics by specialty
• Institutional Review Document
• Attachment 1

32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
ACGME Data
Institution
National Normative
Program

• Future The Portfolio
• Present ADS

• Resident survey
• Case logs

36
Resident Survey
Program

• Results
• Program director comments regarding non-compliant
  responses
• Specialty normative data
• Program vs. specialty non-compliant responses
• Program vs. institutional non-compliant responses

Institution

• Results
• Normative (only 2004 2006 available)

37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
ADS

• Program specific national reports
• Family Medicine
• Case logs

49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
GME Indicators

• Clinical Outcomes

56
Educational Outcomes
Clinical Outcomes

• Patient Surveys
• Hospital/department quality data
• On going measures after specific
  educational events
• National or specialty standardized resources
• Practice guidelines
• Pay for performance
• IHI initiatives
• ABMS maintenance of certificate