Resident Evaluation

1
Resident Evaluation

• Clinical Competency
• Laura Kezar, MD
• Associate Professor
• Physical Medicine and Rehabilitation

2
Roles of Oversight Agencies

• ACGME
• Residents in training
• RRC accreditation process
• ABMS
• Initial board certification and maintenance of
  certification Groups perceived the need for
  developing description of competent physician
• Response to external forces
• Errors in Medicine Report, etc.

3
What is Clinical Competency?

• Critical knowledge and ability to perform
  defining acts of our profession.
• Responsibility of program director and teaching
  faculty to verify that residents possess the
  skills, knowledge, and attitudes necessary to
  competently practice patient care.

4
How Do We Evaluate It?

• Formative Evaluations day to day
• Feedback given to residents on a regular basis to
  help them improve performance
• Daily verbal interaction
• Correction of errors on HPs, notes
• 360 evaluations
• Peer evaluations
• Reflections on lecture, workshop, committees
• Do not go into the permanent record.
• Portfolios often to show work and document
  improvement over time.

5
Daily Feedback

• I see
• give specific information
• I feel
• pleased, disappointed, frustrated
• I think
• this was unprofessional
• you did a great job
• I want
• you to study this tonight and we will re-evaluate
  it tomorrow
• You should put this in your portfolio

6
How Do We Evaluate It?

• Summative Evaluations
• Formal Global Rotation Evaluations
• Semiannual Global Evaluations
• Evaluation at completion of residency to
  specialty board

7
Six Core Competencies1. Patient Care

• Patient care that is compassionate, appropriate,
  and effective for the treatment of health
  problems and the promotion of health.

8
Six Core Competencies2. Medical Knowledge

• Medical knowledge about established and evolving
  biomedical, clinical, and cognate
  (epidemiological and social-behavioral) sciences
  and the application of this knowledge to patient
  care

9
Six Core Competencies3. Practice-based Learning
and Improvement

• Involves investigation and evaluation of
  physicians own patient care, appraisal and
  assimilation of scientific evidence, and
  improvements in patient care

10
Six Core Competencies4. Interpersonal and
Communication Skills

• Interpersonal and communication skills that
  result in effective information exchange and
  teaming with patients, families, other health
  care professionals

11
Six Core Competencies5. Professionalism

• Professionalism, as manifested through a
  commitment to carrying out professional
  responsibilities, adherence to ethical
  principles, and sensitivity to a diverse patient
  population.

12
Six Core Competencies6. Systems-based Practice

• Systems-based practice
• manifested by actions that demonstrate an
  awareness of and responsiveness to the larger
  context and system of health care
• ability to effectively call on system resources
  to provide care that is optimal value.

13
Typical Evaluation MethodsSee the ACGME Toolbox

• Teaching physician observations
• Written exams
• Learning objectives and evaluations
• Medical record audits
• 360 Evals

14
Evaluation Methods

• Computer simulated patient encounters
• Clinical Evaluation Exercise (CEX)
• Mini CEX
• Standardized patients (Objective Structured
  Clinical Examination (OSCE)

15
Learning Objectives and Evaluations

• Supervision of residents by clinically competent
  physicians - the ideal assessment site but
  difficult to document.
• Objectives delineate what resident should be able
  to do after completion of rotation.
• Learning objectives should span all domains of
  learning - cognitive, affective, psychomotor

16
Learning Objectives

• Reasonable, attainable, and measurable
• Should be specific to
• Clinical setting - inpatient or outpatient
• Technical skills needed - ability to perform the
  physical exam, injections, EMGs, surgical
  procedures

17
Global Faculty Evaluations

• Directly reflect objectives
• Be specific to clinical situations
• Advantages prolonged observation, direct
  assessment over time
• Disadvantages one shot phenomenon, time
  consuming, decreasing time with residents due to
  financial constraints, feedback required,
  difficult to standardize

18
Additional Evaluation Methods

• Medical record audit
• Checklist looking for documentation of specific
  information
• Judgment about decision-making
• Can be done longitudinally
• Requires substantial faculty time
• Records do not always reflect what happens in
  patient encounters
• Improves documentation but not health care

19
Computer Simulated Clinical Encounter

• Computerized patient management problems
• At best, a partial representation of a complete
  patient
• Must capture key features of interaction
• Benefits exposure to core of disorders,
  consistency, detailed feedback
• Disadvantages cost, time

20
OSCE

• Benefits consistently display tasks, clinical
  task scaled to skills needed to be assessed,
  predetermined grading scale
• Disadvantages LABOR INTENSIVE, costly

21
Clinical Evaluation Exercise

• Observing a trainee obtaining and performing a
  comprehensive history and physical examination on
  a new patient
• Allowing time to write up case, impression,
  management plans
• Presentation of case with discussion of findings,
  impressions, recommendations
• Immediate feedback to trainee

22
Mini-Clinical Evaluation Exercise

• Observation of focused HP
• Discussion of diagnosis and Rx plan
• Eval of performance and feedback
• Total time 20-35 minutes
• More accurately reflects clinical practice

23
Mini-CEX

• 4 areas evaluated
• history
• physical exam
• clinical judgment
• humanistic qualities
• Rating scale used for evaluation
• 4-10 needed for reliability - more needed for
  borderline performers

24
Mini-CEX

• Coupled with oral exam
• Could be done at end of each rotation or at end
  of academic year
• Real patients or trained standardized patients