Module 4: Evaluation and Feedback

1
Module 4Evaluation and Feedback

• Module Created by
• Charlene M. Dewey, M.D., M.Ed., FACP
• Associate Professor of Medicine
• Teri L. Turner, M.D., M.Ed., MPH
• Associate Professor of Medicine
• Joan Friedland, M.D., MPH
• Associate Professor of Medicine
• Web Page and Module Formatting by
• Maria Victoria Tejada-Simon, Ph.D.
• Assistant Professor of Molecular Physiology and
  Biophysics

Residents as Teachers Leaders
2
Welcome

• This is Module 4 Evaluation and Feedback. You
  should have completed a pretest for this module.
  Did you complete the honesty pledge question?
• In that pledge you agreed to take the pretest
  first, then read the module and then take the
  post-test after reading the module. If you did
  not complete the pretest, please exit the module
  now and complete it then return to the module.
  Your honesty is appreciated.

Click here to continue.
Exit
3
Welcome

• By now you are familiar with the general format
  of the RATL modules. This module also consists
  of a pre-assessment, content of the topic and a
  post-assessment tool.
• There are 148 total slides many are games and
  interactive activities. Overall it should take
  about 45-60 minutes to complete this module.

4
Welcome

• However, at the completion of Module 4 we will
  ask you to complete the Teacher Identity Scale
  for the second time.
• Please remember to provide feedback on the module
  content and format by completing a module 4
  evaluation form found on the main RATL web page.

5
Introduction

• So what are you going to say about these teaching
  modules? First, you need to compare them to
  something that is known to be a good example or a
  gold standard. Then you must decide if they were
  as good as the gold standard and why. Then you
  will have to organize your thoughts to provide us
  with feedback that can help us improve the
  modules for the future. Lastly, you must convey
  those thoughts by writing an evaluation.

6
Introduction

• If you do all that and then never send us your
  comments, we will continue to think these modules
  are helpful in teaching residents how to teach.
  If they are not, and you dont let us know, then
  this is an educational tragedy!
• You do not want to spend a week, a month or a
  semester teaching learners and never let them
  know if they made the grade. That would truly be
  tragic!

7
Introduction

• What we are talking about? This example covers
  assessments, feedback and evaluations. All are
  key concepts in teaching around a learners
  performance. It is part of the teacher identity
  and if done correctly, you will elevate learners
  to new levels of knowledge, skill and attitudes.
  

8
Introduction

• Module 4 Evaluation and Feedback focuses on
  accurately assessing a learner's level of
  ability, providing effective feedback and writing
  appropriate evaluations.
• Your role as a teacher will be enhanced by
  adapting appropriate assessment, evaluation and
  feedback principles.

9
Goals

• Our goals for Module 4 are that participants will
  review
• General concepts of learner assessment
• ACGME core competencies
• How to provide effective feedback
• Documenting written evaluations of learners

10
Objectives

• By the time you complete this module, you should
  be able to
• Explain the difference between formative and
  summative feedback and evaluation.
• List the 6 ACGME core competencies.
• List at least three characteristics of effective
  feedback.
• Listen to examples of feedback.
• Discuss the importance of accurate evaluations.

11
Agenda

• In module 4 we will cover the following
• Definitions of assessment, evaluation and
  feedback
• The 6 ACGME core competencies
• Assessment of learners
• The R.I.M.E model
• Providing feedback to learners
• Writing learner evaluations

12
Evaluation and Feedback

• Imagine yourself as a chef in a top fancy
  restaurant and you are cooking for a food critic
  for the national newspaper in your areaa very
  important person.
• As you are preparing your dish, you periodically
  assess the taste of your soup and make changes by
  adding more ingredients based on your assessment.

13
Evaluation and Feedback

• You also ask a few colleagues in the kitchen to
  assess your soup and you make adjustments based
  on their objective opinion.
• What has just happened? The first important step
  here is to gain an understanding of how the soup
  is progressing as compared to your standard soup
  this is assessment.

14
Evaluation and Feedback

• The second important step is seeking an
  assessment from others in order to improve the
  soup this is asking for feedback.
• After hearing the comments of other chefs, you
  contemplate for a while if and how to make your
  adjustments this is reflection.
• Now your soup is ready to be served and judged by
  the food critic.

15
Evaluation and Feedback

• You then observe the critic eating your soup.
  You try to make assumptions on how they liked it
  based on their facial expressions as they tasted
  the soup. If you ask their opinion you are
  seeking feedback.
• When they assess your soup based on all the soups
  they have tasted and you await their final
  judgment in tomorrows newspaper - this is their
  written evaluation.

16
Evaluation and Feedback

• This process of assessment, providing feedback,
  reflection and evaluation are the key ingredients
  to being an effective teacher.
• Feedback should be something sought after and
  used to enhance learner knowledge, attitudes and
  skills. It should be provided during and after
  every rotation.

17
Definitions

• Assessment The comparison of two things usually
  the item or behavior being assessed compared to
  certain standards or criteria.
• Ex When you assess your learner you are
  observing their behaviors, attitudes, knowledge
  and skills and comparing them to the standard
  medical student. These characteristics must be
  directly observed before you can pass judgment.

18
Definitions

• Evaluation The process of making qualitative
  judgments or value statements about a persons
  work or performance in comparison to a standard
  or expectations.
• Ex When you deem a student excellent you are
  making a final judgment on their ability
  (knowledge, attitudes, behaviors and skills) as
  having exceeded the standard.

19
Definitions

• Feedback A verbal summary of a learners
  performance that is based directly on an
  observation of the learner. It is given in an
  attempt to improve behavior.
• Ex When you tell the learner they are
  performing above average because their knowledge
  on most common medical conditions are above the
  expected levels of knowledge for the average
  learner.

20
Definitions

• What we evaluate in learners
• Knowledge
• Attitudes
• Skills
• Behaviors
• Ex Basic knowledge of common medical conditions,
  empathy, appreciation for patients, ability to
  interpret laboratories, ability to perform
  accurate physicals or basic procedures and their
  daily professional behaviors.

21
Evaluation and Feedback

• There are two (2) basic purposes of assessment,
  evaluation and feedback
• 1) To assess learners.
• 2) To provide a means for changing behavior to
  prevent repeated errors and reinforce good
  behaviors-verbally or written.

22
Evaluation and Feedback

• There are two (2) major types of evaluation and
  feedback
• Formative
• Summative
• What are the four (4) major foci of evaluation
  and feedback?

Click here for the answer.
23
Evaluation and Feedback

• There are four (4) major foci of evaluation and
  feedback
• Knowledge
• Attitude
• Skill
• Behavior/performance - professionalism

Knowledge
Attitude
Behaviors
Skill
24
Definitions

• Formative means during the teaching encounter
  or on the spot.
• Summative means at the end of a teaching
  rotation or training period.
• Both evaluation and feedback can be either
  formative or summative.

25
Question

• Explain the difference between feedback and
  evaluation.

(Click here for the answer)
26
Answer

• Evaluation The process of making qualitative
  judgments or value statements about a persons
  work or performance based on standards or
  expectations.
• Making a judgment on how learners do
• Feedback A verbal summary of a learners
  performance that is based directly on an
  observation of their performance. It is given in
  an attempt to improve behavior.
• Telling learners what they did well
• Telling learners how to improve
• Telling learners want not to do in the future

27
Evaluation and Feedback

• First lets focus on evaluation and feedback
  during day to day teaching activities or
  formative evaluation and feedback.
• We will then return to summative evaluation and
  feedback latter in the module.

Great! I need some feedback on my suturing
skills.
Lets discuss how that last procedure just went.
28
Evaluation and Feedback

• Recall the purpose of learner objectives. In
  module 2 you learned that students have learning
  objectives for each rotation. These are the
  skills, knowledge, attitudes and behaviors that
  they should acquire by the end of each rotation.
  Thus, the expected outcomes.
• Outcomes are the end result. Lets now introduce
  the concept of competencies.

29
Competencies

• Competencies are the outcomes and measures the
  student should achieve. We use competencies to
  assess how well a student has done how
  competent they are for a given area of
  knowledge, skill, attitude or behavior.
• Both residents and students have competencies
  based on the ACGME topics.

30
Competencies

• The 6 ACGME core competencies are
• Professionalism
• Medical knowledge
• Patient care
• Systems-based practice
• Interpersonal communication
• Practice based learning and improvement
• Leadership was added as a 7th core competency
  graduation goal for students at the Baylor
  College of Medicine.

31
Competencies

• The student objectives and core competency
  graduation goals can be found on the RATL
  resource page.
• We suggest you familiarize yourself with the
  competencies because these are the standards that
  your teachers are using to assess your
  performance and how the school assesses their
  graduating students.

32
Competencies

• Here is a brief glimpse into the ACGME core
  competencies document. You may also print them
  from the RATL resource page or click below to go
  to the ACGME web page for a shortened version.

33
Competencies

• As a reminder, the 6 core competencies are
• Professionalism (PRO)
• Medical knowledge (MK)
• Patient care (PC)
• Systems-based practice (SBP)
• Interpersonal communication (ICS)
• Practice based learning and improvement (PBLI)
• Use these abbreviations for the following
  matching game.

34
Question

• Match the examples of ACGME competencies with the
  correct core competency area PC, PRO, SBP,
  PBLI,ICS, MK

1. Communicate effectively and demonstrate caring and respectful behaviors
2. Counsel and educate patients and their families
3. Know and apply the basic and clinically supportive science
4. Facilitate the learning of students and other healthcare professionals
5. Apply knowledge of study designs and statistical methods
6. Use effective listening skills
7. Work effectively as a member or leader of a health care team
8. Demonstrate compassion, respect and integrity
9. Practice cost-effective care and resource allocation
10. Know how to partner with health care managers
11. Know how types of medical practice and delivery systems differ
35
Answers

• How did you do? The answers are below.

1. Communicate effectively and demonstrate caring and respectful behaviors PC
2. Counsel and educate patients and their families PC
3. Know and apply the basic and clinically supportive science MK
4. Facilitate the learning of students and other healthcare professionals PBLI
5. Apply knowledge of study designs and statistical methods PBLI
6. Use effective listening skills ICS
7. Work effectively as a member or leader of a health care team ICS
8. Demonstrate compassion, respect and integrity PRO
9. Practice cost-effective care and resource allocation SBP
10. Know how to partner with health care managers SBP
11. Know how types of medical practice and delivery systems differ SBP
Patient care (PC) Medical knowledge (MK) Professionalism (PRO)
Interpersonal communication skills (ICS) Practice-based learning improvement (PBLI) Systems-based-practice (SBP)
36
Assessment

• First lets focus on assessing learners. Keep in
  mind, you can also use this process for patient
  performance or adherence to lifestyle
  modifications.
• As the teacher, you will have to assess how your
  student or patient is doing and then give them
  the feedback to help them improve.

37
Assessment

• Assessing students requires a basic understanding
  and familiarity with their course learning
  objectives and the descriptions of competencies.
• Competency goals may be set by national or
  sub-specialty organizations.
• Hemoglobin A1C levels less than 6.5 mg/dl
• 15 IV placements
• 24 endotracheal intubations
• Duty hours must be limited to 80 hours per week,
  averaged over a four-week period, inclusive of
  all in-house call activities.

38
Assessment

• You can also use your student evaluation forms to
  compare your students performance with the given
  scale helping to determine those with a high
  level of performance and those with a low level
  of performance.
• For example, if the objective is to accurately
  measure the blood pressure on several different
  occasions, then you must observe the student
  taking the blood pressure.

39
Assessment

• Then you must compare their results to your
  standard measurements or an experts
  measurements.
• How well they measured the blood pressure
  compared to the standard is their assessment. Are
  they competent in this skill given the standards
  or expectations for their level? If no, then you
  made a judgment or evaluation.

40
Assessment

• If you have directly observed your students
  performance or knowledge and they need
  improvement, then you must provide them with the
  feedback to guide them toward improvement and
  future success.

41
Assessment

• In our example of the chef, they sought out
  others opinions in order to improve the soup.
  They must have felt comfortable seeking those
  opinions and they must have valued those
  individuals they asked for opinions.
• As the teacher, you must also make learners feel
  comfortable asking for your assessment and
  feedback and you must provide information that is
  valued.

42
R.I.M.E.

• RIME is a model of assessment of learners in
  medicine created by Louis Pangaro, M.D.. This
  model allows residents to assess student learners
  against a standard set of behaviors that is
  easy to remember. RIME is a mnemonic that stands
  for

R Reporter
I Interpreter
M Manager
E Educator
43
Reporter

• This learner
• Performs a history physical efficiently
• Gathers communicates data
• Recognizes normal vs abnormal
• Identifies and labels new problems
• Has a sense of responsibility

44
Interpreter

• This learner is an
• Effective presenter of information
• Can start to assimilate data and study results
• Can justify and reason on basic level
• Can interpret literature for basic cases

45
Manager

• This learner can
• Deciding when action needs to be taken
• Propose select among at least 3 diagnostic
  therapeutic options
• Takes into account particular patients
  circumstances preferences
• Requires more confidence,
  knowledge judgment

46
Educator

• This learner
• Has mastery of the basics
• Can identify knowledge gaps and develops a plan
  for addressing these gaps
• Shares new knowledge with others
• Defines important questions to research
• Takes control
• Has drive, insight, skill,
• maturity and confidence

47
R.I.M.E.

• Can you think of student behaviors for each level
  of the RIME model?
• This is an easy way of thinking about the
  beginner, intermediate and advanced level
  student. Now lets think about how we would let
  the student know what level they have reached.
  Its time for feedback!

48
Feedback

• How you provide feedback can make a significant
  difference in how your learner receives it and
  responds to it. Remember, the goal is to improve
  their performance compared to the gold-standard.
• To emphasize how feedback can affect a student,
  patient or other learner, try playing the
  feedback game on the next slide.

49
Feedback

• The feedback game is designed to help you better
  understand feedback and some important points
  about providing feedback.
• Click here to play.
• Click here to skip the feedback game.

Feedback Game 1
50
Feedback Game Version 1

• Guess a number between 1 and 10. Use your mouse
  to click on your guess. You have only three (3)
  chances to guess the correct number. Then click
  below to give up.
• I give up. Proceed with module.

1
2
3
4
5
6
7
8
9
10
51
Feedback Game Version 1

• You selected choice 1. You are totally wrong!
• Back to game

52
Feedback Game Version 1

• You selected choice 2. Not! Try again!

53
Feedback Game Version 1

• You selected choice 3. Next time you try, bring
  your brain!

54
Feedback Game Version 1

• Choice 4? You are not even close!

55
Feedback Game Version 1

• Why would you selected choice 5? Cant you do
  better than that?

56
Feedback Game Version 1

• You selected choice 6. Heres a dime to call
  your mother maybe shell think you are a doctor!

57
Feedback Game Version 1

• You selected choice 7. That was just plain
  stupid!

58
Feedback Game Version 1

• Choice 8, I have worked with babies with more
  brains than you!

59
Feedback Game Version 1

• You selected choice 9. You are so far off base
  you are out of the field!

60
Feedback Game Version 1

• You selected choice 10. How pathetic! Try again!

61
Feedback Game Version 1

• Based on the feedback you received during this
  game
• Did you get the correct number?
• Was the feedback provided helpful in guiding you
  to the right number?
• Did you value the feedback you received?

62
Feedback Game Version 1

• After playing this game how do you feel?
• Which of these adjectives most closely reflects
  how you felt after your three attempts?

63
Feedback Game Version 1

• This type of feedback is considered rude,
  unprofessional and unnecessary. It can be
  harmful and wont help improve the learners
  ability.
• This type of feedback is considered a right of
  passage by some. This is part of the hidden
  curriculum and is a problem if you continue it.
  Remember, be part of the solution, not part of
  the problem. Avoid demeaning students,
  colleagues and staff.

McIlwrick, Nair Montgomery, Academ Psych 2006.
64
Feedback Game Version 2

• Since that was not an example of effective
  feedback, lets try again. Remove yourself from
  the last game and start again by guessing a
  number between 1 and 10. You have only three (3)
  chances to guess the correct number.
• I give up, continue with module.

1
2
3
4
5
6
7
8
9
10
65
Feedback Game Version 2

• You selected choice 1. Great try! Try again.

66
Feedback Game Version 2

• Choice 2 - that was really good.

67
Feedback Game Version 2

• You selected choice 3. Great job!

68
Feedback Game Version 2

• Choice 4 - I like the way you guess. Want to try
  again?

69
Feedback Game Version 2

• You selected choice 5. Excellent job. Keep up
  the good work.

70
Feedback Game Version 2

• You selected choice 6. Well done.

71
Feedback Game Version 2

• You selected choice 7. Nice try. Keep it up.

72
Feedback Game Version 2

• You selected choice 8. Bravo! You get the judos
  for the day!

73
Feedback Game Version 2

• You selected choice 9. Way to step up to the
  plate.

74
Feedback Game Version 2

• You selected choice 10. I think you did a great
  job.

75
Feedback Game Version 2

• Based on the feedback you received during this
  game
• Did you get the correct number?
• Was the feedback provided helpful in guiding you
  to the right number?
• Did you value the feedback you received?

76
Feedback Game Version 2

• How did this go around make you feel?
• Select one or more from the list for this go
  around.

77
Feedback Game Version 2

• You have just experienced what some might
  consider feedback. But this feedback was filled
  with empty praise.
• Empty praise may make someone feel good but it
  does not lead to improved behavior because the
  learner doesnt know what they did right. How
  often do you say to your students, You did a
  great job, and thats all you say?

78
Feedback

• In the feedback game, effective feedback would
  provide specific guidance to get you to the
  desired outcome (the correct number or better
  tasting soup).
• Effective feedback requires specific comments
  geared toward improvement in behavior and is
  based on direct observations.

79
Feedback Game Version 3

• Now lets try the game the right way with
  effective and appropriate feedback. Guess a
  number between 1 and 10. Click on your choice.
  You have only three (3) chances to guess the
  correct number.
• Proceed with module.

1
2
3
4
5
6
7
8
9
10
80
Feedback Game Version 3

• You selected choice 1. This was a good guess but
  you need to increase your guess by 6.

81
Feedback Game Version 3

• You selected choice 2. This was a good guess but
  if you increase your guess by 5 you will be where
  you need to be.

82
Feedback Game Version 3

• You selected choice 3. This is a good guess.
  Increase your number by 4 and you will be at your
  target.

83
Feedback Game Version 3

• You selected choice 4. This is a good guess.
  Add to that number 3 and you will be at the
  target number.

84
Feedback Game Version 3

• You selected choice 5. This is an excellent
  guess and only two away from where you need to
  be. Increase your next guess by 2 to hit the
  target.

85
Feedback Game Version 3

• You selected choice 6. This is an excellent
  guess and very close to the target number.
  Increase this guess by one and you are there!

86
Feedback Game Version 3

• You selected choice 7. This is perfect. This is
  the exact number you should be at.
  Congratulations and remember my favorite number
  is seven, so any time I ask you to guess a
  number, that would be the best choice!

87
Feedback Game Version 3

• You selected choice 8. This is an excellent
  guess. You are close to your target and you
  should reduce your next guess by one to get to
  the target number.

88
Feedback Game Version 3

• You selected choice 9. This is an excellent
  guess and you are off by only two. Reduce you
  next guess by two to reach your target.

89
Feedback Game Version 3

• You selected choice 10. This is a good guess.
  You need to reduce your guess by three in order
  to get to your target.

90
Feedback Game Version 3

• Now that was more like it! With appropriate
  feedback you should have gotten to the correct
  answer in only two attempts.
• You should have also learned how closely you
  guessed the first time and what you needed to do
  to get to the correct answer the next time. Now
  that is effective feedback!

91
Feedback Game Version 3

• Now which ones reflect how you felt in the last
  version of the feedback game.

92
Question

• Select the statement that reflects the moral of
  the story. (click with your mouse to see the
  correct response for each.)
• Feedback is very useful.
• Feedback can help guide the learner to improve.
• Inappropriate feedback can be harmful.

93
Answer

• All of the abovefeedback is the key to helping
  your students and patients get to where they need
  to be.
• Remember, feedback does not need to be malignant.
  It should improve a learners ability.
  Inappropriate feedback (negative, rude or full of
  empty praise without information) is useless and
  can be harmful to learners and patients.

94
Feedback

• Think about a time you received feedback that was
  helpful and resulted in an improved behavior.
• What made that feedback effective? Did you value
  the opinion of the person giving the feedback?
  Did you feel comfortable asking for feedback?

Go back to play the feedback game
95
Feedback

• Characteristics of effective feedback include
• It is timely (linked closely to the observed
  behavior)
• It is specific
• It is linked to goals/objectives/competencies
• It suggests correct performance
• It is focused and limited (not too much at one
  time)
• It is balanced (good and bad)
• Overall feedback should be
• Expected
• Provided often
• Programmed to plan for follow-up

96
Feedback

• Just a point on being timely.
• When providing feedback Timing Is Everything!
  The most important and most valued feedback is
  that feedback provide immediately
  during/following a task or activity! It should
  not occur at random times or when too much time
  has passed. Keep it tied to the behavior!

97
Feedback

• This is especially true if the behavior is
  negative and needs immediate changing. If a
  student forgets to include the social history
  during a presentation, inform them of it
  immediately and then provide a summary of their
  entire performance (as feedback) after their full
  presentation is complete.

98
Feedback

• Feedback, just like other aspects of teaching, is
  best if planned. At the beginning of each month,
  provide students with the ground rules on when
  you will provide both formative and summative
  feedback.
• Let them know that you will provide feedback
  often and that they should feel comfortable
  seeking feedback from you.

99
Feedback

• As a reminder
• Always focus on improving behaviors!
• Only focus on 1-2 major points at a time.
• Involve the learner in the process.
• Be specific, focused and limited.
• Create a safe learning environment and set the
  ground rules.

100
Feedback

• Heres Dr. Deweys 5-step plan in giving
  feedback
• Observe and assess first
• Identify behaviors or focus
• Reinforce positive behaviors
• Discuss changes/improvments
• Summarize and create improvement plan

101
Feedback in Five Steps

• Observe assess
• Identify focus

• I watched you as you prepared your soup today
  compared to the guidelines on soup making I have
  some feedback for you.
• I want to discuss the onion concentration and the
  balance of the chicken and stock.

102
Feedback in Five Steps

• Reinforce positive behaviors

• Overall, your chicken soups tastes like chicken
  soup. The balance of chicken and stock is just
  right and I suggest you continue with this
  balance.

103
Feedback in Five Steps

• Discuss changes
• (Suggest specific changes in behavior. Focus on
  only 1 or 2 items.)

• Regarding the onion flavor. It is a bit strong
  for the average chicken soup eater. May I
  suggest you reduce it by about a third. What do
  you think about that? I also noticed that the
  soup simmered at a low temperature and required
  more cooking time. Turn your flame up to a
  medium fire. Do you think this will reduce your
  cooking time?

104
Feedback in Five Steps

• Summarize and create plan

• So, overall your soup is tasty and well balanced
  with chicken and stock. Reduce the onion flavor
  by 1/3 and turn up your simmering temperature to
  medium. We will make another attempt at making
  chicken soup next week. Please read this article
  on the health effects of chicken soup before the
  next session.

105
Feedback in Five Steps

• Now lets hear some example of formative feedback
  with real medical examples.
• The next slide has a few options you can chose by
  clicking on the corresponding sound icon. You
  may listen to as many as you like.

106
Examples of Effective Formative Feedback

• Example 1 (Pediatrics Physical Examination)
• Example 2 (Medicine- History Presentation)
• Example 3 (Surgery Suturing skills)
• Example 4 (Psychiatry Patient counseling)
• Example 5 (Family Medicine - Professionalism
  issue)

107
Feedback

• Notice, they do not take much time! Each of
  these were between 30 to 90 seconds. Formative
  feedback on the spot doesnt have to take a lot
  of time to be effective or have significant
  impact on your learners and patients.
• Providing feedback often is a very efficient use
  of your teaching time. Accurate feedback can
  prevent the need for repetition.

108
Feedback

• Let us reemphasize the importance of feedback
  when teaching invasive procedures. Based on your
  learning from the previous module on teaching
  procedural skills please remember
• Provide step-by-step feedback during the
  procedure.
• When something goes wrongstep in immediately!
• Patient safety is still our first priority!
• Never let a patient suffer in order to teach a
  learner.
• There are ample times to teach. Knowing when is
  key!

109
Question

• List the five steps for providing feedback.

110
Answer

1. Observe and assess first
2. Identify behaviors or teaching focus
3. Reinforce positive behaviors
4. Discuss changes
5. Summarize and create improvement plan

111
Feedback

• Reflect on how you might apply the five steps in
  providing feedback to your medical students and
  patients.
• Click on the icon to hear an example of a
  mid-rotation feedback session with a student not
  so interested in pediatrics and who can use some
  improvement in obtaining more thorough histories.

Part 1 Setting up a mid-rotation feedback
session (lt1 minute) Part 2 The feedback session
(6 -7 minutes)
112
Patient Feedback

• Regarding patient feedback, please keep Coveys
  habit of seek to understand in mind. Not every
  patient can do everything you ask of them. Why
  is this? Is there something they didnt tell
  you? What influences or circumstances prevented
  or inhibited their compliance?
• Do you seek to understand before you label
  patients as non-compliant?

113
Patient Feedback

• Pause here to reflect on what you would say to
  your patient who wasnt able to get their blood
  pressure medication refilled before coming to
  your clinic visit.
• Did you seek information first? Were your
  comments Specific? Geared toward behavior
  change? Limited? Did it include the patient?
  Was it at their level? Click below to hear an
  example.

114
Feedback

• Make it known that feedback is occurring. You
  may have to state clearly, Now I am going to
  provide you with some feedback.
• Feedback should be given
• Immediately as needed
• Daily
• Mid-rotation
• End of rotation

115
Feedback

• There are several times during the day to provide
  students or patients with feedback.
• During/after presentations
• While observing skills
• While reviewing written records
• When professionalism is demonstrated
• After speaking with the staff, employees, etc.
• While educating patients family members
• And more...think of a few for your team

116
Feedback

• Every teaching moment is an opportunity to
  provide feedback!
• You can make it happen!

117
Feedback

• I have come to the frightening conclusion that I
  am the decisive element in the classroom...As a
  teacher, I possess tremendous power to make a
  child's life miserable or joyous. I can be a tool
  of torture or an instrument of inspiration. I can
  humiliate or humor, hurt or heal. In all
  situations, it is my response that decides
  whether a crisis will be escalated or
  de-escalated, and a child humanized or
  dehumanized. Ginott, 1976

118
Things to Avoid!

• To avoid pitfalls of providing feedback stay away
  from these problems
• Not directly observing behaviors.
• Confusing personality, cultural or ethnic issues
  with behaviors.
• Focusing on negatives without giving positives.
• Giving feedback during times of stress or anger.
  (Either yours or your learners!)

119
Things to Avoid!

• You will also want to avoid
• Waiting too long before giving feedback.
• Giving feedback in a threatening environment.
• Not giving any feedback and sending in an
  evaluation!

120
Feedback

• Select the following statement that best
  describes your future use of feedback.
• I plan to use feedback a lot to help guide my
  learners to better behaviors and performances.
• I plan to use feedback some of the time to help
  guide my learners to better behaviors and
  performances.
• I dont plan to use feedback to help guide my
  learners to better behaviors and performances.

121
Feedback

• Every teaching moment is an opportunity to
  provide feedback!
• Dont let your opportunity pass you or your
  learner by! You are there to help them grow!

122
Feedback-Summary

• Feedback A process by which a teacher provides a
  learner with the results of an assessment with
  the purpose of improving the learners
  performance.

123
Feedback-Summary

• Always provide formative feedback based on direct
  observation and assessment. Every teaching moment
  should be followed by feedback. Feedback can be
  written or verbal.
• Summative feedback is the final feedback given to
  a learner at the end of their training rotation.
  Follow the same key guidelines for formative
  feedback.

124
Evaluation

• You have now learned to assess learners based on
  their standard competencies and learning
  objectives.
• You have also learned about and heard some
  examples of how to provide feedback to patients
  and learners.
• Now lets look at how you complete your student
  evaluation forms after each rotation.

125
Evaluation

• If we go back to our soup example, the journal
  critique will most likely get back to you, the
  chef, in a written format. This is the
  evaluation form.
• When you read your evaluation, you will hope that
  the reviewer provided an honest assessment based
  on the facts. Again, how would you feel if they
  slammed your soup and never even tasted it?

126
Evaluation

• So too with evaluations of learners. Base your
  assessment on the facts and document those facts.
  The form you use should be provided by the
  institution and is usually paper or electronic.
• The evaluation form can take different formats
  including a paragraph of text, a rater sheet with
  numbers, grades or rankings, or a combination of
  both.

127
Evaluation

• Completing evaluation forms will become easier as
  you become more familiar with the evaluation
  form.
• Many will cover the ACGME core competencies we
  learned earlier. These criteria can be very
  helpful in assessing your student. Need a
  reminder of the core competencies? Click here.

128
Competencies

• The 6 ACGME core competencies are
• Professionalism
• Medical knowledge
• Patient care
• Systems-based practice
• Interpersonal communication
• Practice based learning and improvement

129
Evaluation

• Try to become familiar with the criteria by
  reviewing them before your student arrives on
  service. Then mid-month, review them with your
  student and then again at the end of the month.
• Knowing the criteria for students helps you make
  a more accurate assessment of their knowledge,
  attitudes and skills.

130
Evaluation

• Accurate assessments are important in medical
  education. We want to be assured that students
  are evaluated fairly and adequately. Those that
  excel should be recognized, but we also want to
  know which students require remediation. The
  goal of the institution is that all students be
  prepared for their roles as interns.

131
Evaluation

• Completing an evaluation for a student that does
  not adequately reflect their ability can be
  misleading and damaging for the student and
  teacher.
• Students need help seeing where they are in their
  professional development. They also need
  guidance to improve themselves so they can become
  more like a resident.

132
Evaluation

• Your comments also strongly influence the
  students grades and deans letters. Please
  think about your comments first and then complete
  evaluation forms.
• Take a look at some example comments from student
  evaluations. Do you think these are appropriate?
  Do they provide information?

133
Question

• Which is the best?
• Excellent job. Gave good conference talks.
  Needs to be more assertive.
• Overall good job! Will make a wonderful House
  Officer.
• Intelligent and empathic student. Responded well
  to feedback, reorganized history for concise and
  well organized presentations. Has excellent work
  ethic and potential as house officer.

134
Answer

• Remember the feedback game? Some of these
  comments are positive without explaining how the
  student rated compared to competencies or their
  objectives.
• Overall, 3 has the most detail about the student
  and describes their work ethic, flexibility,
  knowledge, and humanism. A residency program
  director would feel more assured selecting this
  student over the other two even thought 1 2
  are both positive.

135
Evaluation

• Think about an evaluation your previously wrote
  or one you received.
• Did you find the information reflective of your
  performance compared to your learning objectives
  or competencies? Did the evaluator provide
  details about your performance that was good as
  well as those that needed improvement? How will
  you change your approach to completing
  evaluations?

136
Evaluations

• What if you have a student that is not stellar
  and is in fact poor or below average not meeting
  their objectives or competencies?
• Handling this situation may take some effort.
  You may need the help of the faculty attending
  for some situations. But most likely, the
  student just needs some guidance to get them on
  the right track.

137
Evaluations

• So how do you deal with students that are not
  enthusiastic about your sub-specialty? Or the
  student who lies? Or the student that never shows
  up? Or the student who is potentially dangerous
  to the patients?
• In following our plan outlined, you must first
  seek to understand. Try to identify their issues
  so you identify the correct teaching focus and
  then discuss this with the student.

138
Evaluations

• Try to approach the student in a non-threatening
  way. In these situations expert communication
  skills using the reflective I can be helpful.
  Try to remain neutral and keep the focus on the
  observed behavior that needs improvement. Using
  the reflective I (I understand this may be
  difficult for you to hear) and avoiding blame
  will keep emotions calm. Avoid getting
  emotionally involved and keep the tone of voice
  as one of a concerned teacher.

139
Evaluation

• Dealing with difficult learners can be
  challenging. To learn more about dealing with
  problem learners read the article by Luca
  Stallworth 2003 on the RATL resource page.

140
Evaluation

• In general, evaluations can also be both
  formative (during) and summative (final). For an
  example, written evaluations throughout the
  clinical years are technically formative, while
  the deans letter sent out at the end of the
  students clinical years is summative. It is the
  final written evaluation of the students
  progress.

141
Evaluation

• You can also think of the three individual
  monthly evaluations of a medicine rotation as
  being formative until the final grade at the end
  of the medicine core rotation, which is
  summative.
• Whether formative or summative the key is that
  you complete your student evaluations based on
  observed behaviors and that the evaluations
  accurately reflect the students behaviors
  compared to given standards.

142
Reflection

• Once again, think about an evaluation you
  received. Were the comments based on direct
  observations? Did the comments compare you to
  your expected role or activities (objectives)?
  Did the comments describe your competence? Were
  you aware of the issues discussed in your
  evaluation?
• Now think about an evaluation you wrote and
  compare it to the questions above. Do you need
  to make any adjustments for the future?

143
Summary

• In summary, you assess students by comparing them
  to a standard most likely their learning
  objectives and competencies.
• Throughout your time with a student, you should
  provide them with feedback that is geared toward
  improving their behaviors and based on direct
  observations.

144
Summary

• You want your feedback to be well received so
  make sure you communicate effectively and focus
  the information on direct observations. Try to
  avoid using good job unless you describe what
  was good and keep students and patients involved
  in the process.
• When completing evaluation forms, use descriptive
  terms that provide insight into the students
  actual performance compared to the standards
  provided.

145
Closing Instructions

• Through the four modules, you have learned about
  professional identity, relationships in medicine,
  leadership skills and the basics of teaching
  students patients.
• We covered 1) effective communication, 2)
  organized efficient teaching, 3) teaching
  psychomotor skills, 4) changing patient behaviors
  and 5) evaluation and feedback.

146
Closing Instructions

• Our goal was to provide knowledge on an
  introductory level regarding teaching and
  leadership skills for 4th year medical students
  and residents.
• After learning about the value of assessment,
  feedback and evaluation, we hope you will take
  the opportunity to provide us with a written
  evaluation of the RATL modules.

147
Closing Instructions

• Please take a few minutes to complete the
  following
• Posttest for Module 4 Evaluation and Feedback
• Teacher Identity Scale
• Evaluation for module 4
• Thank you for participating.

148
End of RATL Module 4
Special Acknowledgements Jennifer Green, M.D.,
Mary Ann Nichols and Regina Russell,
M.Ed. Office for Teaching and Learning in
Medicine Vanderbilt University School of Medicine