Benchmarking Clinicians

1
Benchmarking Clinicians

• Farrokh Alemi, Ph.D.

2
Why should it be done?

• Hiring, promotion, and management decisions
• Help clinicians improve

3
Intrusion in clinicians practice?

• Managers understand patient outcomes
• Practice profiles are constructed after the fact,
  when the patient is gone. 
• Both the patient and managers can use benchmarked
  data
• Poor clinicians are bad for the patient as well
  as for the organization

4
How should the analysis be done?

• Compare clinician to average peer
• Compare clinician to average peer taking care of
  same kinds of patients
• Compare clinician to expectations on admission
• Compare clinician and peers on patients matched
  on certain features

In benchmarking,  a clinician's performanceis
compared to an expected value.
5
Compare Clinician to Average Peer

• Calculate peer providers and the clinicians
  average and standard deviation
• Compare using test of hypothesis with unequal
  means
• Problem
• Maybe misleading as providers see different kinds
  of patients and the clinician with more severely
  ill patients will naturally have worst outcomes 

6
Example Data

• 123 internal medicine residents at the New
  York-Presbyterian Hospital in New York City. 
• The outcomes examined included following
  outcomes
• Patients' satisfaction measured by telephone
  interviews of at least 10 patients
• Disease-management profiles for average of 7
  diabetes and 11 hypertensive patients. 
• Patient's condition
• Frequency of use of various medications
• Faculty-evaluations on seven dimensions
• History taking
• Physical examination
• Differential diagnosis
• Diagnostic and/or treatment plan
• Health care maintenance
• Act compassionately
• Team player 

7
Sample Report
8
Compare Clinician to Average Peer Caring for Same
Kinds of Patients
9
Example Data
10
Comparing Clinicians to Expected Prognosis at
Admission

• Assess patients severity
• Predict prognosis
• Calculate pair-wise student-t comparison of
  observed and expected values

11
Example Data
12
Comparing Clinicians When Patient's Severity of
Illness Is Not Known
13
Example Data
14
Event Tree for Clinicians Patients Is Kept,
Outcomes Change
15
Is it reasonable to benchmark clinicians?

• Measurement distorts goals
• Measurement leads to defensive behavior
• No adequate measure of severity maybe available
• Too much time spent on measurement and too little
  on improvement

16
How Should Benchmarked Data Be Presented?

• Before the meeting
• Schedule a feedback time and date as soon as
  possible. 
• Check your data to make sure there are no errors.
  
• Add text, charts or graphics.   Supplement
  numeric data with anecdotal information and the
  customer's voice (e.g. a short audio from a
  patient).
• Distribute handouts ahead of meeting to
  participants

17
How Should Benchmarked Data Be Presented?

• At the meeting
• Make it clear that the evaluation is
  confidential   
• Make a brief introduction of the purpose of the
  session 
• Acknowledge the limitation of the practice
  profiling method
• Present the data and not the conclusions 
• Explicitly ask for clinician's evaluation of the
  data after each section of the report is
  presented 
• DO NOT defend the practice profiling method, the
  benchmarking effort or any aspect of your work 
• Thank the clinicians for their time and describe
  next steps

18
How Should Benchmarked Data Be Presented?

• After the meeting
• Summarize the comments and append it to the
  report.
• Describe resources available.  
• Send a written report to each clinician.
• Ask the clinicians to comment on
• What worked well and what needs improvement?
• Do they plan to change their practice and in what
  way?
• Was it worthwhile?
• Set the time of next benchmarking report.
• .

19
Take Home Lesson

• Expected Outcomes Can Be Benchmarked Using
  Severity of Patients Illness