Moving Toward a Competency Based Curriculum

1
The Outcome Project

• Moving Toward a Competency Based Curriculum

2
Institutional RequirementsEffective 7/1/02

• IIB1- Institutions must ensure that
• each program has defined, in accordance with the
  Program Requirements,
• the specific knowledge skills, and attitudes
  required and
• provide educational experiences as needed
• in order for their residents to demonstrate.

3
Institutional RequirementsEffective 7/1/02

• 1B3d3 the review should appraise
• The educational objectives of each program
• The adequacy of resources
• The effectiveness of each program in meeting its
  objectives
• The use of dependable measures to assess
  residents competence
• The effectiveness of each program in implementing
  a process that links educational outcomes with
  program improvement

4
A Burden?
Or An Opportunity?
5
consider

• a new PGY-1 in internal medicine. He never
  started an I.V. in medical school (or a central
  line or a paracentesis)
• a PGY-3 surgery resident. The attending in
  pediatric surgery was on sabbatical during his
  rotation(so hes assigned to the research lab)
• a PGY-6 cardiology fellow. The
  electrophysiologist quit during his rotation in
  the EP lab(so he was assigned to the CCU again)
• a PGY-2 resident in pediatrics is discovered to
  have dyslexia (or does 6 weeks of drug rehab or
  12 weeks of maternity leave)
• a PGY 4 pathology resident was pulled from her
  bloodbanking rotation to work in chemistry
• the Childrens Hospital shifts its affiliation to
  UF

6
Question

• How do we assure the public, the boards, and the
  accreditors that a resident finishing a training
  program has actually mastered the knowledge,
  skills, and attitudes we deem to be essential for
  the safe practice of ..

7
Competence and Competency

• Competence personal ability to perform a task
  or process (minimal versus desirable)
• Competency complex set of behaviors which is
  required in order to be competent
• Competency Domains
• KNOWLEDGE
• SKILLS
• ATTITUDES

8
Necessary Planning

• Define the required competencies (per year, for
  the total training, etc.)
• Identify the learning objectives for a beginner,
  intermediate and advanced learner (discrete
  measurable outcomes)
• Design the evaluation tool that allows you to
  judge the degree of accomplishment

9
Types of Evaluation

• Formative- done early in the process to evaluate
  the adequacy of the methodology and the
  appropriateness of the tools (judge the education
  process not the student)
• Summative done at the completion of some or
  all of the curriculum (judge the student)

10
Types of Evaluation

• Criterion based (uses an external standard or
  criterion)
• Norm-based (ranks within a peer group)
• Objective based (against defined objectives)
• Domain based (evaluates a specified field)

11
In Service Examination

• Is it a formative tool to evaluate the program
  or a summative tool to evaluate the resident?
• Answer- Both
• What type of evaluation process?
• Normative (peer group) and domain (specific
  field) No criterion, threshold or standard!
• What does it test?
• Short term knowledge of facts.

12
In the days of Osler and Halsted

• Patients were inpatients
• Hospital stays were long
• Insurance companies didnt have to approve
  tests, procedures, operations
• Attendings werent slaves to the practice plan
• IT WAS POSSIBLE TO EVALUATE KNOWLEDGE, SKILLS
  AND ATTITUDES BY OBSERVATION

13
Tools for Evaluating Competency

• Knowledge standardized tests
• Skills logbooks, simulated patients,
  supervised evaluation, self evaluation
• Attitudes- 360 evaluation, faculty evaluation

14
Closing the Loop

• Evaluation is only meaningful when it includes
  feedback to the learner early enough to allow
  change.
• Evaluation is only meaningful when it is
  constructive, consistent, and amenable to being
  fixed.
• Failure of a resident in an advanced year is a
  failure of the program, not the resident.

15
Pitfalls

• Skills are the easiest to measure and document,
  and so tend to be assessed.
• Attitudes are the most difficult to measure and
  tend to be ignored.
• Recent public opinion challenges the
  professionalism and communication of finishing
  residents.

16
Summary

• Identify desired competency
• Define rotation specific learning objectives
• Teach faculty how to evaluate
• Use evaluation tools consistently
• Provide timely feedback
• Document

17
What are Learning Objectives?
18
Learning Objectives

• what learners should be able to do
  (performance)
• how well they should be able to do
  it (criterion)
• what conditions or equipment must be available
  (condition)

19
Learning Objectives

• What are the learning objectives for medical
  grand rounds?
• For faculty?
• For senior residents?
• For PGY-1s?
• For senior students?
• For junior students?

20
Learning Objectives

• What are the objectives of a rotation in the
  Emergency Department?
• For a surgery resident
• For a psychiatry resident
• For a 4th year student interested in ER
• For a neurology resident
• For a pediatrics resident

21
Big Bird does Planning

• Ask yourself what an individual must know, be
  able to do, and act to be an acceptable.
• Break these KSAs into smaller learning units
  and identify beginner, mid, advanced.
• Decide where these will fit into the x years.
• Describe each component with an action verb
• Ask yourself how you are going to evaluate each
  important component.
• Distribute the objectives into the rotations and
  measure (evaluate) success in each rotation.

22
Epilogue

• If you cant define it and cant measure it, you
  dont know what it is!
• If you dont know what it is, you cant improve
  it!
• If you cant improve it, why do it?

23
(No Transcript)