Evaluating CanMEDS Issues and Options

1
Evaluating CanMEDS - Issues and Options

Glen Bandiera, MD MEd FRCPC January 26, 2007
2
What is this?
3
What is this?
4
Objectives

• Identify key componenets of a good evaluation
• Describe strengths and limitations of several
  tools
• Adapt an appropriate tool for a given task

5
Key Messages

• There is a tool for every CanMEDS role
• No single tool can do it all
• Assessment is multimodal
• It is unnecessary to assess all roles all the
  time
• Appropriate tools will be specialty-specific

6
The Plan

• Overview of the Roles and Resources
• Assess your program
• Name your challenge
• Review assessment tools
• Self-exercise re your challenge
• Group discussion
• Summary

7
CanMEDS Roles

8
Foundational Concepts

• Reliability
• The degree to which an assessment procedure will
  provide the same results from the same
  circumstances every time
• The percentage of variability between assessments
  that is due to the signal of interest (and not
  noise)

9
Foundational Concepts

• Validity
• The degree to which an assessment tools measures
  what it is intended to measure
• Content
• Construct
• Face
• Predictive
• Criterion

10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Competency Pyramid Miller (1990)
15
What is this?
16
Case Study - Examinations

• STEM
• A 70 year old female is brought to the ED after a
  fall. Her vital signs are normal and she
  complains of right hip pain.
• THE QUESTION ?

17
Case Study - Examinations

• The Old Standard
• List the FOUR components of the Garden
  classification of femoral neck fractures
• List FOUR complications of hip fractures

18
Case Study - Examinations

• List TEN risk factors for falls in the elderly
  (HA)
• The consultant orthopedic surgeon refuses to see
  the patient because they have no beds to admit
  to. List FOUR possible courses of action to
  address this problem. (Mgr)

19
Case Study - Examinations

• There is no fracture on xray. List FIVE specific
  interventions a physiotherapist may provide to
  assist this patient. (Collaborator)
• A bony metastasis is found on xray. The patients
  son asks that she not be told. List ONE ethical
  principle at play here and briefly outline your
  approach. (Prof.)

20
Case Study Self- Assessment

• Friedson (1970s) professional behaviour is far
  more influenced by the work place and norms of
  practice than by training
• Kruger, Dunning (1999) those who scored lowest,
  had least accurate perception of their ability
• Norman (1999) Learners can self-direct, but
  cannot always self-monitor

21
The Literature the book

• Literature Search
• Comprehensive bibliography
• Abstract and article review

22
The Resource

• Assessment Matrix
• Tools
• One-page descriptions
• Roles
• Detailed description
• Preferred tools
• Key articles (annotated)

23
What is this?
24
CanMEDS Assessment Matrix
25
Self-Exercise (Step 1-3)

• How do you assess the CanMEDS roles in your
  program?
• Create your program Assessment Matrix
• List your biggest remaining challenge
• What are the issues?

26
Your Challenges
27
What is this?
28
Self-Exercise (Step 4-7)

• Choose ONE suggested tool from the matrix
• Read the one-page tool synopsis
• Read the detailed role description
• Can the tool be adapted to your program?

29
Some examples for discussion
30
What is this?
31
Recall

• There is a tool for every CanMEDS role
• No single tool can do it all
• Assessment is multimodal
• It is unnecessary to assess all roles all the
  time
• Appropriate tools will be specialty-specific

32
Questions

• While we contemplate- please evaluate!

33
Evaluating CanMEDS - Issues and Options Glen
Bandiera
http//www.cfd.med.utoronto.ca/index.htm http//ww
w.deptmedicine.utoronto.ca/canmeds.htm