Aligning assessment to intended curriculum outcomes

1
Aligning assessment to intended curriculum
outcomes

• Richard Hays
• School of Medicine
• Keele University

2
Good Medical Practice

• Good clinical care
• Maintaining good practice
• Teaching training, appraising assessing
• Relationships with patients
• Working with colleagues
• Probity
• Health

3
Why bother at medical school?

• Problems at medical school predict problem
  doctors later in life, to some extent (Papadakis
  et al)

4
Selection will fix that!

• Really?

5
Assessment guidelines

• Learning outcomes
• Blueprinting
• Sampling of test items
• Standard setting
• Select method
• Utility index

6
Written methods

• Applied knowledge
• Clinical reasoning?
• Attitudes??

7
Practical methods

• Procedures
• Clinical skills
• Behaviours?
• Attitudes??

8
Gaps

• Professional behaviour
• Clinical skills
• Application of knowledge
• Learning skills?
• Relationships with patients
• Teamwork
• Probity
• Health

9
Competence / Performance

• Does
• In the real world (uncontrolled)
• Over a period of time
• Broader perspectives

• Can do
• Examination conditions (controlled)
• At one point in time
• Narrower perspectives

10
Both

• May fluctuate
• Should be matched to learning outcomes and
  context
• Require careful selection, based on evidence
• Require standard setting
• Require examiner training
• Summation is complex

11
Example OSCE stations

• Taking a history from patient with diabetes
• Examination of patient with a CVA
• Explanation of prognosis to a patient with Ca
• Ethical behaviour

12
Example written questions

• Ability to write a prescription
• Ability to write a referral / order tests
• Knowledge and applied knowledge
• Choice from several options
• Written simulation what to do next

13
Assessment in the workplace

• More performance-oriented
• Problems harder to hide?
• Application of knowledge
• Demonstration of skills
• Behaviours evident (attitudes?)

14
Learning portfolios

• An eclectic and highly variable format
• A collection of pieces of evidence that
  demonstrate learning progress
• Validity can be high
• Reliability?
• Formative role clearer than summative

15
Learning portfolio content(a format, not a
method)

• Logs of clinical skills observed
• Procedures performed ( observed?)
• Clinical cases written up
• Copies of formative assessment
• 360 degree assessments (MSF)
• Self evaluations (insight)
• Evidence of probity and health?

16
Pitfalls in workplace assessment

• Developing individual components
• Standard setting
• Assessor training
• Rules of progression
• Surviving appeals

17
Individual components

• Each one should address learning objectives
• Each should be developed following assessment
  guidelines and consider the Utility Index
• In combination should address a wide range of
  Domain/Theme issues
• Not a dump for the unwanted elsewhere!

18
But beware

• Size of the combined tasks
• Rewarding magpies
• Time to assess
• Deriving overall score
• Not the sole answer!

19
Standard setting

• Difficult, but not impossible
• Consensus-based approaches probably better
• Require input from diverse groups involved in
  professional practice

20
Assessor training
21
Rules of progression

• Formative or summative clarity
• Attendance vs participation
• Compensation from other assessments?
• Dealing with poor professionalism
• Success of appeals