Objective Structured Clinical Examination (OSCE)

1
Objective Structured Clinical Examination(OSCE)

• Nadine J. Kaslow, PhD, ABPP
• nkaslow_at_emory.edu

2
Context

• ACGME Outcome Project Toolbox of Assessment
  Methods
• Royal College of Physicians and Surgeons of
  Canada CAN MEDS Assessment Tools Handbook

3
Examples of OSCE Stations1. resuscitation of a
child2. consultation skills3. physical
examination skills
1
3
2
4
Context

• Competency Assessment Toolkit for Professional
  Psychology
• High fidelity approach for summative assessment
  of myriad foundational and functional
  competencies

5
Context

• Competency Assessment Toolkit for Professional
  Psychology
• OSCEs are useful for assessing
• Readiness for practicum?
• Readiness for internship
• Entry level to practice
• Advanced credentialing

6
What is an OSCE?

• Objective all candidates are presented with
  the same stimuli

7
What is an OSCE?

• Objective
• Structured Specific foundational and functional
  competencies are tested at each station and the
  marking schema for each station is structured

8
What is an OSCE?

• Objective
• Structured
• Clinical Examination Test of performance of
  clinical competencies, with an emphasis on skills
  and attitudes

9
What is an OSCE?

• Evaluation tool that allows people to be
  observed performing in many different clinical
  situations

10
What is an OSCE?

• OSCE combines
• Multiple observations
• Standardization of content
• Range of difficulty

11
What is an OSCE?

• Effective method to assess foundational
  competencies
• Professionalism (e)
• Scientific knowledge methods (e)
• Reflective practice (e)
• Individual cultural diversity (e)
• Relationships (o)
• Ethical legal standards policy (o)

12
What is an OSCE?

• Effective method to assess functional
  competencies
• Assessment (o)
• Intervention (o)
• Consultation (e)
• Research and evaluation (e)

13
Methodology

• Standardized patients (SP)
• Stations
• Marking scheme

14
Standardized Patients (SP)

• SPs are lay persons who are trained to portray
  scripted patient presentations accurately and
  consistently across many encounters
• SP encounters are credible and reliable

15
Standardized Patients (SP)

• SPs are used to teach
• Foundational competencies
• Interpersonal and communication skills (CIS)
• Clinical reasoning
• Functional competencies

16
Standardized Patients (SP)

• SP ratings of CIS provide a good proxy for
  actual patient satisfaction
• SP Communication
• Increases patient satisfaction
• Facilitates doctor-patient relationship
• Increases compliance
• Decreases malpractice claims

17
Standardized Patients (SP)

• Several encounters are required to obtain a
  reliable estimate of a persons competence
• SP assessments are frequently organized as a set
  of cases or stations
• This series of encounters is known as OSCE

18
Implementation

• While there are variants on the original OSCE
  format, key implementation steps are
• Train SPs in the issues of encounter and ensure
  they dont provide any information unless the
  person being assessed requests it

19
Implementation

• SP encounters or tasks requiring interpretation
  of clinical information must be provided at
  separate stations
• These must last 5-10 minutes and be observed by
  the assessor
• Topics relationship development, risk
  assessment, diagnosis, therapeutic technique,
  consultation strategy

20
Implementation

• At each station, the person being assessed
  completes notes about the SP

21
Implementation

• The person being assessed moves between stations
  when an announcer indicates in accord with a
  specified time sequence
• Separate performance scores are generated for the
  task at each station based on input from the SP,
  the person being assessed, and the assessor

22
Implementation

• The scores from multiple informants across the
  stations or tasks are combined by the assessors
  to reach a final outcome determination

23
Role of Examiners - YES

• Observe the performance of the trainee at a
  particular task
• Score according to the marking scheme (get
  practice at marking)
• Contribute to the good conduct of the examination

24
Role of Examiners - NO

• Re-write the station
• Interfere with the SPs role
• Design their own marking scheme
• Teach

25
Psychometrics

• With appropriate attention to design, OSCEs
  have acceptable psychometric properties including
• Good inter-rater, inter-station, and split-half
  reliability
• Good generalizability
• Strong content, construct and concurrent validity

26
Psychometrics

• OSCE scores correlate moderately with other
  performance indicators and other forms of
  evaluations

27
Psychometrics

• OSCE has increased reliability and content
  validity with greater number of stations and
  similarity between tasks at different stations

28
Strengths

• Provides for a high fidelity assessment

29
DOES
Skills Attitudes
SHOWS HOW
Professional Authenticity
KNOWS HOW
Knowledge
KNOWS
Miller et al., 1990
30
DOES
SHOWS HOW
OSCE
KNOWS HOW
KNOWS
Miller et al., 1990
31
Strengths

• Measures clinical competence cross-sectionally
  using standarized means
• Focuses on observable behaviors
• Enables fairer peer comparison

32
Strengths

• Allows for assessing complex competencies
  without endangering patients well-being
• Encourages a collaborative assessment approach
• Has the potential for peer feedback and assessment

33
Strengths

• Has been extensively researched and found to be
  valuable for summative assessment
• Provides valuable information for curriculum
  review

34
Challenges

• Is challenging to create and administer
• Is labor intensive
• Is costly
• Only cost-effective when many individuals are
  examined at one administration

35
Challenges

• Requires high demand of other resources
• SPs
• Assessors
• Time commitment
• Physical resources

36
Challenges

• Doesnt provide longitudinal assessment
• Doesnt adequately tap complex skills requiring
  integrated professional judgment

37
Example SP Information

• I am a 25 year old who comes to the Emergency
  Room for treatment of a 2-3 week old puncture
  wound that I can see but no one else can
• I am having a first episode of psychosis and have
  numerous alterations in my thinking and function
  however it is not immediately obvious that my
  thoughts are disordered when the student begins
  taking my history

38
Example SP Information

• Chief complaint - Stepped on a nail
• Opening statement - I stepped on a nail and the
  wound wont heal If pressed to say more Its
  making me sick and I need help

39
Example SP Information

• Appearance - alert and oriented, calm,
  cooperative, poor hygiene (dirty unmatched
  clothes, hair uncombed, no make up or grooming)
• Look like you havent showered in days
• Have some eye contact but less than normal
• Your affect is flat
• You talk very little and you have no emotion in
  your voice when you are talking

40
Example SP Information

• Thought processes - you are focused on saving
  America and you mostly stay on topic when
  speaking but sometimes you start talking about
  one thing and it turns into a related topic but
  not on point
• An example might be Coke advertisements transmit
  messages to me. The first coke slogan was Drink
  Coca-cola released in 1886.
• Refer to three Coke slogans during your
  interview. If possible say these when you can
  tangentially connect them to something else you
  are saying.

41
Examples
42
Conclusion

• OSCEs are a valuable and well-developed
  assessment approach in other health professions
• It behooves us as psychology to develop and share
  OSCE scenarios for the essential components of
  foundational and functional competencies

43
Conclusion

• OSCEs are advantageous to incorporate in our
  assessment armamentarium for summative
  evaluations particularly related to key
  transition phases
• Attention needs to be paid to increasing the
  feasibility of this high fidelity assessment
  methodology

44
Questions