Using Simulation in TeamSTEPPS

1

Using Simulation in TeamSTEPPS Training
2
Objectives

• To know and be able to apply the Event Based
  Approach to Training
• To know and be able to develop TeamSTEPPS
  training scenarios
• To know and be able to develop TeamSTEPPS
  performance measures
• To know and be able to conduct effective debriefs
  of team performance

3
Course Outline

• Phase 1. Scenario Development
• Exercise
• Phase 2. Measurement Development
• Exercise
• Phase 3. Debriefing
• Lessons Learned

4
Simulation

• Team skills are developed through practice and
  feedback
• Simulation allows health care professionals to
  practice TeamSTEPPS skills and strategies in a
  safe learning environment
• Simulation includes
• Role play
• Part-task trainers
• Human patient simulation
• Standardized patients

5
Keys to Success

• Proper scenario design
• Focus on learning objectives
• Provide more than one opportunity to practice
  team behaviors
• Accurate measurement
• Include process and outcome measures
• Capture behaviors
• Debriefing
• Learning happens in the debrief
• Include feedback on how to improve performance

6
TeamSTEPPS Resources

• Scenarios Tab I
• 131 Vignettes
• Organized by department and TeamSTEPPS skill and
  tool
• Team Performance Observation Tool Tab A,
  Appendix C
• Tool for observing team performance
• Observers should practice using the tool
• Can be adopted to a particular type of care

7
EBAT

• Event-Based Approach to Training (EBAT)
• Not a new method
• U.S. Navy, Targeted Acceptable Responses to
  Generated Events (TARGETs) (Fowlkes, Lane, Salas,
  Franz, Oser, 1994)
• FAA Advisory Circular 120-35C
• Line Operational Simulations
• How to design simulation scenarios
• Health care domain
• Simulation Module for Assessment of Resident
  Targeted Event Responses (SMARTER) (Rosen, Salas,
  Silvestri, Wu, Lazzara, 2008)

8
Phase 1

• Scenario Development
• Specify teamwork skills
• Define learning objectives
• Choose clinical context
• Define event sets
• Define targeted responses

9
Specify Teamwork Skills

• Skills are general categories of behavior you
  want to train
• Teamwork is too complex to train in a single
  scenario
• Need to focus on a subset of competencies
• There are four core skills in TeamSTEPPS

10
Define Learning Objectives

• Learning objectives should be explicit and
  measurable
• Focus on specific TeamSTEPPS behaviors
• Include the following information
• Performance Specifies the behavior
• Condition(s) Specifies the conditions under
  which the behavior occurs
• Standards Specifies the level of expected
  performance

11
Example Objective
Communication
Learning Objective Uses two-challenge rule or
CUS to advocate for a patient in labor
experiencing frequent, strong contractions
12
Choose a Clinical Context

• Considerations
• Medicine is a notoriously task-specific domain
• Not all contexts are equal for training purposes
• Choose a clinical context that affords
  opportunities to perform
• TeamSTEPPS provides 131 scenarios (Tab I)
• Contexts should be appropriate for eliciting the
  team behavior of interest

13
TeamSTEPPS Scenario 86

• Sally Rodgers, a 25-year-old nullip in labor at
  term who is dilated 3 cm. This is a change from
  2cm over the previous 90 minutes. Sally is having
  frequent, strong to palpation contractions that
  are extremely uncomfortable. She is trembling,
  complaining of nausea, and begging her nurse for
  pain relief. The patients primary nurse believes
  epidural anesthesia would be appropriate and
  informs the obstetrician (OB) attending. The OB
  attending states he wants the patient to be
  dilated 45 cm before she receives the epidural.
  The nurse reiterates to the attending OB that her
  assessment is that the patient is in active
  labor. Although Sallys cervix has not
  demonstrated active labor yet, her nurse believes
  the pain relief and relaxation resulting from an
  epidural would be beneficial for the patient. The
  attending OB agrees to the epidural placement.
  The patient is fully dilated and begins pushing 3
  hours after the placement.

14
Clinical Context

• Scenario 86
• Sally Rodgers, a 25-year-old nullip in labor at
  term who is dilated 3 cm. This is a change from
  2cm over the previous 90 minutes. Sally is having
  frequent, strong to palpation contractions that
  are extremely uncomfortable. She is trembling,
  complaining of nausea, and begging her nurse for
  pain relief.

15
Define Event Sets

• Event sets are the building blocks of a scenario
• Event sets consist of
• Trigger The incident to elicit the team
  behavior
• Distracters Characteristics of the situation
  (i.e., conditions) in which the behavior is to
  occur
• Examples
• Trigger The OB attending states he wants the
  patient to be dilated 45 cm before she receives
  the epidural
• Distracter Sally has a strong contraction and
  screams

16
Define Targeted Responses

• Targeted responses
• Behavioral responses to each event set trigger
• Expected levels of performance (i.e., standards)
• Example
• Trigger The OB attending states he wants the
  patient to be dilated 45 cm before she receives
  the epidural
• Response The nurse uses two-challenge rule or
  CUS to advocate to the physician that Sally
  should receive pain medication now

17
Guidance

• Simulation scenarios should consist of 3-5 event
  sets
• Provides more than one opportunity to demonstrate
  behaviors
• Creates reliability
• Each event set should not include more than one
  trigger event
• Event sets can be created by breaking a clinical
  procedure into chunks
• Embed triggers into these chunks

18
Trauma ExamplePre-Hospital/Transport
Trauma Phase Event Set Triggers Expected Team Behaviors
Pre-Hospital/Transport Patient arrives without advance notice ("zero time") Leadership Team leader assembles team quickly Team leader advocates a plan
Pre-Hospital/Transport Inaccurate or incomplete information from EMS/rescue team Leadership Team leader advocates a plan based on known information Communication Team members use call-outs and check-backs to share accurate patient injury information and discoveries through surveys Team members request information from EMS/rescue when information has not been provided
19
Trauma ExamplePrimary Survey
Trauma Phase Trigger Expected Team Behaviors
Primary Survey Too many people in room Leadership Team leader may ask non-team members to leave Communication Team members communicate findings and orders with clear call-outs and check-backs Team members are not distracted by others
Primary Survey Equipment not working or is missing Situation Monitoring Team member identifies equipment issue Team member remedies issue (e.g., replaces, fixes, or obtains needed equipment) quickly
20
Scenario Development

• Select a scenario from Tab I
• Identify/develop the following items
• TeamSTEPPS skill
• Learning objective(s)
• Clinical context
• Event set
• Trigger and distracters
• Targeted responses
• Be prepared to present your results

21
Phase 2

• Develop Measures
• Consider level of analysis
• Clarify purpose
• Decide what to measure
• Select a measure

22
Team Performance Observation Tool

• Leadership
• Utilizes resources efficiently to maximize team
  performance
• Balances workload within the team
• Delegates tasks or assignments, as appropriate
• Conducts briefs, huddles, and debriefs
• Empowers team members to speak freely and ask
  questions
• Situation Monitoring
• Includes patient/family in communication
• Cross monitors fellow team members
• Applies the STEP process when monitoring the
  situation
• Fosters communication to ensure team members have
  a shared mental model

• Mutual Support Rating
• Provides task-related support
• Provides timely and constructive feedback to team
  members
• Effectively advocates for the patient
• Uses the Two-Challenge rule, CUS, and DESC script
  to resolve conflict
• Collaborates with team members
• Communication Rating
• Provides brief, clear, specific, and timely
  information to team members
• Seeks information from all available sources
• Verifies information that is communicated
• Uses SBAR, call-outs, check-backs and handoff
  techniques

23
Consider Level of Analysis

• Individuals
• MD
• Nurse
• Anesthesia
• Teams
• Delivery team
• C-Section team
• Multi-team Team Structure
• Core team
• Contingency team
• Administrative team

24
Clarify the Purpose

• Diagnose root causes of performance deficiencies
• Identify specific weaknesses
• Provide feedback
• Relay information regarding strengths and
  weaknesses as a remediation plan
• Assessment
• Evaluate the level of proficiency or readiness

25
Decide What to Measure

• Outcomes tend to be more quantifiable and answer
  the question What happened?
• Processes answer the question Why did it
  happen?

26
Outcomes

• Outcomes
• Sometimes referred to as measures of
  effectiveness (MOEs)
• Provide an indication of the extent to which the
  outcome of the task was successful
• Are important for most measurement purposes
• What outcomes could we assess?
• Accuracy Precision of performance (e.g.,
  correct medication)
• Timeliness How long (e.g., time to incision)
• Productivity How much (e.g., patient volume in
  ED)
• Efficiency Ratio of resources required versus
  used (e.g., OR supplies)

27
Process

• Processes
• Sometimes referred to as measures of performance
  (MOPs)
• Explain how and why certain outcomes may have
  happened
• (Was the decision made right? versus Was the
  right decision made?)
• Important when diagnosing root causes of
  performance deficiencies and providing feedback
  or follow-on training
• Types of Process
• Procedural Taskwork
• Non-procedural Taskwork
• Teamwork

28
Measurement Tips

• Assess processes as well as outcomes when
  diagnosing performance deficiencies or providing
  feedback
• Consider measuring at multiple levels to identify
  the weak link
• Provide multiple opportunities to perform the
  same task or skill over the course of an exercise

29
Select a Measure

• Checklists
• Consist of items/actions that have dichotomous
  answers such as Yes/No Right/Wrong
  Performed/Not Performed
• Example Primary Survey ABCs
• Frequency counts
• Indication of the number of times that a
  behavior, action, or error occurs
• Example Use of CUS, SBAR, Two-Challenge
• Rating scales
• Numerical or descriptive judgment of how well a
  task was performed

30
Checklist Tips

• Best used with a scripted scenario versus free
  play
• Items should be related to triggers embedded into
  the scenario
• Each item should represent a single action taken
  by the individual, team, or battlegroup
• Label or define the response category that you
  are using

31
Checklist
No
Yes
Action/Behavior
Assess Airway
Breathing
Circulation/FAST Exam
Disability

Exposure and Environment
32
Frequency Count Tips

• Frequency counts are better when measuring acts
  of commission than acts of omission
• Overt actions or errors versus failing to
  demonstrate a particular behavior
• Frequency counts are good when you want to know
  how often a specific action is taken or task is
  performed
• Frequency counts can be recorded during a
  critical event in an exercise or throughout the
  entire scenario

33
Frequency Count
Positive Instances
Communication
Check-back Call-outs SBAR Unintelligible
Communications
Mutual Support
Positive Instances

Two-Challenge CUS Task Assistance
34
Graphic Rating Scale

• The team leader assigned roles to the Trauma
  Team.
• 2. The PGY2 used check-back to confirm orders.

Very Effective
Ineffective
x
1 2 3 4
5 6
Strongly Disagree
Strongly Agree
x

1 2 3 4
5 6

35
Anchored Rating Scale
Communication Used check-back during trauma
resuscitation.
x
36
Rating Scales

• Uses
• Rating scales are good for assessing quality when
  it does not equate to a sum of quantity
• Rating scales are good for assessing tasks that
  are less procedural in nature
• Rating scales are better to use when you are
  evaluating performance on a continuum
• Tips
• Define or clearly describe what you are measuring
• Include a specific stem or item to accompany the
  rating scale

37
Measures

• Develop a measure to assess performance for the
  scenario you developed earlier
• Be prepared to present your measures

38
Phase 3

• Debriefing
• Introduce the debrief process
• Describe what happened
• Conduct an analysis of performance
• Identify lessons learned

39
Description Phase

• Recap of what happened in the scenario
• Team members share their perspectives on what
  occurred during the scenario and reach common
  ground
• This helps ensure everyone takes away similar
  lessons from the experience
• How measurement can help
• Provides a structure for understanding the
  scenario
• The event sets or behavior categories can be used
  to structure discussion
• Focus on critical aspects of performance
• Helps to keep the discussion focused on events
  relevant to the learning objectives

40
Analysis Phase

• A systematic investigation of why things happened
  in the scenario as they did
• The team focuses on understanding what went well
  and what could have been done better
• How measurement can help
• Can help compare the teams performance with
  standards of performance
• Were the TeamSTEPPS behaviors performed when
  necessary? If so, were they performed correctly
  or could they be improved?

41
Application/Generalization Phase

• A look ahead to how the team can generalize what
  they learned in the scenario to their daily
  practice
• The team generates lessons learned and discuss
  what needs to be corrected and how to correct it
• How measurement can help
• Explicit event sets can be used to draw parallels
  between the scenario and the actual clinical
  environment
• Explicit measures associated with these events
  help promote reflection about how to transfer
  what went well to the actual clinical environment

42
Tip for Success 1

• Tip 1 Dont overwhelm learners or observers
  Keep it simple
• People can integrate only a few key learning
  points from a scenario
• Observers have a limited attention span and
  frequently have to multi-task. Dont ask too
  much you wont like what you get
• Rule of thumb Have a key event every 1 to 2
  minutes of scenario time

43
Tip for Success 2

• Tip 2 Telling someone how well they did is not
  good enough
• They need diagnostic feedback
• Specific
• Behaviorally focused
• Descriptive

44
Tip for Success 3

• Tip 3 Train observers
• Training, by definition, is systematic. For
  measurement to guide this process, it too must be
  systematic, reliable, and valid
• Make sure everyone has common expectations about
  performance
• Develop and use a scoring guide

45
Rater Scoring Guide
Team Behaviors Hit Variation in Quality Miss
Communication SBAR uses structured communication (situation, background, assessment, recommendation) all components (situation, background, assessment, and recommendation) are present several, but not all components were used (e.g., the situation was presented and the background leading to the situation was stated, BUT NO assessment or recommendation is provided) structured communication was NOT used at all when appropriate (e.g., during a handoff)
Communication Call-out verbalizes critical information to the entire team a team member clearly verbalizes critical information to all team members critical information was only verbalized to one of the team members and not all of them critical information was not verbalized
Communication Check-back uses closed-loop communication a sender initiated a message, the receiver accepted the message and provided feedback confirmation, and the sender verified message was received a team member initiates communication and the other team member provides verification of receipt and understanding, but the initiator DOES NOT verify this understanding is correct. a sender initiated a message but the receiver DID NOT provide feedback confirmation, and the sender did not verify message was received
46
Tip for Success 4

• Tip 4 Keep teamwork and clinical skills separate
• Use dual debriefs
• Provide feedback on teamwork skills as a team
• Correct an individuals major clinical
  deficiencies in a follow-up session
• Dont overcomplicate the clinical nature of the
  scenario when your main purpose is to train
  teamwork
• Teamwork novices especially should be given the
  chance to focus on teamwork in the scenario, not
  complex clinical issues
• As the team members teamwork skills advance,
  more complex clinical scenarios can be used

47
Tip for Success 5

• Tip 5 Event-based methods involve more than
  just measurement
• Good training design practices
• Good scenario design practices
• Good debrief facilitation practices

48
Teamwork Actions

• Apply the Event Based Approach to Training
• Able to develop TeamSTEPPS training scenarios
• Able to develop TeamSTEPPS performance measures
• Know how to conduct effective debriefs of team
  performance