Collective Mistrust of Alarms

1
Collective Mistrust of Alarms

• James P. Bliss, Ph.D.
• Susan Sidone
• Holly Mason
• Old Dominion University

2
Collective Mistrust of Alarms - A Few Thoughts
Before We Begin...

• Novelty of this project
• Not Automation per se Alarms inform, do not
  control
• Alarms convey system state to operator
• May help to push the envelope of etiquette
  research
• Mistrust/distrust may be different with alarms
• Simplistic paradigm deceptive - multiple trust
  components involved
• Information accessibility, technology
  improvements means operators expect more from
  alarm systems
• Operator mental models very important

Bliss, Sidone, Mason, 2002
3
Collective Mistrust of Alarms - Introduction

• Investigations of Individual Alarm Mistrust
• Aviation (Bliss, 1997)
• Mining (Mallett et al., 1992)
• Ship Handling (Kerstholt et al., 1996)
• Driving (Nohre et al., 1998).
• General Findings People Reacted
  Slower, Less Frequently, Less
  Appropriately to Unreliable Alarms.
• No Studies of the Impact of Marginally Reliable
  Alarm Signals on Teams of Operators

Bliss, Sidone, Mason, 2002
4
Collective Mistrust of Alarms - Introduction

• Teamed Alarm Reactions
• Aviation
• Critical Care Units
• Nuclear Power Plants
• Air Traffic Control Centers
• To Effectively React to Alarms, Team Members Must
• Share Information
• Troubleshoot Systems
• Determine Relative Signal Priority
• Allocate and Coordinate Reaction Responsibility

Bliss, Sidone, Mason, 2002
5
Collective Mistrust of Alarms - Introduction

• Team Member Interdependence Often Varies with the
  Task and the Environment (Thompson, 1967).
• Dependent Teams React to Alarms More
  Appropriately, More Slowly (Bliss et al., 2002)
• Implications of Teamed Alarm Reactions for
  Human-Automation Etiquette
• Human-Alarm Trust
• Human-Human Trust
• Human-Human(Alarm) Trust

Bliss, Sidone, Mason, 2002
6
Collective Mistrust of Alarms - Introduction

• Goals of the Current Research
• Investigate Reactions of Dependent and
  Independent Teams to Alarm Signals of Various
  Reliability Levels.
• Determine How Collateral Alarm Systems Mediate
  Alarm Mistrust.
• Approach
• Dual-Task Approach (Damos, 1991).
• Independent Variables Manipulated Using a 2 X 3
  Mixed Design.
• Dyads Reacted to Two Separate Alarm Systems.
• Temperature Alarm Reliability 80 true alarms).
• Pressure Alarm Reliability Fluctuated (40, 60
  or 80).

Bliss, Sidone, Mason, 2002
7
Collective Mistrust of Alarms - Experimental
Design

• Interdependence Manipulated Between Two Groups
• Dependent Team Members Required Interaction to
  React Appropriately Independent Team Members Did
  Not.
• Pressure Alarm Reliability Manipulated Within
  Groups
• Pressure Alarms Were 40, 60, and 80 Reliable
  During Sequential Task Sessions. Temperature
  Alarms Were 80 Reliable.
• Dependent Measures
• Ongoing Task Gauge Monitoring Accuracy,
  Tracking Error.
• Alarm Task Reaction Speed, Appropriateness
  Response Frequency.

Bliss, Sidone, Mason, 2002
8
Collective Mistrust of Alarms - Method

• Participants 40 student dyads from Old Dominion
  University (18-43 yrs) worked for course credit
  and the chance for a monetary performance bonus.
• Primary Task Multi-Attribute Task (MAT) battery
  (Comstock Arnegard, 1992) presented to each
  member.
• Dual-Axis Compensatory Tracking
• Gauge Monitoring
• Resource Management
• Participants Performed the MAT Back-to-Back

Bliss, Sidone, Mason, 2002
9
Collective Mistrust of Alarms MAT Battery
Bliss, Sidone, Mason, 2002
10
Collective Mistrust of Alarms - Method

• Auditory and Visual Alarms Digitized Fire Bell
  From a Boeing 757/767 simulator.
• Alarms Occurred 90 to the Side of the Primary
  Task.
• Alarm Procedure Determine Whether Corresponding
  MAT Gauges Are Out of Tolerance. If so, Reset
  Gauges and respond to the alarm. If Not, Cancel
  the alarm and resume the primary task.
• Interdependent team members had to communicate
  because they shared the out-of-tolerance gauges.
  Independent team members monitored all gauges.

Bliss, Sidone, Mason, 2002
11
Collective Mistrust of Alarms - Procedure

• Informed Consent Form
• Experimental Instructions - Dependent team
  members told to communicate.
• MAT Task Practice
• Individual 120-second sessions (Each Subtask)
• Combined 200-second session (MAT and Alarms)
• Three experimental sessions
• Ten alarms presented during each session.
• Pressure alarm reliability randomly
  counterbalanced
• Participants Knew Alarm System Reliability Before
  They Began
• Debriefing, dismissal.

Bliss, Sidone, Mason, 2002
12
Collective Mistrust of Alarms Results (Response
Frequency)

• Response Frequency to Temp Alarms
• No Interaction (pgt.05)
• Linear main effect, F(1,38)129.600, plt.001.

Bliss, Sidone, Mason, 2002
13
Collective Mistrust of Alarms Results (Reaction
Appropriateness)

• Significant Interaction, F(2,76)10.193, plt.001.
  
• Main Effect for Interdependence, F(1, 38) 4.000,
  p.05.
• Quadratic Main Effect for Reliability,
  F(1,38)19,563, plt.001.

Bliss, Sidone, Mason, 2002
14
Collective Mistrust of Alarms Results (Reaction
Time)

• No significant interaction
• No Interdependence main effect
• Linear Reliability Main Effect, F(1,38)8.181,
  p.007.
• NOTE No Primary Task Differences

Bliss, Sidone, Mason, 2002
15
Collective Mistrust of Alarms - Discussion

• Results Similar to Past Efforts, Except for Lack
  of Primary Task Differences.
• Multiple alarm systems may have led participants
  to rethink their trust levels, a reflection of
  workload (Bliss Dunn, 2000).
• Alarm designers should consider the effects of
  multiple alarm systems on operator behavior.
• Recognize that complex reaction responsibilities
  may cause cognitive load as team members adjust
  trust levels.

Bliss, Sidone, Mason, 2002
16
Collective Mistrust of Alarms - Discussion

• Dimensions of Trust in This Experiment
• Basic Trust of the Experimenter (Human-Human)
• Trust of the Primary (MAT) Task (Human-Computer)
• Trust of the Alarm Task (Human-Computer)
• Manipulated by the Experimenter
• Trust of Teammates (Human-Human)
• Questionable in this Experiment, Due to
  documented Unfamiliarity

Bliss, Sidone, Mason, 2002
17
Collective Mistrust of Alarms - Discussion

• Past reactions to unreliable alarm systems
• Fluctuations in physiological responses
  (Breznitz, 1983)
• Degraded performance (Getty et al., 1995)
• Complete Lack of Trust (Bliss, 1993)
• Complete Trust
• Probability Matching Participants Response
  Rates Mirror the Perceived Reliability of the
  Alarm System.
• These Patterns Take Time to Appear (Bliss et al.,
  1996).
• Question What if Researchers Apply human trust
  facilitators to human-alarm relationships? WHAT
  ARE THOSE VARIABLES?

Bliss, Sidone, Mason, 2002
18
Collective Mistrust of Alarms - Discussion

• Documented Ways to Improve Alarm Responsiveness
• Maximize alarm reliability (Bliss, 1993)
• Advertise high alarm reliability rates (Bliss et
  al., 1995)
• Add Redundant Sources of Alarm Information (Bliss
  et al., 1996)
• Augment alarm stimuli and response options
  (Bliss, 1997).
• Etiquette Related Possibilities
• Give alarm systems human qualities
  (include verbiage, etc.)
• Make alarm stimuli emotional
  (Sorkin et al.s
  likelihood alarm displays
  altering Edworthys parameters)
• Vary teammate trustworthiness

Bliss, Sidone, Mason, 2002