Human Factors: DecisionMaking

1
Human Factors Decision-Making
Associate Professor Mark Wiggins School of
Psychology and MARCS Auditory Laboratories
2
Outline

• Introduction to Human Factors
• Human Factors and Decision-Making
• Approaches to the Analysis of Decision-Making
• Theories of Decision-Making
• Non-experimental Techniques and Decision-Making
• Experimental Techniques and Decision-Making
• Decision-Making and System Design

3
Human Factors
A multidisciplinary approach that seeks to
optimise the relationship between the operator
and the environment

• System Design
• Training
• Selection

4
The Practice of Human Factors
Understand the capabilities and limitations of
the operator
Understand the capabilities and limitations of
the system

• When will the system be used?
• How will it be used?
• Under what conditions will the system be used?
• Who are the likely users of the system?

5
System Designs and Human Performance Failures

• Failed to meet the expectations of users
• Too complex/ simple for users
• Too cumbersome to use in specific situations
• Failed to provide feedback to users
• Were untrustworthy for users
• Failed to assist information processing

System Failures
6
The Problem for Designers

• Lack of standards for product development
• Inability to predict user performance
• Reliance on probabilistic assessment
• Costs associated with human factors assessments

Failure to assess products from a human
performance perspective
Assumption that users will adapt to the potential
inadequacies of a system or product
7
Decision-Making Defined

• The selection of one choice from a number of
  choices
• Information is available with respect to the
  choice
• Time is available to make a choice
• The choice is associated with a level of
  uncertainty

8
Human Error and Decision-Making
Impetus to Mitigate Human Error
Information Error 23 Decision Error 42 Action
Error 35
(Hooey Foyle, 2006)
Decision-Making Training
System Design
9
Human Error and Decision-Making (contd)
Mistakes
Intentional Behaviour
Knowledge-Based
Increasing Conscious Awareness
Rule-Based
Slips/ Lapses
Unintentional Behaviour
Skill-Based
(Rasmussen, 1993 Reason 1991)
10
Approaches to the Analysis of Decision-Making
Normative Assessments of human decision-making
against rules
Prescriptive Assessments of human
decision-making using decision aids
Descriptive Assessment of expert human
decision-making
To understand the nature of human decision-making
To improve human decision-making
11
Normative Approaches to Decision-Making
Assumption that humans fail to account for the
base rates with which events occur
Rules of thumb that specify the relationship
between phenomena
Heuristics

• Representativeness
• Availability
• Confirmation bias

Mechanisms that reduce the demands on information
processing
12
Normative Approaches to Decision-Making (contd)
(Wickens, 1991)
13
Normative Approaches to Decision-Making (contd)
Representativeness Bias
Steve is very shy and withdrawn, invariably
helpful, but with little interest in people, or
in the world of reality. A meek and tidy soul, he
has a need for order and structure and has a
passion for detail. Which of the following do
you think is Steve's occupation? An Airline
Pilot A Farmer A Salesman A Librarian A
Physician
14
Normative Approaches to Decision-Making (contd)
Representativeness Bias (Categorisation)
20,000 Librarians
30 6,000
150,000 Farmers
10 15,000
Steve is more likely to be a farmer
15
Normative Approaches to Decision-Making (contd)
Availability Bias
Which of the following is more likely? (a) Dying
in Israel from a terrorist attack (b) Dying in
Israel from a car accident
In the English Language, which of the following
is more likely? (a) Words beginning with letter
R (b) Words in which the third letter is R
16
Normative Approaches to Decision-Making (contd)
Eddy (1982) Asked doctors to assume that they
had just examined a lump on a womans breast and
have ordered a mammogram.
Probability of malignancy is approximately 1 in
100, while the mammogram has an accuracy of 80
percent for malignant tumours and 90 percent for
benign tumours.
Probability of malignancy given a positive test
95 of respondents considered that the
probability of malignancy in this case was
approximately 75 (Eddy, 1982).
17
Normative Approaches to Decision-Making (contd)
Bayes Theorem Calculates the probability of an
outcome as a proportion of the total possible
outcomes (Fischoff Beyth-Marom, 1983)
Probability of cancer given a positive test
result is the probability of an accurate test
result calculated as a proportion of the total
probability of cancer, divided by the total
probability of an accurate test (either malignant
or benign)
18
Normative Approaches to Decision-Making (contd)
Choose between A A sure gain 240 B A 25
chance to gain 1000, and a 75 chance to gain
nothing
Expected Value A 240 Expected Value B
(.25)(1000) (.75)(0) 250
Choose between A A sure loss 750 B An 80
chance to lose 1000, and a 20 chance to lose 0
Expected Value A -750 Expected Value B
(.80)(-1000) (.20)(0) -800
19
Normative Approaches to Decision-Making (contd)
Prospect Theory
If the reference point is such that the outcome
is viewed as a gain, then decision-makers tend to
be risk averse. If the reference point is viewed
as a loss, then decision-makers tend to be risk
seeking (Kahneman Tversky, 1979)
Value Function
20
Normative Approaches to Decision-Making (contd)
Experts tend to be no more accurate than
novices in the accuracy of their assessments
Methodological issues
Simple rules are equally or more accurate than
experts in assessments under uncertainty.
What is an expert (Shanteau, 1991)
21
Normative Approaches to Decision-Making (contd)

• Humans are not particularly accurate in
  estimating the frequency of probabilistic events
• Humans prefer to use decision strategies that
  minimise the application of cognitive resources
• Humans often use intuition when making decisions.

22
Prescriptive Approaches to Decision-Making
Models or procedures that advocate the
application of cognitive resources to ensure that
all of the information available is acquired and
processed, and that the decision that is arrived
at is logical and rational.
Fire fighting
Contextual
Aviation
Finance
23
Prescriptive Approaches to Decision-Making
(contd)
Mathematical Formulae
Procedure
The accuracy of the outcome is priority
Assumption that this occurs through the
application of cognitive effort (resources)
Reminders
24
Prescriptive Approaches to Decision-Making
(contd)
Examples
Fire Fighting
DECIDE
D Detect that a change has occurred E Estimate
the extent of the change C Choose an appropriate
goal I Identify an appropriate strategy D Do E Eva
luate the outcomes
25
Prescriptive Approaches to Decision-Making
(contd)
Examples
Aviation
GRADE
G Gather the facts R Review the
information A Analyse the options D Decide on a
response E Evaluate the outcomes
26
Prescriptive Approaches to Decision-Making
(contd)
Examples
Subjective Expected Utility
Value-Based Theory
Satisfaction
Option A
Option B
Utility
Utility
Option with the greatest utility is selected
Weighted average of the relative satisfaction
associated with the features of each option
27
Prescriptive Approaches to Decision-Making
(contd)
Examples
Risk Analysis
Reason-Based Theory
Ratio of the frequency with which an event occurs
and exposure to the environment within which the
event occurs
50 instances of the event within a given
time-period Exposure (eg. Number of hours)
Where P denotes the probability of an event
28
Complete Exercise One
29
Descriptive Approaches to Decision-Making
Models that have been developed on the basis of
descriptions of decision-making in a range of
contexts.
Focus on expert decision-makers
Through experience, expert decision-makers have
developed the skills necessary for accurate and
efficient decision-making/
Ecological validity associated with the outcomes
30
Descriptive Approaches to Decision-Making (contd)
Novices
Experts

• Repertoire of experiences
• Recognise a situation
• Process information unconsciously
• Detailed understanding
• Diagnose a situation quickly
• Chunk information
• Can Improvise

• Few experiences
• Analyse a situation
• Process information consciously
• Limited understanding
• Unable to diagnose a situation
• Process information in sequence
• Cannot Improvise

Analytical (Knowledge-Based)
Intuitive (Skill-Based)
31
Descriptive Approaches to Decision-Making (contd)
Recognition-Primed Decision Theory
Experts possess a repertoire of exemplars/ cases
that are triggered by cues within the
environment.
The most similar case is considered in terms of
its application
Akin to intuitive reasoning
32
Descriptive Approaches to Decision-Making (contd)
Observation of Expert Fire Fighters (Klein, 1989,
1990)
Rehearsal

• Serial processing
• Recognition-driven
• Dependent on diagnosis
• Relatively rapid process

Look for Indicators
Look at the whole
Efficient
Proposes a goal that represents the
characteristics of expertise
33
Descriptive Approaches to Decision-Making (contd)
Cues represent signals or reminders that occur
within a particular context. They may be visual,
auditory, olfactory, or tactile, or
proprioceptive in nature.
In decision making, cues are presumed to act as a
trigger for diagnosis or a trigger for a response
34
Descriptive Approaches to Decision-Making (contd)
Steve is very shy and withdrawn, invariably
helpful, but with little interest in people, or
in the world of reality. A meek and tidy soul, he
has a need for order and structure and has a
passion for detail. Which of the following do
you think is Steve's occupation? An Airline
Pilot A Farmer A Salesman A Librarian A
Physician
35
Descriptive Approaches to Decision-Making (contd)
What are the features in this image that
immediately draw your attention?
36
Descriptive Approaches to Decision-Making (contd)
Heuristics
Majority of Confirming Decisions Options are
compared in pairs and the option with the
greatest number of features is retained (Mann
Ball, 1994)
Elimination by Aspects Features are ranked and
assessed across each option. Options for which
the features fall below a cut-off value are
discarded (Tversky, 1972)
Frequency Frequency of positive and negative
features for each of the options available
Satisficing Sequential comparison between
alternative options (Stokes et al., 1997)
37
Descriptive Approaches to Decision-Making (contd)
Majority of Confirming Decisions
Option A Feature 1 Feature 2 Feature 3
Option B Feature 1 Feature 2 Feature 3
Option C Feature 1 Feature 2 Feature 3
38
Descriptive Approaches to Decision-Making (contd)
Elimination by Aspects
Option A Feature 3 Feature 1 Feature 2
Option B Feature 3 Feature 1 Feature 2
Option C Feature 3 Feature 1 Feature 2
39
Descriptive Approaches to Decision-Making (contd)
Frequency
Option A Feature 3 Feature 1 Feature 2
Option B Feature 3 Feature 1 Feature 2
Option C Feature 3 Feature 1 Feature 2
40
Descriptive Approaches to Decision-Making (contd)
Satisficing
Option A Feature 3 Feature 1 Feature 2
Option B Feature 3 Feature 1 Feature 2
Option C Feature 3 Feature 1 Feature 2
41
Theories of Decision-Making Adaptive Decision
Theory
Accuracy
Cognitive Resources
Assumption that experienced practitioners possess
a repertoire of strategies (Payne et al., 1993)
The type of decision strategy employed is
dependent upon the need for accuracy and the
effort involved in achieving that level of
accuracy
42
Theories of Decision-Making Adaptive Decisions
Cognitive Effort
Analytical
Intuitive

• Bayesian
• Expected Utility

• Frequency Strategy
• Satisficing Strategy

• Recognition-Primed
• Recognition

43
Theories of Decision-Making Adaptive Decisions
Analytical versus Intuitive Strategies

• Most decision-makers use intuitive and
  quasi-analytical strategies spontaneously
• Intuitive strategies are subject to biases and
  therefore, can be unreliable
• Analytical strategies are generally more accurate
  than intuitive strategies
• Analytical strategies can be prescribed in a
  decision tree or linear sequence.
• Analytical strategies require the application of
  significant cognitive resources and
• Analytical strategies are time consuming

44
Theories of Decision-Making Compensatory
Decisions
Compensatory decision-making involves the
integration of information pertaining to
features, and trading-off the value of one
feature with another.
Weighted-additive models
The value of a feature is weighted and summed to
form an overall assessment of the feature.
There is no stopping rule for information search
45
Theories of Decision-Making Non-compensatory
Decisions
Non-compensatory decision-making involves
sequential assessments of options based on one of
a number of information search hierarchies.
Elimination models
The value of a feature is considered above or
below a threshold and an option is ruled in or
out on this basis
Incorporates stopping rules for information search
46
Theories of Decision-Making Aspiration
Adaptation
Choices in decision-making are driven by
aspiration, rather than utility
Satisfying these goals to the greatest extent
possible is perceived positively
A decision-maker has multiple goals
The aspiration level is a vector of the values of
each of the goals with which a decision is
associated.
(Selten, 2001)
47
Theories of Decision-Making Aspiration
Adaptation
New Aspiration Level
Alternative 2
Alternative 1
Previous Aspiration Level
(Goal vectors)
The search for options is concluded when the
alternative is feasible but transition to higher
levels of aspiration is not possible
48
Theories of Decision-Making Bounded Rationality
Bounded rationality is based on the principle
that behaviour that is regarded as non-optimal
may actually have a rational basis.
Radar target information Point of departure Time
of departure Route Context
Observe that military commander opened fire on an
aircraft based on the assumption that it was
descending to attack
Non-optimal
49
Theories of Decision-Making Bounded Rationality
(contd)
Despite the potential for inaccuracy, the
application of heuristics is rational, since it
affords the greatest probability of success for
the least time and cognitive load
Adaptive mechanism
Enables a likelihood of success in
time-constrained environments while releasing
cognitive resources for other tasks.
50
Data Acquisition Techniques and Decision-Making
Decision Error Taxonomies

• Failure to diagnose accurately
• Failure to select an appropriate goal
• Failure to select an appropriate strategy
• Failure to select an appropriate procedure

Diagnostic error
Goal Error
Strategy Error
Procedural Error
(OHare et al., 1994)
51
Data Acquisition Techniques and Decision-Making
(contd)
Cognitive Interview
What were the features of the situation that
indicated to you that you would need to make a
decision? (Situation Assessment) What was it
that you were looking for when you were deciding
how to approach the problem? (Expanded Perceptual
Network) Why was this information important for
your interpretation of the situation? (Goal
Structured Search) How did you know that this
information was important in your interpretation
of the information?
52
Data Acquisition Techniques and Decision-Making
(contd)
Cognitive Interview
Well usually, I like to see, as well as the
horizon, a difference in the colouring, say the
sun for instance. If I can see the sun, through
the other side, then I know that theres a hole
and that its not 8/8s, and there will be a way
to get up if I have to climb, without going into
cloud. But I hadnt seen that actual lightening
on the other side. In other words, I could see
that the clouds werent down on the hills. In
other words, I wasnt just going to fly into
clouds without knowing where the tops were. But
it wasnt like Ive experienced lots of times
where you are on the bad weather side of the
hills, and you go to the lee side of the hills
and you can also see sunlight.
53
Data Acquisition Techniques and Decision-Making
(contd)
Conceptual Mapping
54
Data Acquisition Techniques and Decision-Making
Critical Decision Method
Selection of a stimulus (event) that requires a
comparison between options, usually under a
time-constraint (not necessarily an emergency)
Rerouting trains following a fatality
Objective
Elicit those cognitive features of
decision-making performance that enable experts,
in particular, to perform successfully
Accuracy
Response Latency
55
Data Acquisition Techniques and Decision-Making
Process Tracing
The process of information acquisition is traced
and performance is inferred
56
Data Acquisition Techniques and Decision-Making
Eye Tracking
Implication that the frequency with which
information is interrogated and the time spent
interrogating the information represents a
window into the processing of information
during decision-making.
57
Data Acquisition Techniques and Decision-Making
Modelling
c
Decision Complexity
58
Decision-Making and System Design
Phenomena
Display
User
Does the symbology and representation assist or
impede decision-making?
Transformation of information from raw data to an
artificial representation
59
Decision-Making and System Design (contd)
Iconic Auditory Warnings
Abstract Auditory Warnings
Faster response time
Is poor decision-making a function purely of
cognition or the relationship between system
design and cognition?
Weather Radar
60
Decision-Making in Multi-Agent Environments
Biases in Joint Decision-Making
Groupthink The tendency to conform to the
perceived consensus of a group
Bay of Pigs
Risky Shift The tendency to accept greater risks
in a group than would be accepted as an individual
61
The Future

• We know some of the underlying features that
  prescribe the nature of human decision-making
• We know relatively less about how these features
  are impacted in a multi-agent environment
• We need to work towards understanding how
  decision-making is impacted by the representation
  of information

Do we need to change of basis of human
decision-making?
Do we change the nature of the environment to
improve decision outcomes?