Analyses of Selective Military Sensemaking Cases

1
Analyses of Selective Military Sensemaking Cases

• Prepared for
• CCRP Sensemaking Symposium
• 23-25 October 2001

Prepared by Evidence Based Research,
Inc. Richard E. Hayes, Ph.D. Stevana M. Allman
John R. M. McDaniel Cara N. Christie Telyvin
F. Murphy Eric J. Cochrane Kristina M.
Thompson Stacey R. Lakind William S. Wood
2
Purpose

• Exploratory empirical analysis of military
  Sensemaking in order to shed light on
• which parts of the process tend to go well
• which parts of the process tend to go badly
• Examination of the military Sensemaking process
  to identify elements associated with success and
  failure

3
Sensemaking Conceptual Framework
Judgment
Decision processes
Cognitive Domain

• Planning
• Missions
• Assets
• Boundaries
• Schedules
• Contingencies

• Emotions
• Physiological Factors
• Beliefs
• Perceptions

Information Domain

• Directives
• Requests for support
• Queries
• Reports
• Efforts to consult

Decision Support Models Tools
Information (data in context)
Data (representation)
Synchronization
Physical Domain
Objects/events
Actions
4
Diagnosing Sensemaking

• Decisions
• Choices among alternatives
• including contingent choices
• Choices to wait
• Choices to seek information
• Choices to consult others

Was shared awareness of the situation correctly
understood?
Judgment
Was command intent developed collaboratively?
Was Sense made of the situation?
Decision processes
Were emotions, beliefs and cognitive factors
taken into account?
Cognitive Domain
Was the SA shared?

• Emotions
• Physiological Factors
• Beliefs
• Perceptions

• Planning
• Missions
• Assets
• Boundaries
• Schedules
• Contingencies

Did the individuals develop appropriate Situationa
l Awareness (SA)?
Was a quality plan developed?
Were the appropriate models and tools used?
Information Domain
Was it put in a form that facilitates awareness?

• Directives
• Requests for support
• Queries
• Reports
• Efforts to consult

Information (data in context)
Were the decision and driving factors shared?
Was it put together appropriately? (correlation,
context)
Data (representation)
Was the right data collected?
Synchronization
Physical Domain
Objects/events
Was the plan executed effectively?
Actions
5
Situations Examined
A total of 30 situations were examined

• Historical Military
• Waterloo (1815)
• Stonewall Jacksons Valley Campaign (1862)
• Lee at Chancellorsville (1863)
• Gettysburg (1863)
• Jutland (1916)
• The Battle for France (1940)
• Battle of Britain (1940)

• Pearl Harbor (1941)
• Sicily (1943)
• Anzio (1944)
• Normandy (1944)
• Battle of the Bulge (1944)
• Leyte Gulf (1944)
• Monte Cassino (1944)
• Vietnam Tet Offensive (1968)

6
Situations Examined (cont.)

• Contemporary Military
• Vincennes shootdown of an Iranian airliner (1988)
• F-15 shootdown of US helicopters over Northern No
  Fly Zone (1994)
• Scott OGrady shootdown over Bosnia (1995)
• Insurgencies
• Mogadishu Ranger raid (1993)
• Grozny (Chechnya) (1995)
• Grozny (2000)

• Terrorism
• King David Hotel bombing (1946)
• Munich Olympics (1972)
• Entebbe raid (1976)
• Beirut Marine barracks bombing (1983)
• Khobar Towers bombing (1996)
• Attack on Japanese embassy in Peru (1996)
• African US embassy bombings (1998)
• USS Cole bombing (2000)
• September 11 terrorist attacks (2001)

7
Analytic Groupings

• From these 30 situations, 149 specific decisions
  and decision processes were identified
• Classification of decisions and decision
  processes
• 93 historical military
• 10 contemporary military
• 8 insurgency
• 38 terrorism
• The 149 specific decisions were also grouped
  according to the success or failure of the
  decision
• 60 successes
• 89 failures

8
Research Issues
Fifteen Sensemaking elements were grouped into 9
decision process groups

• Information System Inputs
• Was the right data collected?
• Was it put together appropriately (correlation,
  context)?
• Was it put in a form that facilitated awareness?
• Situational Awareness
• Did the individuals develop appropriate SA?
• Was the SA appropriately shared?
• Cognitive Factors
• Were emotions, beliefs, and cognitive factors
  taken into account?
• Was prior knowledge incorporated and used?
• Were mental models used?

• Understanding
• Was shared awareness of the situation correctly
  understood?
• Sensemaking
• Was Sense made of the situation?
• Decision Effectiveness
• Was the chosen decision one that worked?
• Command Intent
• Was command intent developed collaboratively?
• Plan
• Was a quality plan developed?
• Execution
• Were the decisions and driving factors shared?
• Was the plan executed effectively?

9
Scaling

• Each element in each decision process was
  initially scored on a five-point scale
• 5 - Yes, thoroughly
• 4 - Quite well, but not thoroughly
• 3 - Average, basics were covered
• 2 - Some, but not sufficiently
• 1 - No, not at all

However the Cognitive Inputs were not reliably
reported even in excellent sources. Therefore,
the importance of these factors was rated, not
their correctness in the cases.
5
1
Very Important
Not Important
10
Cognitive Inputs
To what extent were emotions, prior knowledge,
and mental models important?

• x s
• 4.25 1.00
• 3.87 1.34
• 3.19 1.42

• Emotions, beliefs, cognitive factors
• Mental Models
• Prior knowledge

• Strong indication that cognitive inputs were
  important across all examples
• Prior knowledge was relatively less important
  than
• emotions, beliefs and cognitive factors
• mental models

11
Best Elements

• x s
• 3.64 1.35
• 3.37 1.45
• 3.37 1.46
• 3.31 1.41

3.28 1.08

• Average over all issues
• What tended to go right?
• Right data collected
• Data put together appropriately
• Decision and driving factors were shared
• Information was put in a form that facilitated
  awareness

Information systems generally tended to do their
job
Significantly different from the overall mean
12
Worst Elements
x s 2.57 1.32 2.78
1.65 2.90 1.65
3.28 1.08

• Average over all issues
• What tended to go wrong?
• Collaboration
• Effectiveness of decision
• Effectiveness of execution

Significantly below the overall mean
13
Typical Elements
x s 3.17 1.51 3.17
1.38 3.15 1.58 3.15 1.49 3.07 1.57
3.28 1.08

• Average over all issues
• SA appropriately developed
• SA appropriately shared
• Quality plan developed
• Shared awareness correctly understood
• Sense was made of the situation

14
What Most Strongly DifferentiatedSuccesses From
Failures?

• Did the individuals develop appropriate SA?
• Was Sense made of the situation?
• Was the data put together appropriately?
• Was the information put in a form that
  facilitated awareness?
• Were the decision and driving factors shared?
• Was shared awareness correctly understood?

xsuc- xfail 1.86 1.82 1.70 1.69 1.59
1.58
15
Conclusions

• Cognitive inputs are important elements and must
  be addressed for successful military Sensemaking
• Best overall performance tended to be things
  within the information systems, or supported by
  the information systems
• Weakest elements tended to be collaborative
  processes and ability to synchronize
• Differentiation between success and failure
  tended to focus on situation awareness,
  understanding, Sensemaking, and sharing decisions
  and driving factors
• Analysis continues...