David W. Roberts

1
Computational Models of Team Decision-MakingOve
rview a Proposed Model Structure

• David W. Roberts Christine M. Mitchell,
  Ph.D.
• Center for Human-Machine Systems Research
• School of Industrial and Systems Engineering
• Georgia Institute of Technology
• Atlanta, Georgia 30332
• Ph.D Candidate in Human-Machine Systems in the
  School of Industrial and Systems Engineering
• Research Scientist II, Georgia Tech Research
  Institute (GTRI), Georgia Institute of Technology
• dw.roberts_at_gtri.gatech.edu, http//dss.gtri.gat
  ech.edu [phone]
• Director and Professor, Center for
  Human-machine Systems, School of Industrial
  Systems Engineering
• Adjunct Professor of Computer Science,
  Georgia Institute of Technology
• Faculty-affiliate of Georgia Techs Graphics
  Visualization and Usability Center (GVU) and
  Cognitive Sciences program
• cm_at_chmsr.gatech.edu http//www.chmsr.gatech.
  edu [phone] fax [phone]

2
Overview

• Background
• Models of Human-Machine Systems
• Operator Function Model
• OFMspert
• Collaborative Decision Making in Air Traffic
  Management
• Preliminary ATM Team Model
• Extensions for Modeling Team Decision-Making
• Conclusions

3
Background

• Real-world applications, particularly in complex
  systems, often involve teams and team decision
  making.
• Developing intelligent training and aiding
  systems for teams requires robust computational
  models of multiple decision makers, including
  team members, the team as a whole, and the
  coordination among teams.
• Teams may be distributed in time and space.
• There may be multiple roles within a team or
  between teams with respect to overall team goals.
• To date, research on computational models has
  primarily addressed models of individual decision
  makers or operators (Pew Mavor, 1998)

4
Models of Human-Machine Systems

• Thus, robust computational models of groups or
  collaborating decision makers are few.
• The models that do exist are typically quite
  primitive and restricted in domain of application
  (Pew and Mavor, 1998)
• Flexible, scalable computational models of team
  decision making are urgently needed to meet the
  training and aiding needs of the armed forces.

5
Operator Function Model (OFM)Extending the OFM
to Teams

• Originated as a descriptive tool and knowledge
  repository to document and visualize operator
  interaction with complex dynamic engineering
  systems, essentially a cognitive task analysis
  methodology and structure to document cognitive
  and physical operational knowledge.
• Uses operational language of engineering systems
  and operators, goals are embedded in high-level
  activities (activities include functions,
  sub-functions, tasks, procedures).
• Though developed predominantly to model
  individual operators and decision-makers, the OFM
  has been used to model and design aids,
  assistants, and tutors for small teams.
• Two-pilot team comprising the crew of modern
  aircraft
• Two-person team controlling NASA near-earth
  satellites
• In several applications, the OFM was used to
  model teams comprised of both human and computer
  agents.

6
Operator Function Model (OFM) Properties

• Represents how operators organize system
  complexity (hierarchy).
• Represents the concurrent activities of operators
  in engineering systems and available operator
  choices (heterarchy non-determinism).
• Network of finite-state automata that includes
  both deterministic and non-deterministic state
  transitions.
• Nodes are operator activities at various levels
  of abstraction and aggregation. The lowest level
  nodes represent semantic operator actions.
• Actions can be of several types
• physical
• perceptual
• cognitive
• The domain is represented as state transition
  functions between activity nodes.

7
OFMspert Computational Implementation of the
OFM

• A Living Task Analysis
• Merges the OFM with the Stanford blackboard model
  of problem solving
• hierarchy
• heterarchy
• evolving hypothesis
• scalable architecture

8
(No Transcript)
9
OFMspert Properties Components

• OFM and OFMspert have been integrated into one
  computer-based modeling and run-time system
• OFM/OFMspert (1996) is both domain- (e.g.,
  aviation) and application-independent, (e.g.,
  training)
• Current Problem Space
• state variables
• derived variables
• workstation description
• Operations Model
• static OFM
• case-base of anomalies/plans
• initial plan
• Dynamic Operations Knowledge
• Currently active plan
• ACTIN (Actions Interpreter)

10
Operator Function Model (OFM) Syntax (1996)

• Example of OFM for pilot response to directive
  from ATC to turn to assigned heading.
• The OFM is a normative model of operator behavior
• Properties
• Decomposition
• AND (do all of these)
• OR
• XOR (exactly one)
• SEQ (sequence)

11
Merged OFM/OFMspert Methodology (1996)
12
OFM/OFMspert Applications
13
Assumptions Underpinning theOFM/OFMspert
Methodology

• The OFM is an engineering, as opposed to a
  psychological, model. It is an engineering
  cognitive task analysis. It can anticipate with
  high accuracy when and what operators do in
  carrying out nominal and off-nominal system
  functions.
• Like GOMS, the OFM is intended to model behavior
  of well-trained, well-motivated operators or
  decision makers carrying out activities for which
  normative behavior is rewarded. Given overall
  mission goals and current system state, behavior
  is very predictable within some envelope of
  choices.
• In such situations, the authors of the GOMS model
  suggest that the models are hardly psychology at
  all, but rather an analysis of the domain (p. 10,
  Human-Computer Interaction, Card, Moran,
  Newell).

14
OFM/OFMspert Research The Future

• Each of these OFM/OFMspert applications has been
  implemented in proof-of-concept form for
  high-performance dynamic engineering systems and
  empirically evaluated .
• Without exception, systems designed with the OFM
  or derived from the OFMspert architecture,
  consistently and significantly enhanced overall
  system performance as well as individual operator
  performance. Moreover, for many measures of
  performance the subject effect was
  insignificant. Thus OFM/OFMspert based systems,
  especially used in training, reduce
  student-to-student variance.
• The OFM/OFMspert methodology derives its face
  validity by domain practitioners to help define
  model requirements using a bottom-up methodology.
  The methodology depends extensively on on-site
  observation of operations to develop and
  subsequently validate the model.
• The visual form of the OFM is a major advantage
  that enables domain practitioners to
  interactively validate the model.

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
Extensions for ModelingTeam Decision-Making

• As the OFM/OFMSpert methodology has been used for
  small teams, there is evidence to suggest that
  the methodology can be extended to larger and
  more diverse teams.
• The OFM/OFMspert methodology has been used to
  model and provide the foundation for simulator
  design, and the design of intelligent tutors and
  aids for teams.
• Small, two-person teams (two-pilot cockpit
  two-person satellite control)
• Typically identical or very similar visions of
  the mission
• Tightly coupled in space
• Given the success of initial team models, the
  next step is to define formal OFM/OFMspert
  extensions to model teams that are more complex.

31
Collaborative Decision-Making In Air Traffic
Management Proof-of Concept
32
Air Traffic Management

• Realistic distributed systems are critically
  dependent on effective decision making (Orasanu
  et al., 1997)
• Air traffic management is a tightly coupled
  system comprised of three primary components.
  These components collaborate to keep aircraft
  flowing safely and efficiently through the
  system
• Pilots (on flight deck)
• Dispatchers (in the Airline Operations Center)
• Air Traffic Controllers (in enroute and Terminal
  Area Control or TRACON centers)
• The relationship between the pilot and ATC is
  well-defined.
• Given new technology, the relationship between
  the pilot and the dispatcher can be designed to
  enhance safety, while increasing traffic flow.

33
Air Traffic Management

• The relationship between dispatchers and pilots
  is less defined and open to knew ideas.
• By law, however, all routing decisions for major
  airlines must be made collaboratively between
  pilot-in-command and dispatcher
• New concepts require the development of models of
  air traffic management and its associated team.
  Given these models various potential scenarios
  can be implement and evaluated.
• OFM/OFMspert extensions for more diverse teams
  will use air traffic management as a proof-of
  concept application.
• Initial efforts suggest that air traffic
  management is a robust environment in which to
  explore the strengths and limitations of
  OFM/OFMspert as a computational team model

34
Preliminary ATMCollaboration Model

• Model depicts two team members, each with
  different roles
• First level decomposition for the dispatcher
• Monitoring sources of available information
• Flight planning
• Flight re-planning when unanticipated events occur

35
Preliminary ATMCollaboration Model

• Pilot-in-command model is only a partial
  representation of pilot activities
• Represents activities that relate to
  collaboration with the dispatcher
• Maintain current course (active--green)
• Deviate from planned route (inactive--with an
  initiator for ATC reroute

36
(No Transcript)
37
(No Transcript)
38
OFMspert Team ModelInitial Insights

• Team members that have different roles are likely
  to require different model representations,
  depending on the collaboration or team decision
  scenario being modeled.
• Existing OFM/OFMspert models of pilot navigation
  look significantly different than the model shown
  here.
• Pilot navigation models include much more detail
  for how to navigate under various state
  conditions.
• While the pilot role in this model is typically
  event-driven, the dispatcher role includes some
  activities that are on-going as well as some that
  are event-driven.
• The current OFM/OFMspert model requires
  extensions to support multiple types of
  activities, e.g., event-driven and on-going, in
  the same model.

39
Conclusions

• This modeling exercise provides initial evidence
  that the OFM/OFMspert methodology holds promise
  as a foundation for a robust computational team
  decision making model.
• Strengths Represents
• Multiple team members with very different roles
• Collaboration among multiple team members
• Weaknesses Extensions needed to easily support
• Different roles
• Different activity types
• This exercise shows that modeling and run-time
  use of the OFM/OFMspert methodology require
  extensions to represent the diversity and
  activities endemic to teams.