Army Research Lab Presentation

1
Modeling Teamwork and Command and Control in
Multi-Agent Systems Thomas R. Ioerger Department
of Computer Science Texas AM University
2
TaskableAgents Architecture

• similar to process networks, or HTNs
• has a customized knowledge representation
  language (TRL) for encoding knowledge about tasks
  and methods (doctrine, mission)
• agents run as independent processes
• each may have multiple parallel activities
• agents represent staff positions (S2, S3...)
• communicate with each other for teamwork
• interact with humans (via forms info/cmds)
• interact with OTB for scenario simulation

3
Simulation
OneSAF Testbed
DIS
Periodic Updates From Simulation
Cache
Agents
KB
PDUs Translated To Facts (speed, location, unit
type, etc.)
4
High-Level Architecture of DBST
PDUs
OTB
Agents
actions
inform, request, direct, approve, respond
PDUs
RFS, CFF
mouse
Puckster Interface
BDE Interface
text, forms, map
5
TaskableAgents Architecture

• Written in Java
• TRL Knowledge Representation Language
• - For Capturing Procedural Knowledge (Tasks
  Methods)
• APTE Method Selection-Algorithm
• - responsible for building, maintaining, and
  repairing task-decomposition trees
• Inference Engine JARE
• - Java Automated Reasoning Engine
• - Knowledge Base with Facts and Horn Clauses
• - back-chaining (like Prolog)
• - Updating World With Facts

6
TaskableAgents
TaskableAgents
TRL Task Decomposition Hierarchy
assert, query, retract
APTE Algorithm
TRL KB tasks methods
JARE KB facts Horn-clauses
Process Nets
results
messages
sensing
messages
operators
OTB (simulation)
7
Task Representation Language (TRL)

• Provides descriptors for goals, tasks, methods,
  and operators
• Tasks what to do
• Can associate alternative methods, with
  priorities or preference conditions
• Can have termination conditions
• Methods how to do it
• Can define preference conditions for alternatives
• Process Net
• - Procedural language for specifying how to do
  things
• - While loops, if conditionals, sequential,
  parallel constructs
• - Can invoke sub-tasks or operators
• - Semantics based on Dynamic Logic
• Operators lowest-level actions that can be
  directly executed in the simulation environment,
  e.g. move unit, send message, fire on enemy
• Each descriptor is a schema with arguments and
  variables
• Conditions are evaluated as queries to JARE

8
Example TRL Knowledge
(Task Monitor (?unit) (Term-cond (destroyed
?unit)) (Method (Track-with-UAV ?unit)
(Pref-cond (not (weather cloudy)))) (Method
(Follow-with-scouts ?unit) (Pref-cond
(ground-cover dense))))   (Method Track-with-UAV
(?unit) (Pre-cond (have-assets UAV))
(Process (seq (if(cond(not(launched
UAV)))(launch UAV)) (let((x y)(loc ?unit ?x
?y))(fly UAV ?x ?y)) (circle UAV ?x ?y))))
9
Alternative Methods

• useful for describing multiple ways of
  accomplishing tasks
• may encode preference conditions
• APTE algorithm will automatically try another if
  one method fails
• examples
• use of UAV vs. ATK helicopters vs. scouts for
  recon
• suppression of direct fire with Arty/CAS
• use of FASCAM to slow or re-direct advancing
  enemy
• maintaining security
• flank guard, patrols
• neighboring units
• use of terrain features
• electronic surveillance

10
Task-Decomposition Hierarchy
level 1

T1



level 2

M1



T3

level 3


T2 T5





T4


level 4

M7 M12 M92
M60


level 5

T40

T15 T18

T40 T45

T2

C

T45

Tx Task Mx Method C Condition
11
TOC Staff - Agent Decomposition
Maintain friendly situation, Maneuver sub-units
Control indirect fire, Artillery, Close
Air, ATK Helicopter
S3
FSO
Maintain enemy situation, Detect/evaluate
threats, Evaluate PIRs
S2
CDR
Move/hold, Make commands/decisions, RFI to
Brigade
Companies
Scouts
Maneuver, React to enemy/orders, Move along
assigned route
Move to OP, Track enemy
12
S2 Agent and Interactions
Enemy info
BDE/DIV Sensors/ Recon
BDE S2
RFI/RFS SALT/ INTSUM
intel
intel
Move to OP
S2
Scout
spot reports
Threat level, PIRs
CDR
S3
CCIR
DP approval
13
Vignette 2 Decision Point 1 Shift Main Effort
Company Size forces
Company Size forces
IB
AA5
AA5
1-235
234
3-234
3
235
IB
1
234
1
1-234
Company Size forces
AA5a
AA5a
AA3
AA4
AA3
AA4
AA5c
PL
PL
AA6c
Main Effort(ME)
ME Switch
Main Effort(ME)
3-661-22
3-661-22
PL
PL

• 2 Companies of 234 heading along AA3 AA4
• 3-234 lead Bn of 234 Regt intent is unclear.
• Lead Bn (1-235) of 235 Regt on AA5a

• 2 Companies of 3-234 heading along AA3
• 3-234 lead Bn of 234 Regt.
• Situation unclear on AA5

INTEL
INTEL
DP1
DP1
Variation A Enemy major threat is on main route
(AA) as in route (planned AA).
Variation B Enemy major threat changes to
secondary approach
Shift ME from Tm B to Co C?
Shift ME from Tm B to Co C?
N
Y
Y
N
14
Vignette 3 Decision Point 2 Commit TF Reserve
Company Size forces
Company Size forces
IB
AA5
AA5
234
3-234
3
235
1-235
IB
1
234
1
1-234
Company Size forces
AA5a
AA5a
AA3
AA4
AA3
AA4
Blocking Positions
AA5c
TAI
PL Y
PL
AA6c
Main Effort(ME)
(-)
Main Effort(ME)
3-661-22
3-661-22
()
PL
PL

Res

• 2 Companies of 234 heading down AA3
• Uniform pressure on AAs 4 5ar.
• Calculations indicate Tm A unit can move
• to PL Y prior to lead of enemy unit.

• Company units of 3 different Bns on all 3 AAs
• Estimate enemy will reach PL Y at same time
• 238 Regt lead units not committed

INTEL
INTEL
DP2
DP2
Variation A Heavy enemy threat across across
entire sector.
Commit the TF Reserve platoon?
Commit the TF Reserve platoon?
Variation B Major enemy movement on one avenue
(AA).
Y
N
Y
N
Go to DP 3
15
Modeling Teamwork

• TOC is more than just collection of staff members
• helping/backup behavior, information fusion,
  resource sharing, and joint decision-making
• how to model collaborative behavior?
• CAST (extension to TaskableAgents)
• adds features to TRL language for encoding team
  structure and process (MALLET)
• adds algorithms for coordination and
  communication within teams
• semantics based on joint intention theory and
  mutual awareness (beliefs)

16
Key Concept Shared Mental Models

• Various components
• static structure of the team, communication
  policies...
• goals and plans
• dynamic current situation, others
  workloads/status
• Team knowledge needed by agent team members
• roles, responsibilities, capabilities, team plans
• need to know who should act and when
• need to reason about each other
• need to know when to communicate for
  synchronization, coordination, disambiguation,
  infomation sharing, etc.

17
The CAST Agent Architecture

• MALLET - team knowledge repres. language
• team structure (roles, capabilities,
  responsibilities)
• team process (plans, policies)
• CAST kernel (interpreter)
• convert to Petri nets (track progress, select
  actions)
• use back-chaining theorem-prover for inference
• dynamic role selection - make choices in context
• DIARG - information exchange algorithm
• proactive offer new info to those who need it
• primary references (Yen et al., IJCAI, 2001),
  (Yin et al., Autonomous Agents
  Conf., 2000)

18
MALLET Multi-Agent Logical Language for Encoding
Teamwork

• extension of TRL
• basic definitions
• (team search-and-rescue (bill ted))
• (role pilot) (role spotter)
• (plays-role bill pilot)
• (capable spotter use-IR-binoculars)
• conditions (ltpredicategt) with variables
  prefixed by ?
• e.g. ((forward-scout ?unit) (location ?unit ?x
  ?y))
• team operators
• (team-oper lift-heavy-object (?obj)
• (pre-cond (at ?obj) (num-agents gt 2))
• (share-type AND)))
• share types ANDtogether, ORany, XORonly 1
  (excl.)

19

• team plans can select certain agents or roles to
  do steps (like SharedPlans of Kraus and Grosz)
• (team-plan indirect-fire (?target)
• (select-role (scout ?s)
• (in-visibility-range ?s ?target))
• (process
• (do S3 (verify-no-friendly-units-in-area
  ?target))
• (while (not (destroyed ?target))
• (do FSO (enter-CFF ?target))
• (do ?s (perform-BDA ?target))
• (if (not (hit ?target))
• (do ?s (report-accuracy-of-aim FSO))
• (do FSO (adjust-coordinates ?target))))))
• compile these into TRL using methods of Biggers
  and Ioerger (2001)
• other scouts can take over as backup in case of
  failure of ?s
• responsibilities (such as monitoring, reporting)
  semantics similar to joint intentions (Johnson
  and Ioerger, 2001)

20
CAST Kernel

• compile team plans into Petri nets (expand
  sub-tasks)
• cycle sense/decide/act loop
• 1. update beliefs about environment in selfs KB
• 2. check for any incoming messages from other
  agents
• 3. find active steps in plan (transitions with
  tokens in all input places)
• 4. if self is uniquely resp., consider executing
  oper.
• 5. if oper is XOR and resp. is ambiguous, offer
• 6. if oper is AND, broadcast READY and wait for
  others
• 7. randomly choose among remaining actions and
  execute
• 8. inform others of completed steps
• Dynamic Role Selection (DRS)
• check role definitions, must satisfy any
  constraints, capable?
• communicate when ambiguity exists
• sync. for AND operators select for XOR operators
• could also allow individuals to vote/negotiate

21
DIARGDynamic Inter-Agent Rule Generator

• Info. sharing is a key to flexible teamwork
• Want to capture information flow in team,
  including proactive distribution of information
• Want to restrict to only the most relevant cases
• Ideal criterion
• (Bel A I) (Bel A ?(Bel B I)) (Bel A (Goal B
  G)
• ?(Bel B I) ? ?(Done B G)
• (Bel B I) ? ? ?(Done B G)
• ? (Goal A (Inform B I))
• where is the temporal operator for always

22
DIARG, continued

• Explanation - A should send message I to B iff
• A believes I is true
• A believes B does not believe I (or believes it
  is false)
• I is relevant to one of Bs goals
• i.e. pre-cond of current action that B is resp.
  for in team plan,
• and that action would not succeed without knowing
  the info.
• Reasoning about observability
• agents can sometimes infer that other team
  members already believe certain information
• e.g. based on common observability in environment
• use this to filter out superfluous messages
• recent work (Rozich and Ioerger, submitted)

23
Command and Control

• Need for tactical decision-making
• more flexibility in unplanned situation
• commander agent
• How to represent of tactics?
• battlefield geometry, relative force strength,
  combined arms theory
• terrain, effects on mobility
• discovery of enemy intent

24
Naturalistic Decision-Making

• NDM (Klein) - a cognitive model of human
  decision-making in complex environments
• based heavily on situation assessment (SA)
• 3 stages (Endsley)
• acquisition of factual information
• comprehension (abstraction, relevance, goal
  impact)
• projection (prediction of consequences)
• NDM is satisficing
• take first adequate match dont extensively
  evaluate and compare alternatives respond

25
Evidence for NDM in TOCs

• verbal protocol analyses
• characterize types of utterances and interactions
• representative studies
• Serfaty, Entin, et al. (NDM, 1997)
• expertise as independent variable
• Pascual and Henderson (NDM, 1997)
• reliance on recall from experience
• Schmitt and Klein (CCRTS, 1999)
• recognitional processes in MDMP/COA
• Endsley (ARL report)
• information flow in infantry platoon/urban combat
• lots of other similar C2 environments...
• CIC/AAW, AWACS, fire fighting, ATC

26
Recognition-Primed Decision Making (RPD)

• a model of NDM
• characterized by feature-matching
• look for enough cues to trigger recognition
• features could be weighted
• for each situation, there is a typical response
  (doctrinal, or learned from experience)
• role of mental simulation?

27
RPD Flow Chart

• Len Adelman and Denny Leedom
• integrate RPD within battalion TOC staff

monitor progress
current mission plan
generate response, mental sim, compat.
test, modify plan ...
Yes
situation clear?
modify
No
status quo acceptable?
generate new options
No
Yes
reduce uncertainty
28
Methods to Deal With Uncertainty

• often features are unknown, cant evaluate
• options include
• 1. suppress uncertainty
• 2. make default assumptions
• 3. confirmation bias (expectations)
• 4. take probing actions
• 5. forestalling until situation is more clear

29
Implementation of RPDin TaskableAgents

• task DetectSituation runs in parallel with other
  routines...
• loop until match enough features for one
  situation
• while some features are unknown, try various
  methods to findout (e.g. UAV, scouts, radar,
  JSTARS, Bde Int, feint...)
• drive information collection to discriminate
  situations
• trigger plans for response maneuvers once
  situation is determined (takes priority)
• go back to original mission once threat is
  handled
• all of this can be implemented as tasks and
  methods in TRL

30
Characterization of Situations

• Situations must have lists of typical features
• Characterize by
• location of nearby enemy, combat strength
• in regions relative to local axis (frame of ref.)
• distance, speed, direction
• intent? (e.g. attacking, bypassing, objective)
• cover, uncertainty
• effect of terrain on mobility/reachability
• roads, mountains, streams, bridges, marshes,
  forests
• also minefields, targeted areas of interest...

31
Types of Situations

• Defensive
• getting flanked, ambushed, enveloped, etc.
• response shift, request for support, withdraw...
• over-whelming enemy maneuver/main effort
• response impede, divert, CAS, inform Bde...
• forked maneuvers (intent?) bypass attempt
• Offensive (opportunities worth recognizing)
• exploit gaps, isolate enemy units, envelopment,
  use fixing force flank attack, canalize enemy,
  bypass

32
Practical Issues

• Managing priorities
• when to abandon mission plan react tactically?
• return to mission plan once done?
• Dependence of response on goals
• ROE, aggression/initiative vs. defense/security
• should SA involve impact on goals? ( threat?)
• Need to have critics to revise responses
• avoid enemy TAIs, minefields, contact...
• stay near adjacent friendly units, air defense...

33
Conclusion

• TaskableAgents architecture
• CAST extensions for teamwork
• TOC staff agent model
• modeling command and control (on-going)
• HLA interoperability (on-going)