Integrating a Distributed AgentBased Simulation into an HLA Federation

1
Integrating a Distributed Agent-Based Simulation
into an HLA Federation

• Gary Kratkiewicz
• Amelia Fedyk
• Daniel Cerys

2
Purpose of this Session

• Understand how to
• Use distributed multi-agent systems, with a
  specific emphasis on the Cougaar architecture
• Integrate a distributed multi-agent simulation
  into an HLA federation
• Design a full-scale Cougaar society for
  integration with an HLA federation

3
Outline

• Motivation and Goals
• Distributed Agent Computing and Cougaar
• Integration Approach
• Lessons Learned
• Design considerations for Cougaar societies
• Conclusion

4
Motivation

• Military combat simulations lack sophisticated
  logistics components
• Logistics support is critical to combat
  operations
• Cougaar-based logistics simulations provide
  capabilities not available in current military
  simulation systems
• Roadblock lack of interoperability

5
Purpose

• Demonstrate the ability of Cougaar societies to
  interact with other simulations via HLA
• This would allow the following
• Credible proposals of simulation systems based on
  Cougaar-HLA linkages
• The addition of realistic simulation of logistics
  to combat simulations

6
Technical Goals (1)

• Demonstrate that a Cougaar-based society of
  agents can act as an HLA federate
• Show that Cougaar time mechanisms can integrate
  with HLA time synchronization
• Confirm that a Cougaar society can work with a
  variety of RTIs
• HLA 1.3 and HLA 1516
• Fully asynchronous and partially asynchronous
• Java and non-Java based

7
Technical Goals (2)

• Develop techniques for creating or modifying
  Cougaar societies to be HLA federates
• Identify
• Further areas of research
• Additional functionality required in a full-scale
  Cougaar HLA federate

8
Outline

• Motivation and Goals
• Distributed Agent Computing and Cougaar
• Integration Approach
• Lessons Learned
• Design considerations for Cougaar societies
• Conclusion

9
Agents

• Agents are independent software entities that
  react to events and initiate actions by
  themselves
• Different agent-based architectures yield agents
  with varying capability levels
• Agents can have or define roles, tasks, beliefs,
  desires, or intentions
• Agents can be static or mobile
• Some agents or agent systems are considered
  intelligent

10
Distributed Agent Computing

• A distributed system is a collection of separate
  processes or information systems that can act
  together as a single system
• Distributed agent computing involves agent-based
  systems that operate in a distributed manner
• Purpose
• Implement complex behavior
• Model or simulate complex systems

11
Cougaar

• Cognitive Agent Architecture
• Java-based architecture
• Rich design can model diverse objects and
  interactions
• Agents are Cougaar components with a defined
  functionality and local memory store (Blackboard)
• The behavior of the agent emerges from the
  composite of plugins

12
Cougaar

• Blackboard(BB) implements a publish/ subscribe
  API
• Plugins view objects on the BB by creating
  subscriptions
• Logic Providers are lightweight agent components
  responsible for messaging and BB modifications
• A collection of agents make up a cougar society

13
Cougaar
14
UltraLog

• 4 year project sponsored by DARPA
• Layer built on top of Cougaar Architecture
• Developed to model military logistics within a
  distributed multi-agent system
• Added Security, Robustness and Adaptivity to the
  cougar infrastructure.
• Test society models interaction between large set
  of military organizations.

15
UltraLog

• Society runs in strictly planning mode(predictive
  solution) and execution mode(simulated solution)
• Solution includes detailed transportation and
  supply chain plan
• Society capable of garbage collection of
  completed/stale BB objects

16
FCS Supportability

• Future Combat Systems (FCS) Supportability
  extends Ultralog Functionality
• Agents designed to reduce the logistics footprint
• Additional business logic added to plugins to
  model highly mobile, self-sufficient units.
• Plugins enhanced to accept 3rd party simulation
  data

17
Outline

• Motivation and Goals
• Distributed Agent Computing and Cougaar
• Integration Approach
• Lessons Learned
• Design considerations for Cougaar societies
• Conclusion

18
HLA Overview
Simulations
Data Collectors
Interfaces to Live Players
Simulations
Data Collectors
Interfaces to Live Players
Simulations
Data Collectors
Interfaces to Live Players
Simulations
Data Collectors
Interfaces to Live Players
Interface
Runtime Infrastructure (RTI)
19
Federation and Society Design

• Created small UltraLog logistics society
• Pared down to model only bulk fuel consumption
• 2 fuel-consuming organizations (agents)
• 18 other organizations in supply chain and chain
  of command
• Two society federates based on above
• High-fidelity logistics society for general
  simulation
• Demand generation society to model fuel demand

20
Prototype Society
21
Non-Cougaar Federate

• Performed verification with Java-based,
  non-agent, non-Cougaar federate
• Demand generation society to model fuel demand

22
RTI Selection

• Tested prototypes with following HLA RTIs
• Pitch 1516 LE
• Pitch 1.3 LE
• DMSO NG 1.3
• Demonstrated the feasibility of using Cougaar
  with
• HLA 1.3 and IEEE 1516 RTIs
• Fully asynchronous (Pitch) and partially
  asynchronous (DMSO) RTIs
• Java-based (Pitch) and non-Java (DMSO) RTIs

23
Simulation Models

• Cougaar time
• Continuously advances with real (wall clock) time
• Can be advanced ahead to some future time
• Mapping to simulation model
• Advance time with real time or ahead in equal
  steps ? time-stepped simulation model
• Advance time in unequal steps ? event-driven
  model
• Most Cougaar simulations run for hours and
  simulate operations over months
• Equal steps ? scaled real-time
• Unequal steps ? non-real-time

24
Time Management

• Cougaar time
• Initialized to physical time
• Advances at same rate as wall clock
• Can be stepped to future point in time
• Society should be in quiescence before stepping
• Cougaar and HLA time synchronization
• None not useful for simulation, OK for planning
• Conservative appropriate
• Optimistic not appropriate Cougaar cannot roll
  back time
• OK since different federates can use different
  synchronization methods

25
Iterative Development Approach

• Build a federation without HLA time management
• Two federates
• Allowed us to get running quickly
• Build a federation with HLA time management
• Two federates
• Added RTI time synchronization
• Build a federation with a multi-node Cougaar
  society
• Three federates total
• Two federates map to one society

26
Phase 2 HLA Time Management
27
Phase 3 Multi-Node Society
28
Results of Integration Experiments

• Demonstrated interoperability between Cougaar and
  HLA RTIs in various combinations
• Demonstrated various operations
• Subscribing a Cougaar federate and plugins to
  specific HLA interactions
• Interfacing a Cougaar society to the RTI via a
  single ambassador class
• Interfacing a Cougaar society to the RTI via a
  Cougaar service
• Synchronizing society time via the HLA mechanism
  using conservative synchronization
• Detecting tasks in one society, transferring the
  task info via HLA interactions, reconstituting
  the tasks in a second society
• Learned techniques and identified design
  considerations

29
Outline

• Motivation and Goals
• Distributed Agent Computing and Cougaar
• Integration Approach
• Lessons Learned
• Design considerations for Cougaar societies
• Conclusion

30
Interfacing Between Society and RTI

• Singleton ambassador class
• Separate ambassador class per RTI
• Performs following actions
• Instantiates the RTIs ambassador class
• Joins the federation execution (creating it first
  if necessary)
• Connects to interactions and objects
• Handles the interactions and object attributes
  from the federation
• Distributes them to the appropriate agent in the
  society
• Later moved ambassador class to Cougaar service
• Separate class instantiated per Cougaar node

31
Time Synchronization Phase 1

• Used a servlet-based, user-directed time
  advancement mechanism
• Allowed us to operate with a variety of
  simulation models
• Used for all phases
• Initially implemented without HLA time management
• For development purposes only
• Advanced time in high-fidelity society
• Time advance passed to demand generation society
  via interactions
• Small society always ready

32
Time Synchronization Phase 2

• Conservative time synchronization
• Standard HLA mechanism
• Modified version of the user-directed Cougaar
  time advancement mechanism.
• Cougaar mechanism modified
• Obtains permission from the RTI before advancing
  time in the society
• Required ambassador class, adding plugins to
  handle time, and modifying the time advance
  servlet

33
Time Management Details

• User manually advances time via servlet
• Must manually determine quiescence
• Servlet places time change request object on
  agents blackboard
• Time plugin subscribes to object on blackboard
• Reads it when it appears and calls ambassador
  service
• Service checks for pending request if not, sends
  request to RTI
• RTI grants advance when appropriate
• Service calls callback method in plugin
• Changes society time
• Stores new federation time
• RTI delivers appropriate messages to society

34
Mapping Agents, Actions, and Objects

• Cougaar societies contains many objects
• Map these objects to HLA actions and objects
  carefully to ensure acceptable performance
• Data can be transferred in HLA via interactions
  (events) and/or object attributes
• Mapping is heavily dependent on the society
  design
• Cougaar agents or asset objects map to HLA
  objects
• Cougaar events map to HLA interactions
• Cougaar objects such as UltraLog tasks could be
  mapped either way
• Depends on how long they can live and whether
  their contents change

35
Outline

• Motivation and Goals
• Distributed Agent Computing and Cougaar
• Integration Approach
• Lessons Learned
• Design considerations for Cougaar societies
• Conclusion

36
Federate Granularity

• Federate granularity the level of mapping of
  federates to portions of a Cougaar society
• Consider size and organization of the society,
  and design of other (non-Cougaar) federates
• In our demonstration prototypes, each Cougaar
  node mapped to a federate.
• In a large complex society, a logical group of
  agents (covering multiple nodes) might constitute
  a single federate
• Transportation organization, combat organization,
  etc.
• Single agents could map to individual federates
• Make sure this makes sense

37
Society Design

• Selection of agents when federation contains
  multiple societies
• Functional split
• Same agents, different functions
• Organizational split
• Different agents in logical groupings

38
Time Synchronization

• Need automated time advance mechanism
• Trigger time advance requests automatically
• Based on schedule or events
• Check for quiescence before making request
• Society time is always advancing
• Time step needs to be large enough that society
  time advances an insignificant amount between
  steps

39
Adaptive Society Design

• Prototype HLA ambassador class hardcodes HLA
  interactions, objects, attributes
• Need to modify class when plugins change
• Better way hardcode nothing in the ambassador
  class
• Each plugin would
• Register with the ambassador class
• Tell it which interactions and objects it was
  interested in
• The ambassador class would then
• Dynamically subscribe to the requested
  interactions
• Read the FOM file to get the attributes for each
  interaction
• Dynamically get the attribute handles
• Leverage Cougaar object discovery to match up
  objects in federates with those in the society

40
Operation Independent of Federation

• For more flexibility and survivability, design
  your society to operate
• Whether or not the appropriate federates are
  present
• If not available, use default plugins
• Or, use the Cougaar Message Transport Service
  (MTS) and create an HLA-specific link protocol
• Appropriate where a society
• May or may not be a member of a federation
• Has a number of other communication channels at
  its disposal
• MTS would select communication method based on
  availability and prioritization of channels
• Allows interoperation with simulations that exist
  outside of both the Cougaar society and the HLA
  federation

41
Outline

• Motivation and Goals
• Distributed Agent Computing and Cougaar
• Integration Approach
• Lessons Learned
• Design considerations for Cougaar societies
• Conclusion

42
Conclusion (1)

• Successfully Integrated a Cougaar-based agent
  society as a federate in V1.3 and 1516 HLA
  federations
• Integration leverages the benefits of both
  architectures
• Distributed agent-based architecture
• Standard simulation interoperability architecture
• Established how to
• Interface the society and the HLA RTI
• Synchronize society time with HLA RTI time
• Map agents, objects, and actions in the society
  to HLA objects and interactions

43
Conclusion (2)

• Examined integrating a full-size Cougaar society
  with an HLA federation
• Lessons learned can be applied to large-scale
  efforts
• Such as integrating large Cougaar-based logistics
  simulations with combat simulations
• Such integrated simulations would provide greater
  effectiveness than combat-only simulations

44
Integrating a Distributed Agent-Based Simulation
into an HLA Federation

• Gary Kratkiewicz
• Amelia Fedyk
• Daniel Cerys