Use of SPEEDES for BMDSsim

1
Use of SPEEDES for BMDSsim
Bill Grenard Metron High Performance
Computing [phone] grenard_at_ca.metsci.com
Dr. Ron Van Iwaarden Metron High Performance
Computing [phone] vaniwaar_at_ca.metsci.com
26 July 2004
2
Overview

• Metron
• SPEEDES
• Approach to BMDSsim Clustering
• Summary

3
Metron, Inc.

• Founded in 1984
• Products include Naval Simulation System, SPEEDES
  simulation engine
• Technical staff of 63 . . . 30 based in Reston VA
  and 33 in San Diego CA.
• Onsite simulation experts at Pentagon (OPNAV),
  JNIC, COMPACFLT and JWARS

Highest Clearance Held
Academic Concentration
Highest Degree Attained

• Missile Defense projects include
• - SPEEDES simulation engine for MDWAR wargames --
  MDA
• - Bayesian algorithm development for target
  designation -- MDA
• - Theater missile defense simulation -- Navy

4
SPEEDESSynchronous Parallel Environment for
Emulation and Discrete-Event Simulation

• Powerful optimistic-processing parallel
  processing engine
• Developed, maintained, and distributed by Metron
  since 1996
• Open source code downloadable to qualified users
• On-line documentation
• On-line change request system
• Primary users MDA projects
• MDWAR
• IMDSE (conservative-processing variant)
• Windows NT
• SPEEDES just works and lets them model
• C2BMC
• ABL
• Other SPEEDES-related efforts
• Air Force Research Lab (Rome Labs)
• Rome Labs funded iterator improvement,
  incorporated in Version 2
• Distributed Information Enterprise Modeling and
  Simulation (DIEMS)
• Parallel multiple-course-of-action SPEEDES
  enhancement
• NASA KSC
• Independent IVV of the JSIMS Simulation Engine
  (the CCSE)

5
Metrons SPEEDES Team
Ron Van Iwaarden PhD, Applied Mathematics,
University of Colorado MS, Applied Mathematics,
University of Colorado BS, Mathematics,
University of Colorado SPEEDES developer for 7
years Lead, SPEEDES development and
documentation MDWAR wargame support at the
JNIC
Gary Blank MS, Comp. Science, University of
Virginia BS, Applied Mathematics, Brown
University SPEEDES developer for 7 years Lead,
SPEEDES Multi-COA Enhancement project SPEEDES
enhancements for MDWAR SPEEDES support of AFRL
(DIEMS, GIEsim) SPEEDES FAAsim prototype HLA
RTI developer HLA Federations
Jacob Burckhardt BS, Computer Science, UC
Berkeley SPEEDES developer for 7 years Lead,
JSIMS sim engine IVV SPEEDES enhancements for
MDWAR SPEEDES testing SPEEDES Configuration
Management
Scott Shupe BS, Computer Sci., Rensselaer
Polytechnic Inst. SPEEDES Developer for 5
years SPEEDES enhancements for MDWAR SPEEDES
load balancing studies for AFRL SPEEDES FAAsim
GUI HLA RTI developer At MITRE, developed RTI
verification test set
Steve Heistand BS, Aerospace Engineering, Iowa
State Univ SPEEDES developer for 6
months Extensive previous work in tuning,
porting and developing of parallel algorithms
Aircraft flight dynamics models Jet turbine
engine models Global weather models ASCI
codes.
6
SPEEDES Early Development and Modern Versions

• Chosen as framework for MDWAR in late 1996
• Early beta versions concentrated on functionality
  rather than reliability
• Frequently buggy, undocumented, poor performing
• Version 1.0 (November 2000)
• Completed the Unified API
• Added the SPEEDES Users Guide
• Added the API Reference Manual
• Much of the obsolete code was removed
• Version 2.0 (September 2001)
• Added object proxy attribute subscription
• Added automatic lazy re-evaluation
• General code optimization (size and speed)

On-line at www.speedes.com
7
Approach to BMDSsim Clustering

• Clustering can lead to high performance
  federations
• Retains ability for easy debugging modes
• Can link up through shared memory or TCP/IP
• Design allows for MDWAR Standard Gateway (MSG)
  connections
• Elements could hook together in variety of
  fashions
• Optimistic Full optimistic time management with
  rollbacks
• Includes the option of connecting through shared
  memory on the same machine or TCP/IP for those
  that are remote.
• Conservative Linked through MSGs
• Playback A element could be replaced by an MSG
  Playback for standalone testing/debugging
• Any combination of the above

8
How clustering works
SPEEDES can connect simulations using
conservative time management . . .
. . . or as one large simulation using optimistic
time management and high-speed communications
Cluster 4
ABL
SPEEDES Server
ABL
MDWAR Ghost
MDWAR Ghost
Cluster 2
Massive HPC
High Fidelity Threat
MSG
High Fidelity Threat
MSG
Cluster 1
MDWAR Ghost
MDWAR Ghost
MDWAR
MDWAR
ABL Ghost
Hi-FiThreat Ghost
ABL Ghost
Hi-FiThreat Ghost
Cluster 3
C2BMC
MSG
C2BMC
C2BMC Ghost
MDWAR Ghost
MDWAR Ghost
C2BMC Ghost
ABL Airborne Laser simulation C2BMC Command
and Control, Battle Management, and
Communications system MDWAR Missile Defense
Wargaming and Analysis Resource MSG MDWAR
Standard Gateway
9
Summary

• Main SPEEDES focus is and will be on stability,
  and reliability
• Performance has already been proven
• Continuing use in wargames provides rigorous test
  environment
• Mature set of tools help optimize performance,
  minimize overhead
• SPEEDES instrumentation
• MDWAR simulation instrumentation and analysis
  tools
• Rules of thumb
• Continuing improvement
• Changes for usability
• Reduction in memory and CPU footprint
• AFRL funded parallel course-of-action engine (due
  March 2006)
• MDA can have confidence in high performance, low
  risk for BMDSsim

10
Back-ups
11
Lessons learned

• SPEEDES has been extraordinarily resilient
• Almost all performance problems have been due to
  improper modeling
• Framework bugs are now rare
• Significantly impacted development in the early
  (lt v 0.8) years
• Proxy mechanism is solid but tightly couples
  models
• Use of proxy updates has decreased significantly
• Proxies use often indicates incorrect modeling
• Not communicating through message sets
• Unnecessary or excessive notifications
• Attribute subscription carries a small penalty
• Often used to simply unsubscribe totally to proxy
  updates

12
Lessons learned (cont)

• Performance tuning requires analysis tools
• Real time performance does not come for free
• Built up suite of analysis tools
• SPEEDES instrumentation has been extensive and
  varied
• MDWAR has many tools to analyze the
  instrumentation files
• Rules of thumb learned about modeling
• New APIs added to improve parallelism
• SPEEDES overhead is minimal (usually 10s of
  micro-seconds/event)
• Recent tests using MDWAR 5.0 (SPEEDES 2.1) on 1
  node (optimistic) shows a 15 framework
  overhead.
• BTW is within 20 of sequential on SPEEDES 2.1,
  should be 10-15 with 2.2
• I/O is a killer.
• Data collection has a minor impact
• Biggest problem is making sure we collect enough

13
SPEEDES, IMDSE Release History