modeling, simulation,

1
modeling, simulation, AnalysisDoD Lessons
Applied to NASA Problems

• By
• Roy E. Rice, Ph. D., P.E.
• Chief Scientist
• Teledyne Brown Engineering
• May 19, 2005

2
OUTLINE

• Introduction
• What is Analysis?
• Why do we perform Analysis?
• Analysis Methodologies/Applications
• Examples
• Whats missing?

3
INTRODUCTION

• Analysis, Operations Analysis, and
  Operations Research (OR) used interchangeably
• OR began out of wartime necessity in WWII
• Great Britain
• Radar employment and search theory
• Multidisciplinary
• US Military OR services set up Ops Analysis
  Shops shortly after WWII
• Practitioners use Models and Simulations (MS) as
  tools
• Analogy Carpenters use saws.
• Its the Analyst, Stupid! focus on the
  carpenter, not the saw.
• I can do some very bad analysis with the worlds
  best models!

4
WHAT IS ANALYSIS?

• Analysis is the application of the Scientific
  Method in pursuit of insight to a problem or
  question posed by decision makers
• OR Analysismilitary definition (from DoD
  dictionary)
• An analytical study of military problems
  undertaken to provide the responsible commanders
  and staff agencies with a scientific basis for
  decisions or action to improve military
  operations.
• Informs the decision maker
• Provide insights
• Frame issues and questions
• Examine alternatives
• One of many inputs
• Not tantamount to the decision itself

5
WHAT ANALYSIS ISNT?

• Running a model
• Reading/commenting on others work
• Personal opinion
• Group opinion
• Organizational position
• Justifying a predetermined answer
• The final decision

Being in a shop where people run models no more
makes you an analyst than sitting in your garage
makes you a car! -- Lt Col Kirk Yost (ret)
6
WHY DO WE PERFORM ANALYSIS?
Key pillars of analysis Helping the decision
maker understand problems and candidate solutions
7
WHY DO WE PERFORM ANALYSIS?
The Analysis Process
Problem Statement
Determine Issues
Determine Mission Tasks (MTs)
Determine Analysis Methodology
Determine Alternatives
Select Models and Data
Develop Database
Perform the Analysis
8
ANALYSIS METHODOLOGIES/APPLICATIONS
OR Methodologies
Military Problems
9
EXAMPLES
Dormant Reliability
Air Force had an Air-to-Surface missile system it
was going to deliver in canisters (full-up
rounds) to the operational units. They were
going to check them out to ensure they were
working, then put them into an igloo for 10
years and expect them to work. Our job was to
assess the Probability that the missiles would be
fully operational after that 10 years.
We chose to analyze this problem using Markov
Chains.
1 2 3 4 5
lim pij(n) ?j
p11
p12
p13
p14
p15
State 1
n? ?
State 2
P5x5
Steady-state probability of being in any specific
state plus First Passage
State 3
State 4
State 5
p51
10
EXAMPLES (cont.)
Most Critical Design Requirement 39
combinations 19,683 runs per case
Employ DoE and Optimization
Parameter 5
Parameter 4
Parameter 1
Parameter 9
Parameter 6
Parameter 8
Parameter 3
Parameter 2
Parameter 7
Days
Low
Low
Low
Low
Low
Low
Low
Low
Low
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Factors
11
EXAMPLES (cont.)
Most Critical Design Requirement 39
combinations 19,683 runs per case
Parameter 1
Parameter 4
Parameter 9
Parameter 5
Parameter 6
Parameter 8
Parameter 3
Parameter 2
Parameter 7
Aircraft Lost
Low
Low
Low
Low
Low
Low
Low
Low
Low
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Med
High
Factors
12
EXAMPLES (cont.)
Allocation of Scarce Resources - Sensors,
Platforms, Comm Architectures, PEDS
2-D Geographic Grid, typically
ISR Req'ts
ISR Req'ts
ISR Req'ts
ISR Req'ts
ISR Req'ts
ISR Req'ts
?
?
ISR Req'ts
ISR Req'ts
ISR Req'ts
?
?
?
U2
UAV
Problem What is the Optimum Mix of Numbers and
Types of Sensors on the Numbers and Types of
Platforms? Constrained by Budget, Quantities,
Performance, Geography, CONOPS, other
?
National Assets
JSTARS
Phases Peacetime Initial Attack Halt Counter
Attack (user defined)
?
?
IR
?
EO
?
SAR
?
MTI
13
EXAMPLES (cont.)
Optimization (MIP) currently 600,000
constraints, 1.6M variables Sensor-Platform
Allocation Analysis Tool (SPAAT)
Objective Functions Minimize Cost Z Si
NEW SENSORSi acquisition cost i Sj NEW
PLATFORMSj acquisition cost i Sj PLATFORMS
USEDj sortie cost j Minimize Coverage Z
Sijklm NPLATSENSijklm sortiesj
coverage/sortie ij
Subject to Sij NPLATSENSijklm sortiesj
coverage/sortie ij gt min cov klm
for all k,l,m, INT Sij NPLATSENSijklm
sortiesj coverage/sortie ij lt max cov klm
for all k,l,m, INT Sij f
(NPLATSENSijklm sortiesj coverage/sortie ij
) gt min BDA klm for all k,l,m, INT Sij f
(NPLATSENSijklm sortiesj coverage/sortie ij
) lt max BDA klm for all k,l,m, INT S NEW
PLATFORMSj lt max PLATs j
for all j S NEW SENSORSi lt max SENSi
for all i S NEW
SENSORSi acquisition cost i lt max spend
SENSORS S NEW PLATFORMSj
acquisition cost j lt max spend PLATFORMS S
PLATFORMS USEDj sorties/platform j
attrition/sortiej lt max attrition j for all j
plus various accounting constraints..

Coverage thresholds

BDA thresholds

Asset bounds
i SENSORS j PLATFORMS k TARGETS l GRIDS m
TIME PERIODS
14
WHATS MISSING?

• Campaign and Operational Level Methodology for
• Space Exploration

Hierarchy of Analysis
Increasing Scope
Examples of Models
MTs
THUNDER, CFAM, SPAAT, LCOM
Theater/ Campaign
MOEs
EADSIM, SUPPRESSOR
Mission/Battle
ESAMS, RADGUNS, MOSAIC
Engagement
Increasing Resolution
MOPs
Engineering
Metrics Mission Task (MT) Measures of
Effectiveness (MOEs) Measures of Performance
(MOPs)
15
SUMMARY

• NASA problems are not THAT different than DoDs
• DoD has a long history of applying Operations
  Research or Operations Analysis to solving
  problems
• Applications of OR cut across a wide spectrum of
  problem types
• Need to focus on the ANALYSIS and ANALYSTSnot
  the tools (models)!
• Start by developing Strategic-Operational-Tactical
  level taxonomy.