VV

1
VVA Forum

• Thoughts on VVA
• from a
• Scientific SME
• Phil Zimmerman
• for
• S. K. Numrich, Dep. Dir., Technology
• DMSO
• snumrich_at_dmso.mil

2
In the Beginning

• The starting point for VVA in fact, the
  starting point for a simulation is a question
• WHY?
• Why is this simulation being built?
• Why is this federation being assembled?
• What question are we trying to answer?
• What purpose does this simulation serve?

3
The Response

• The Conceptual Model is the response to the
  question
• The more accurate and detailed the response, the
  better the question will be addressed
• Arguments and changes at this level cost frazzled
  nerves and frayed egos, but not overwhelming
  resources
• Failures at this stage may be detected later, but
  the later they are detected, the more they cost
  to fix.

4
Necessary, but not Sufficient

• Between the conceptual model and the resultant
  simulation dwell pitfalls of every sort
• Approximations because we just cant solve that
  problem exactly
• Data or, most often lack thereof
• Languages that cannot express the operations
  (human or computer)
• Humans (all of us) who execute imperfectly
• Lack of an adequate test plan for the results
• The Rush to the finish line or demo
• The obligatory black box whose author no longer
  works here
• In the end, we dont execute the concept model

5
E Pluribus Unum

• The system of systems!
• We almost never simulate one system
• The essence of most simulations beyond the
  engineering domain is to explore system of
  systems
• But we measure and pedigree individual systems
• We dont have validation data for systems of
  systems

6
Sharing the Universe

• We can federate these models, but do the radars
  share the same universe?
• Common data is necessary, but not sufficient
• Individual system pedigrees are essential
• BUT that is not sufficient

7
Operating in the Same Universe

• Operating in the same universe implies that the
  systems perceive and respond to shared components
  of the universe in a consistent way
• Response is not skin deep it reaches down to
  the algorithm level
• All radars respond to the environment
• Propagation models
• Clutter models
• Each radar may be perfectly validated for its
  original purpose, but if there are conflicts in
  the assumptions of the underlying models between
  one radar and another, they are not operating in
  the same universe

8
Thats the Easy Problem

• Consistency can be verified and validated
• Its still s single system problem!
• Once you determine your standard, you can use it
  with all systems as appropriate.
• How do you get validation data for any specific
  mix of systems?
• System tests are performed for individual systems
• It is complex and expensive to gather data for
  different mixes of systems
• But TRANSFORMATION is all about effects of
  different mixes of systems

9
And then theres that pesky thing called ERROR!

• Remember when?
• Error tracking was mandatory
• You measured error
• From component to component
• When we need solid answers, how do we track
• Errors in model performance
• Uncertainties in data
• How do we manage error when face validation is
  not sufficient
• Test and evaluation requires more than face
  validation

10
How Much Is Enough?

• In the Beginning
• You start with the purpose
• Not every simulation has the same requirements
  for correctness or whatever term you choose to
  apply
• Part of the conceptual model must be
• How well do the components need to perform
  individually and as a set operating in the same
  universe?
• Error analysis and estimation begin at the start
  with the conceptual model

11
The Art of the Possible

• How do you reach the point where you can
• Understand how good it must be
• Estimate how good you can make it
• And then
• Create the methodology that builds the best
  possible
• Systems Engineering
• into each federation WITH DOCUMENTATION

12
And Now

• About Composability
• Not without Formal Foundations
• Not without VVA Methods