DoN VVA Process Standard

1
NDAT Process Adopted as DoN VVA Process Standard
2005 Spring Simulation Interoperability
Workshop 6 April 2005
Harriette L. Tullos-Banks VVA Project
Lead NAVSEA Dahlgren, N14 Telephone
540-653-1278 e-mail htullos_at_nswc.navy.mil
Catherine T. Parker VVA Task Lead NAVSEA
Dahlgren, N14 Telephone [phone] e-mail
ctparke_at_nswc.navy.mil
Kenneth W. Collins VVA Specialist EGG Technical
Services Telephone [phone] e-mail
kcollins_at_egginc.com
2
Introduction
The NAVSEA Dahlgren Accreditation Team (NDAT)
Verification, Validation, and Accreditation
(VVA) process has been approved and used by
various Program Executive Offices and Commander,
Operational Test and Evaluation Force
(COMOPTEVFOR) for 5 years.
The process defines the products, processes and
participants necessary to establish the
credibility of Modeling and Simulations (MS)
including hardware-in-the-loop and
land-based-test-sites.
Even though this process has been primarily used
on legacy MS, it is easily tailorable for new
MS.
The N14 VVA process is in compliance with
Department of Defense (DoD) and by unanimous vote
has been approved as the Department of Navy (DoN)
standard.
3
VVA Definitions
What VVA is
Verification, Validation, and Accreditation
(VVA) are three interrelated but distinct
processes that gather and evaluate evidence to
determine, based on the simulations intended
use, the simulations capabilities, limitations,
and performance relative to the real-world
objects it simulates.
What VVA is not
Certification - Formal Process of Ensuring and
Act of Attesting that a certified system is safe
and acceptable for Operational Use.
Independent Verification and Validation (IVV) -
The verification and validation of a software
product by an organization that is both
technically and managerially separate from the
organization responsible for developing the
product.
4
VVA Definitions (cont.)
5
Six-Step Process to Designation
What is the Designation Process?
The designation process involves a set of steps
that when implemented correctly leads toward the
selection and formal designation of MS for
accreditation.
Step 1 Identify Need Define the Specific
Intended Use and Establish the Necessity of
Accreditation
Step 2 Establish Requirements Identify Usage and
Functionality Requirements to Support the
Specific Intended Use
6
Six-Step Process to Designation (cont.)
MS Requirements

• Clear and unambiguous with unique designation for
  each requirement

• Expressed so as to be understood consistently by
  the User, Domain SMEs, and the Developer

• Defined even if obvious (user and developer may
  differ about what is obvious)

• Consistent with other requirements for the
  simulation

• Testable or, at least, satisfaction of a
  requirement is demonstrable in some objective and
  measurable way

• Viable and achievable

• Organized to facilitate requirements modification
  during the course of the project

• Traceable both forward to simulation design and
  implementation and back from simulation
  implementation to the original objectives

7
Six-Step Process to Designation (cont.)
Step 3 Identify Sponsor Identify the Program
Office or Program Manager Who Will Be Responsible
for Funding and Approving MS Results That Will
Satisfy the Requirements
Step 4 Understand MS Functionality Document the
Candidate MS Capabilities as They Support the
Defined MS Requirements and Assess How Well the
MS will Support the Specific Intended Use
Step 5 Develop A Plan of Action and
Milestones Develop an Overall POAM to Quantify
All Issues
Step 6 Make Designation Decision Determine the
Technical and Programmatic Assessments to
Designate MS for Accreditation
8
Participants
Accreditation Authority (AA) - An individual
occupying a position with the appropriate rank,
grade, responsibility and/or authority to
accredit a model, simulation, or federation of
models and/or simulations for a particular
purpose or purposes.
Accreditation Review Panel (ARP) - The Program
Office organization responsible for managing the
gathering and review of VV information and
presenting an Accreditation Recommendation to the
Program Manager.
Simulation Control Panel (SCP) - A technical
review panel established by the ARP to guide the
gathering of VV information.
Modeling and Simulations Proponent (MSP) - A
developer, maintainer, modifier, user, or sponsor
of MS.
9
Products
Designation Phase
Execution Phase
Accreditation Phase
VVA Designation Letter
CM Plan
MS Accreditation Recommendation Letter
ARP Charter
Modeling and Simulations Support Plan (MSSP)
MS Accreditation Decision Letter
MS Accreditation Plan
VV Plan
MS Catalog Entry
SCP Charter
VV Report
VV Assessment Report
10
COMOPTEVFOR (Accreditation Authority)
Program Office (Accreditation Authority)
Accreditation Review Panel
Simulation Control Panel
Models and Simulations Proponent
MS VVA Designation Letter
CM Plan
ARP Charter
MS Accreditation Plan (AA Approved)
MSS Plan
VV Plan Guidelines
VV Plan (SCP Approved)
SCP Charter
VV Report (SCP Approved)
VV Assessment Report (ARP Approved)
MS Accreditation Recommendation Letter
MS Recommendation Letter
MS Accreditation Decision Letter
MS Accreditation Report
MS Catalog Entry
11
Benefits

• Stays in compliance with DMSO

• Increases MS usability for future applications
  by following a process that makes sense and can
  be reproduced universally

• Increases confidence in MS usage by
  understanding objectives for specific intended
  use(s)

• Reduces risk associated with MS use

• Capitalizes on existing information and expertise

• Provides documentation for a clear and
  supportable rationale for accreditation

• Saves Time and Money in Test Evaluation keeping
  within program costs

• Provides better understanding of participants
  roles and responsibilities by having meetings and
  briefings which promotes open communications
  throughout the VVA process

12
NDAT STAFF