Value-based Empirical Study for IV&V

1
Value-based Empirical Study for IVV

• ISERN2005
• Masa Katahira, JAXA
• Daniel Port, Univ. of Hawaii

2
Background

• IVV is a common method applied to
  safety-critical software in order to gain quality
  and safety. NASA, ESA, and JAXA have started the
  study of strategic IVV together.
• Main issue strategic planning of IVV activities
• Balancing safety vs. cost

3
Generating an Optimal Strategy

• What is an optimal strategy with respect to cost?
• For each attribute, pick technique that gives
  maximum benefit
• Order these pairs from max cost-benefit (RE
  reduction / cost) to lowest

4
Comparison of Strategies
strategic
5
Important Problems

• How much is enough IVV to perform?
• How can we make best use of IVV efforts with
  limited resources (budget, schedule, project
  constraints)?
• Must provide rationale for plans and budgets
• Explain in a tangible way why IVV is important
  and its benefits to managers, customers, and
  developers
• How can we select most cost-effective techniques
  from hundreds of IVV techniques?
• Which organization should perform which
  techniques?
• When should we stop IVV?
• How can you assess IVV Return On Investment?

IVV Is clearly not Value neutral!
6
Current Case Study

• System Characteristics
• Budget Limitation
• Environment

Planning is by expert opinion and best guess
IVV activity

• It must be addressed on evidence and past
  results, not just expert opinion and best
  guess.
• We must incorporate a continuous improvement and
  control program based comparing expected results
  with actual results.

7
Current Study

• Collaboration Study has been just started
• Daniel Port, University of Hawaii
• Masa Katahira, JAXA
• Haruka Nakao, JAMSS
• Expectation of ISERNers
• Opinions for value based IVV
• Suggestion for gathering the empirical data on
  IVV
• We will show our data collection format if
  requested

8
Back Up Slides
9
JAXA IVV techniques structure

• Main consideration is built in the strategic
  planning mechanism which need to indicate the
  attributes having system parameters (numeric
  numbers) and techniques having IVV conditions
  (numeric numbers)

IVV conditions
System Parameters
Target
Object
Attribute
Technique
Attribute
Technique
Object
Attribute
Attribute
Technique
10
System Characteristics

• 13 characteristics

Functional
Architecture
Development Process
11
Potential risk for each IVV attribute
Positive 0 Negative
(A5) Completeness of state transition
Characteristics parameter
Potential risk VALUE
Total Potential risk if the attribute is not
assessed
3
12
IVV conditions

• 8 IVV conditions

13
Efficiency of IVV techniques
Ex. System A
Efficiency of Completeness Analysis(T6) for
Completeness of state transition(T6)
3