IV

1
IVV of Engineering SoftwareAn Experience
Workshop on Engine Control Systems
Technology WECST-2004 November 19-20, 2004

• K Sudhakar
• Department of Aerospace Engineering
• Indian Institute of technology, Mumbai

2
Verification Validation

• What?
• Why?
• When?
• How?
• Who?

3
Verification Validation
What?

• No computer code can be proved to have no errors.
  But Can be proved wrong if evidence to this
  effect is collected.
• Process of increasing our confidence in results
  of a computation
• Practiced with rigor in critical applications.
• IVV, ADA, Bangalore
• ISRO, DRDL, . .
• CFVDS - Centre for Formal Verification Design
  of Software, IIT Bombay

4
Verification Validation
What?
Reality
5
Validation
What?

• Validation - Is the correct model being solved?

If resistance due to air is significant the
model will be incorrect. How to confirm if model
is good? Analytically integrate? Conduct
experiments. Compare Possible only for simple
Models.

• Verification - Is the model being solved
  correctly

6
Verification Validation
What?
7
Verification Validation
Why?

• Incorrect results

8
Verification Validation
When?

• Most effective if conducted in parallel with s/w
  development
• Concept VV
• Requirements VV
• Design VV
• Implementation VV
• Test VV
• Installation Checkout VV
• Operation VV
• Maintenance VV

9
Verification Validation
When?

• VV are defined as the process of determining
  ie.
• They are on-going activities
• No end point

10
Verification Validation
How?

• Are the results given by a computer program
  correct?
• How to investigate this?
• Any standard processes to perform VV?
• IEEE Std 1012-1998, etc.
• Common sense, logic, discipline, . .

11
Verification
How?

• Verification - Is the model being solved
  correctly?
• Solution Verification. Numerical techniques
• Sources of error - spatial temporal
  discretisation, iterative convergence, round off
• ODEs
• Time step adequate for all situations? Adaptive?
• PDEs
• round off, iterative convergence are better
  understood and can be addressed.
• Discretisation errors in PDE are less understood

12
Verification
How?

• Verification - Is the model being solved
  correctly?
• Code Verification
• Coding language,
• Coding practices, . . .
• Model to code translation
• Static dynamic analysis.
• Statically unreachable code
• Dynamically unreachable code
• FTNCHK, FORCHEK etc.

13
Verification
How?
14
Verification
How?

• Checks must cover all paths/possibilities!
• if (0ltmlt0.9) then
• gama 1.4
• call abc (.)
• elseif (0.9ltmlt1.2) then
• gama 14.
• call def (. . . .)
• elseif (1.2ltmlt2.0)
• gama 1.4
• call ghi ( .)
• endif

Verification checks m 0.6, 0.8, 0.9, 1.3,
1.5, 2. Does not check one path. m 0.6,
0.8, 0.9, 1.0, 1.3, 1.5, 2. Checks all paths
15
Validation
How?

• Is the model correct?

16
Validation for Complex Systems
How?
17
Validation for Complex Systems
How?
Computational Predictions
Code
Computational Results of Exp outcome
Model
Difference
Validation Experiments

• Outcome of
• Experiment

18
Validation
How?
Prediction Confidence?
Predictive domain
Validation domain
Can validation domain cover the predictive
requirements?
19
Validation
How?
x - Validation experiments
Where to conduct experiments? How to assess
predictive confidence?
20
Validation
How?
21
Validation
How?

• Design Analysis of Computer Experiments
• Where to conduct computational physical
  experiments for validation?
• How to build predictive models?
• How to assess predictive confidence?

22
Validation
How?
23
Verification Validation
Who?

• Independent Verification Validation (IVV)
• Technical Independence. VV done by people who
  are not involved in design development
• Managerial Independence. An organization
  separate from that did design development.
• Financial Independence. Budget for VV is not in
  anyway controlled by those responsible to design
  development.

24
VV in CFD

• Less valued in CFD than in FEM (for historical
  reasons)
• AIAA Aerosp. Sciences Meeting Exhibit 2002
• DLR-F4 Wing Body
• 35 different CFD Calculations (Advanced research
  codes commercial codes)
• After weeding out outliers this gave a standard
  deviation in CD 0.0021 (for wind tunnel testing
  this figure is put at 0.0004)

25
Thank you
26
Coding Practice - Not Done!
How?

• if (x) 100, 200, 300
• 200 if (y) 400, 500, 600
• z z 1
• goto 750
• 500 xsq x2.
• goto 850
• 300 xcu x3
• ..

t1 a12. b1 t2 sqrt(t1)c1 t3
t21.3b1 . . . . . .
real a(100) integer b(50) complex
c(10) equivalence (a(1),b(1)),