Figure 1. Basic Software Testing Words

1
Figure 1. Basic Software Testing Words

• Error An error is a mistake of commission or
  omission that a person makes. An error causes a
  defect. In software development one error may
  cause one or more defects in requirements,
  designs, programs, or tests.
• Defect Also called a fault or a bug, a defect is
  an incorrect part of code that is caused by an
  error. An error of commission causes a defect of
  wrong or extra code. An error of omission
  results in a defect of missing code. A defect
  may cause one or more failures.
• Error of omission missing design components OR
  design omission
• Failure A failure is a deviation from expected
  behaviour exhibited by software and observed as a
  set of symptoms by a tester or user. A failure
  is caused by one or more defects.
• The Causal Trail A person makes an error that
  causes a defect that causes a failure.
• Root Cause Analysis needs traceability mechanism
  and pref. A TOOL
• Sources of errors FIXES
• REQTS
• DESIGN
• CODE
• CHANGES
• Based on VV FAULT MODEL

2
Process Management

• SEI Capability Maturity Model Level
• 1 2 3
  4
• Initial SE Repeatable Defined SE
  Managed SE
• Process Process
  Process
• Chaotic, Measurements
  Metrics
• over budget (compilers,
  TOOLS Testing
• late word processors, time
  (real) Tools
• Powerpoint)
  (elapsed) Traceability
• 
  of people Tool
• 
  LOCC
• 
  Excel, Micro Planner
• 5
• Optimized SE Process
• Continuous Improvement (modify process to achieve
• quality
  or productivity gains)
• REQTS DESIGN capturing,

3
Figure 3. Sample STL Sentence

• Action A01
• is allowed in state normal
• receives eventitem warning_interrupt
• transmits event safety_action1
• uses dataitem water_temp_reading,
• water_temp_base
• produces dataitem safety_action_report
• has duration time maximum 3
• has duration time unit second.
• Tool can analyze REQTS. in Formal notation for
• consistency (pairwise)
• ?ij if Ri is satisfied, Rj can also be
  satisfied
• Criticism Too low level (states)
• e.g. SDL too low level in TAU? For REQTS., try
  use cases or MSCs.

4
Verification Validation ? SQE

• SQE contains
• management strategies
• milestones
• measurements
• techniques
• REQTS-based (Black Box)
• DESIGN-based (Grey Box)
• Code-based (White Box)
• Behaviour-based (Reliability
  Robustness)
• Fix-based (Regression)
• Tools

Progress quality
5
RTM MSCs REQTS M1
M2 M3 M4 M5
M6 R1 R2

• Benefit - MSCs have - tool support
• - precise syntax
• - some checking for consistency and
  ambiguity
• - customer friendly
• - graphical (with abstraction)
• Weakness - graphical (?)
• - finite of MSCs
• (same as reqts. List -
  completeness issue)
• - tend to leave original
  reqts. behind

6
SQE ideally with tools,REQTS gathering,
representation, validation

• REQTS. Engineering tools (RTM)
• Arrival of New REQT.
• How does this affect integrity of R1 - R29? not
  many tools for checking consistency or
  completeness of requirements
• need for check that set of requirements is
• (worked phrases requirements)
• must
• must not

RTM Test REQTS Cases T1
T2 T3 T4 R1 C
0 R11
0 C R12
C R2 R30
unambiguous consistent complete
7
Problems with RTM

• Updating RTM to code and to test cases
• needed for functional tests
  regression tests
• 
  need for new or revised tests
• Ideally
• hyper-links REQTS.
• DESIGN
  TESTS
• 
  CODE
• Allows change documentation to flow from REQTS.
• A RTM tool can be very useful in keeping REQTS,
  DES., CODE, TESTS unambiguous, consistent

? 1 ?
? 1
? 1
? 2
not recommended
? 2
? 1
? 2
8
Requirements-based Testing

• Mapping traceability (REQTS, Tests)
• test plans
• scenarios or MSCs
• generate reqts.-based tests
• test oracle whether a test result is a PASS or
  FAIL or INCONCLUSIVE
• avoid obsolete tests when reqts. change, but
• incur additional testing cost
• automate test execution
• measure test quality (coverage)
• optimize testing cost-benefit

9
Specification Languages

• ASN.1 Neufeld, G. and S. Vuong, An Overview of
  ASN.1, Comput. Networks ISDN Syst., Vol. 23,
  1992, pp. 319-415.
• Estelle A Formal Description Technique Based
  on an Extended State Transition Model, Intl.
  Org. Standard, IS 9074, 1988.
• Larch Guttag, J., J. Horning, and J. Wing, The
  Larch Family of Specification Languages, IEEE
  Software, Vol. 2, No. 5, 1985, pp. 24-36.
• LOTOS A Formal Description Technique Based on
  Temporal Ordering of Observed Behavior, Intl.
  Org. Standard, IS 8807, 1988.
• SDL Saracco, R. and P. Tilanus, CCITT SDL
  Overview of the (Specification description)
  Language and Its Applications, Comput. Networks
  ISDN Syst., Vol. 13, No. 1, 1987, pp. 65-74.
• STL The Semantic Transfer Language, in IEEE
  Standard 1175-1994, Standard Reference Model for
  Computing System Tool Interconnections. IEEE
  Press, New York, pp. 37-117, 1994.
• Z Wordsworth, J., Software Development with Z
  A Practical Approach to Formal Methods in
  Software Engineering, Addison-Wesley, Workingham,
  England, 1992.
• VDM Jones, C.B., Systematic Software
  Development Using VDM, Prentice-Hall, Englewood
  Cliffs, NJ, 1986.