ICS 52: Introduction to Software Engineering

1
ICS 52 Introduction to Software Engineering

• Lecture Notes for Summer Quarter, 2003
• Michele Rousseau
• Topic 13
• Partially based on lecture notes written by
  Sommerville, Frost, Van Der Hoek, Taylor
  Tonne. Duplication of course material for any
  commercial purpose without the written permission
  of the lecturers is prohibited

2
Todays Objectives

• Testing continued
• How to use a Test Matrix to organize test cases
• More examples of test coverage
• Formal Specification based testing

3
Recap Choosing Test Cases

• Identify what to test
• functionality from requirements (black box)
• classes, methods, statements (white box)
• Determine set of all inputs
• Identify one or more bases
• For each basis, define equivalence partitions
• For each eq. partition, choose test cases
• Use combinations of eq. partitions when
  appropriate
• For each test input, determine correct output
• Use a matrix to organize your test cases
• Rows test cases Columns equivalence
  partitions

4
Example
1 float homeworkAverage(float scores)
2 float min 99999
3 float total 0
4 for (int i 0 i lt scores.length i)
5 if (scoresi lt min)
6 min scoresi
7 total scoresi
8
9 total total min
10 return total / (scores.length 1)
11
5
Testing Matrix
Test case(input) Basis(subdomain) Equivalence Partitions Basis(subdomain) Equivalence Partitions Basis(subdomain) Equivalence Partitions Basis(subdomain) Equivalence Partitions Expected output Notes




6
Testing Matrix - homeworkAverage 1
Test case(input) Basis Array length Basis Array length Basis Array length Basis Array length Expected output Notes
Test case(input) Empty One Small Large Expected output Notes
() x 0.0
(87.3) x 87.3
(90,95,85) x 92.5
(80,81,82,83, 84,85,86,87, 88,89,90,91) x 86.0
7
Testing Matrix - homeworkAverage 2
Test case(input) BasisPosition of minimum BasisPosition of minimum BasisPosition of minimum Expected output Notes
Test case(input) First Middle Last Expected output Notes
(80,87,88,89) x 88.0
(87,88,80,89) x 88.0
(99,98,0,97,96) x 97.5
(87,88,89,80) x 88.0
8
Testing Matrix - homeworkAverage 3
Test case(input) BasisNumber of minima BasisNumber of minima BasisNumber of minima Expected output Notes
Test case(input) One Several All Expected output Notes
(80,87,88,89) x 88.0
(87,86,86,88) x 87.0
(99,98,0,97,0) x 73.5
(88,88,88,88) x 88.0
9
Security Dialog Example
10
Testing Matrix - Security Dialog 1
Test case(input) Choice of Button (Black Box) Choice of Button (Black Box) Choice of Button (Black Box) Choice of Button (Black Box) Expected output Notes
Test case(input) Ok Button Cancel Button Cancel Button Cancel Button Expected output Notes
Press Ok x x See Security Dialog 2
Enter ID, Press Cancel x Text Boxes are cleared, dialog remains
Dont Enter ID, Press Cancel x Text Boxes are cleared, dialog remains

11
Testing Matrix - Security Dialog 2
Test case(input) Basis Correctness of Input, with OK Button Press Basis Correctness of Input, with OK Button Press Basis Correctness of Input, with OK Button Press Expected output Notes
Test case(input) Valid ID, Correct PW Valid ID, Incorrect PW Invalid. ID. Expected output Notes
Test1, 77775555 x Proceed to Main Screen
Test1, 7777555 x Audible Beep, Dialog Remains
Tes1, 77775555 x Audible Beep, Dialog Remains

12
Another Structural Testing Example
// In a linked list class Node
getSecondElement() 1 Node head
getHead() 2 if (head null) 3
return null 4 if (head.next null) 5
return null 6 return head.next.node
3
1
2
5
4
6
13
Testing Matrix - getSecondElement (White)
All-Statement (white box)
Basis _______________________
Test Case (Input)
Expected Output
Notes
1 2 3 4 5 6
null list x x x null
(A ? null) x x x x
null
(A ? B ? x x x x
B null)
(A ? B ? x x x x
B C ? null)
14
Testing Matrix-homeworkAverage 3(White)
All statements (White box)
Basis _______________________
Test Case (Input)
Expected Output
Notes
123,4,5,6,7,8,11,4,5,8,11,4,13,14,15
Scores30,30 x all
scores are the same 30
15
Test Oracles

• Where does expected output come from?
• Critical to testing
• Difficult to create systematically
• Typically done with a lot of guesswork
• great dependence on the intuition of testers
• Formal specifications make it possible to
  automate oracles

A test oracle is a mechanism for deciding whether
a test case execution failed or succeeded
16
Formal Verification

• Techniques for proving consistency between two
  software descriptions
• subject to assumptions of proof system
• only as good as formal specification
• to prove consistency of specification
• to prove correctness of implementation
• does not show other qualities, especially the
  subjective ones
• has not been shown to scale up to large-scale
  software systems
• only informal techniques for validating against
  user needs

17
Verification with Formal Specifications
18
Testing with Formal Specifications
19
Formal Specification-Based Testing

• Formalize functional testing for formal
  specifications
• Extend implementation-based techniques to be
  applicable to formal specifications
• Dependent on the specification language
• Model-based specifications test cases selected
  to structurally cover specification
• Algebraic specifications test cases selected to
  test against axioms or axioms tested by
  interpretation on test data
• State-based specifications test cases selected
  to cover operations state changes

Specification-based testing should
augment implementation-based testing, not replace
it
20
Something to think aboutTesting
Process/Integration

• It is generally accepted that no single technique
  is sufficient to verify and validate software
• How do you choose?
• strengths and weaknesses
• integration issues
• incremental issues