White-Box Testing Techniques II

1
White-Box Testing Techniques II
Software Testing and Verification Lecture 8

• Prepared by
• Stephen M. Thebaut, Ph.D.
• University of Florida

2
White-Box Testing Topics

• Logic coverage (lecture I)
• Dataflow coverage (lecture II)
• Path conditions and symbolic evaluation (lecture
  III)
• Other white-box testing strategies (e.g.,
  fault-based testing) (lecture IV)

3
Dataflow Coverage

• Based on the idea that program paths along which
  variables are defined and then used should be
  covered.
• A family of path selection criteria has been
  defined, each providing a different degree of
  coverage.
• CASE tool support is very desirable.

4
Variable Definitions and Uses

• A program variable is DEFINED when it appears
• on the left hand side of an assignment statement
  (e.g., Y 17)
• in an input statement (e.g., input(Y))
• as an OUT parameter in a subroutine call (e.g.,
  DOIT(XIN,YOUT))

5
Variable Definitions and Uses (contd)

• A program variable is USED when it appears
• on the right hand side of an assignment statement
  (e.g., Y X17)
• as an IN parameter in a subroutine or function
  call (e.g., Y SQRT(X))
• in the predicate of a branch statement (e.g., if
  Xgt0 then...)

6
Variable Definitions and Uses (contd)

• Use of a variable in the predicate of a branch
  statement is called a predicate-use (p-use).
  Any other use is called a computation-use
  (c-use).
• For example, in the program statement
• If (Xgt0) then
• print(Y)
• end_if_then
• there is a p-use of X and a c-use of Y.

7
Variable Definitions and Uses (contd)

• A variable can also be used and then re-defined
  in a single statement when it appears
• on both sides of an assignment statement (e.g., Y
  YX)
• as an IN/OUT parameter in a subroutine call
  (e.g., INCREMENT(YIN/OUT))

8
Other Dataflow Terms and (Informal) Definitions

• A path is definition clear (def-clear) with
  respect to a variable v if there is no
  re-definition of v within the path.
• A complete path is a path whose initial node is
  a/the start node and whose final node is a/the
  exit node.

9
Other Dataflow Terms and (Informal) Definitions
(contd)

• A definition-use pair (du-pair) with respect to
  a variable v is a double (d,u) such that d is a
  node in the programs flow graph at which v is
  defined, u is a node or edge at which v is used,
  and there is a def-clear path with respect to v
  from d to u.
• (Note that the definition of a du-pair does not
  require the existence of a feasible def-clear
  path from d to u.)

10
Example 1

• 1. input(A,B)
• if (Bgt1) then
• 2. A A7
• end_if
• 3. if (Agt10) then
• 4. B AB
• end_if
• 5. output(A,B)

input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
11
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
12
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
13
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
14
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
15
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
16
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
17
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
18
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
19
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
20
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
21
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
22
Identifying DU-Pairs Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
23
Identifying DU-Pairs Variable B
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
24
Dataflow Test Coverage Criteria

• All-Defs for every program variable v, at least
  one def-clear path from every definition of v to
  at least one c-use or one p-use of v must be
  covered.

25
Dataflow Test Coverage Criteria (contd)

• Consider a test case executing path
• 1. lt1,2,3,4,5gt
• Identify all def-clear paths covered (i.e.,
  subsumed) by this path for each variable.
• Are all definitions for each variable associated
  with at least one of the subsumed def-clear
  paths?

26
Def-Clear Paths Subsumed by lt1,2,3,4,5gt for
Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
27
Def-Clear Paths Subsumed by lt1,2,3,4,5gt for
Variable B
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
28
Dataflow Test Coverage Criteria (contd)

• Since lt1,2,3,4,5gt covers at least one def-clear
  path from each of the two definitions of A/B to
  at least one c-use or p-use of A/B, All-Defs
  coverage is achieved.

29
Dataflow Test Coverage Criteria (contd)

• All-Uses for every program variable v, at least
  one def-clear path from every definition of v to
  every c-use and every p-use of v must be covered.
• Consider additional test cases executing paths
• 2. lt1,3,4,5gt
• 3. lt1,2,3,5gt
• Do all three test cases provide All-Uses coverage?

30
Def-Clear Paths Subsumed by lt1,3,4,5gt for
Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
31
Def-Clear Paths Subsumed by lt1,3,4,5gt for
Variable B
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
32
Def-Clear Paths Subsumed by lt1,2,3,5gt for
Variable A
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
33
Def-Clear Paths Subsumed by lt1,2,3,5gt for
Variable B
input(A,B)
1
Bgt1
2
A A7
B?1
3
Agt10
4
B AB
A?10
5
output(A,B)
34
Dataflow Test Coverage Criteria (contd)

• Since none of the three test cases covers the
  du-pair (1,lt3,5gt) for variable A, All-Uses
  Coverage is not achieved.

35
What would be even stronger than All-Uses?

• We have considered the All-Defs and the All-Uses
  criteria so far. How could the definition of
  All-Uses be modified to make it even stronger?
  Recall
• All-Uses for every program variable v, at
  least one def-clear path from every definition of
  v to every c-use and every p-use of v must be
  covered.

36
Example 2

• 1. input(X,Y)
• 2. while (Ygt0) do
• 3. if (Xgt0) then
• 4. Y Y-X
• else
• 5. input(X)
• end_if_then_else
• 6. end_while
• 7. output(X,Y)

1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
37
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
38
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
39
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
40
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
41
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
42
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
43
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
44
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
45
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
46
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
47
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
48
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
49
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
50
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
51
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
52
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
53
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
54
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
55
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
56
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
57
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
58
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
59
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
60
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
61
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
62
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
63
Identifying DU-Pairs Variable X
1
input(X,Y))
2
Ygt0
Y?0
3
Xgt0
X?0
Y Y-X
input(X)
4
5
6
Y?0
Ygt0
7
output(X,Y)
64
More Dataflow Terms and Definitions

• A path (either partial or complete) is simple if
  all edges within the path are distinct (i.e.,
  different).
• A path is loop-free if all nodes within the path
  are distinct (i.e., different).

65
Simple and Loop-Free Paths
66
Simple and Loop-Free Paths (contd)

• Which is stronger, simple or loop-free?

67
More Dataflow Terms and Definitions

• A path ltn1,n2,...,nj,nkgt is a du-path with
  respect to a variable v if v is defined at node
  n1 and either
• there is a c-use of v at node nk and
  ltn1,n2,...,nj,nkgt is a def-clear simple path, or
• there is a p-use of v at edge ltnj,nkgt and
  ltn1,n2,...njgt is a def-clear loop-free path.

68
More Dataflow Terms and Definitions

• A path ltn1,n2,...,nj,nkgt is a du-path with
  respect to a variable v if v is defined at node
  n1 and either
• there is a c-use of v at node nk and
  ltn1,n2,...,nj,nkgt is a def-clear simple path, or
• there is a p-use of v at edge ltnj,nkgt and
  ltn1,n2,...njgt is a def-clear loop-free path.

NOTE!
69
Identifying du-paths
70
Another Dataflow Test Coverage Criterion

• All-DU-Paths for every program variable v, every
  du-path from every definition of v to every c-use
  and every p-use of v must be covered.

71
Exercise

• Identify all c-uses and p-uses for variable Y in
  Example 2.
• For each c-use or p-use, identify (using the
  notation) all def-clear paths.
• Identify whether or not each def-clear path is
  feasible, and whether or not it is a du-path.

72
Summary of White-Box Coverage Relationships
Path
Compound Condition
All du-paths
Branch / Condition
All-Uses
Basis Paths
Loop

Condition
Branch
All-Defs

Statement
- nominal cases

73
White-Box Testing Techniques II
Software Testing and Verification Lecture 8

• Prepared by
• Stephen M. Thebaut, Ph.D.
• University of Florida