Unit Testing Continued

1
Unit Testing Continued

• Control / Control Flow Testing
• Decision
• Statement
• Decision/Condition, Multiple Condition, etc
• Subsumption Hierarchies
• Loops
• Path Predicates
• Data Flow Testing
• Mutation Testing

2
Decision Coverage
If X
Process A
If Y
Process B
Process C
2

• Denoted by Process names
• A,B,C
• A

3
Statement Coverage
If X
Process A
If Y
Process B
Process C
2

• Denoted by Process names
• A,B,C

4
Control versus control flow

• Note distinction between control and control
  flow
• Control - focuses on statements that define
  control flow
• E.g., Statement, Decision, Condition,
  Condition/Decision, Modified Condition/Decision
  Coverage (MCDC)
• Control Flow - focuses on flow of control through
  statements
• E.g., Paths, Basis Paths, Multiple Condition Paths

5
Control Definitions

• Decision - Statement that changes flow of control
  in program
• Condition a boolean valued expression, which is
  part of a decision
• if ((a lt b) (c lt d) (e lt f))
• Decision is entire expression
• ((a lt b) (c lt d) (e lt f))
• Conditions are
• (a lt b)
• (c lt d)
• (e lt f)

6
Control Flow Criteria

• Decision - Every decision has a true and a false
  outcome
• Condition -- Every condition has a true and a
  false outcome
• Multiple Condition -- All feasible combinations
  of condition within a decision
• Decision / Condition -- Decision Condition
  Coverage
• Modified Condition Decision Coverages -- Decision
  / Condition show condition independence

7
Multiple Test Sets
If X
Process A
If Y
Process B
Process C
2

• Note Multiple ways to achieve coverageE.g., for
  decision coverage
• ABCA
• ABAC
• ABC ABAC

8
SubsumptionHierarchies

• Note decreasing number of test cases
• 4 Path Coverage
• 3 Basis Path Coverage
• 2 Decision Coverage
• 1 Statement Coverage
• Testing criteria A subsumes criteria B if by
  achieving criteria A implies achieving criteria B
• Subsumption doesnt imply less test cases will be
  required

9
Subsumption and Minimum Test Cases
If X
Process A
Process B
2
Process C
Minimum test cases to achieve coverage 2 -
path 2 - basis path 2 - decision 2 - statement
10
Loops

• There are only three kinds of loops nested,
  concatenated, and horrible -- Boris Beizer
• Loop Taxonomy
• Single Loop
• zero minimum, n maximum, no excluded values
• non-zero minimum, no excluded values
• excluded values
• Nested Loops
• Concatenated Loops
• many of the examples come from Boris Beizer
  Software Testing Techniques

11
Single Loop 0-M

• By pass the loop (zero iterations)
• One pass through the loop
• Two passes through the loop
• A typical number of iterations
• One less than the maximum number of iterations
• The maximum number of iterations
• Attempt one more than the maximum number of
  iterations
• Huang theorem shows that some data flow
  anomalies can by detected with only two iterations

12
Single Loop N-M

• Try one less than expected minimum
• The minimum
• One more than the minimum
• Once, unless covered by previous
• Twice, unless covered by previous
• A typical value
• One less than maximum
• The maximum number of iterations
• Attempt one more than maximum

13
Single Loop with Excluded Values

• Split into cases with no excluded values and test
  the boundaries
• For example say total range of loop is 1 to 20,
  but 7,8,9 excluded
• Split into two test
• 1-6 tests 0,1,2,4,6,7
• 11-20 tests 10,11,15,19,20,21
• where underlined are not suppose to work, but
  should be attempted

14
Nested Loops

• 1) Start at innermost loops. Set outer loop to
  minimum values
• 2) Test min, min 1, typical, max -1, and maximum
• 3) Expand tests to include out of bounds if
  possible
• 4) Test next outer loop with all inner loops set
  to typical
• 5) Do the five tests for all loops simultaneously
• This amounts to twelve tests for two nested
  loops. Why?

15
Nested Loop Example

• Assume the following loop and properties (min 1,
  max 10, typical 5)
• For (I 0 I lt M I)
• For (J0 J lt N J
• Inner Loop Tests
• M1,N1 M1,N2 M1,N5 M1,N9 M1,N10
• Outer Loop Tests
• M2,N5 M5,N5 M9,N5 M10,N5
• Combined
• M2, N2 M9, N9 M10, N10

16
Concatenated Loops

• Loops are concatenated if its possible to reach
  one after existing the other on the same complete
  path
• If the loops are independent of one another treat
  them as much
• If they are dependent, treat them as you would
  nested loops
• Not as difficult as nested loops in terms of
  processing time as processing time is additive
  (rather than multiplicative for nested)

17
Path Predicates

• Determine which inputs will force execution of
  particular statement (true / false)
• xx5
• if (X gt 20) lt--(true/false)
• Statement Value Predicate
• (xgt20) true (x5gt20)
• (xgt20) false (x5lt20)

18
Forming Path Predicates

• Start with expression
• (x gt y)
• For each variable work backwards to beginning of
  program
• Replace left hand side of variable assignment
  with righthand side
• Example
• xx5yxz
• if (x gt y) --gt (x5gtx5z)

19
Decisions Path Predicates

• xx5yxzif (x gt y) zz2else zz5
  if (z gt x)
• (z gt x) true --gt
• (x5gtx5z) z2gtx5)
• (x5ltx5z) z5gtx5)
• (z gt x) false --gt
• (x5gtx5z) z2ltx5)
• (x5ltx5z) z5ltx5

20
Forming Path Predicates

• (z gt x) true --gt
• (x5gtx5z) z2gtx5)
• (x5ltx5z) z5gtx5)
• (z gt x) false --gt
• (x5gtx5z) z2ltx5)
• (x5ltx5z) z5ltx5
• (z gt x) true --gt
• (0 gt z) z gt x3)
• (0 lt z) z gt x )
• (z gt x) false --gt
• (0 gt z) z lt x3)
• (0 lt z) z lt x )

21
Data Flow Testing

• Data flow testing is a family of test strategies
  through control flow to help identify data
  anomalies.
• Motivation Rapps and Weyuker
• It is our believe that, just as one would not
  feel confident about a program without executing
  every statement in it as part of some test, one
  should not feel comfortable about a program
  without having seen the effect of using the value
  produced by each and every computation

22
Data State Anomalies

• Symbols
• d, defined, created, initialized
• k, killed undefined, released
• u, used for something
• c, used in a calculation (e.g., right hand side
  of assignment)
• p, used in a predicate
• dd--suspicious
• dk--probably a bug
• kk--harmless but potential bug
• ku--a bug

23
Data Flow Anomaly State Graph
K
u,k
k
u
u,k,d
d
d,k
U
D
A
u
d
S
A anomalous S start U Used K killed D
defined
24
Selected DataFlow Criteria

• All-uses requires that for each definition of
  variable V at least one test case executes a
  definition-clear subpath from the definition to
  all reachable c-uses and p-uses of V.
• All-du-paths that for each definition of V test
  cases cover all definition-clear subpaths from
  the definition to all reachable uses of V

25
Selected DataFlow Criteria

• All-defs requires that for each definition of a
  variable V at least one test case executes a
  definition-clear subpath to a use of V
• All-c-uses requires that for each definition of a
  variable V at least one test case executes a
  definition-clear to each c-use of V
• All-c-uses/some-p-uses that for each definition
  of variable V at least one test executes to each
  c use of V, if there are no c uses, then at least
  one p use.

26
DataFlow Subsumption
all paths
all du-paths
all uses
all-p/some-c
all-c/some-p
All-c uses
all defs
all-p uses
decision
statement
27
Example DataFlow Analysis

• X Integer.read(args0)
• Y Integer.read(args0)
• If (x gt y)
• Z X
• Else
• Z Y
• If (x gt 10)
• Z
• ..
• Defs P-use C-use
• X_at_1 3,7 4
• Y_at_2 3 6
• Z_at_4 8
• Z_at_6 8
• Z_at_8

28
Mutation Testing

• Based on four assumptions
• The competent programmer hypothesis Experienced
  programmers write programs that are correct or
  almost so
• The coupling effect Test data that detects
  small faults is also likely to detect large
  faults
• Mutant operators There exists a small set of
  predefined incremental mutation operators that
  capture nearly all simple faults
• Test oracle Must be some systematic way to
  ascertain correctness

29
Mutation Testing

• Given a program P and test cases T
• Execute T with P (we assume this produces no
  errors)
• Create a alternative programs called mutants of P
• Create small change in program (e.g., change and
  to )
• Execute T on each mutant program
• If P and mutant produce different output, kill
  mutant
• If not mutant survives

30
Mutation Testing

• If large number of mutants survive, then test set
  T might be insufficient for program P
• Problems
• Need automated support for creating mutants and
  evaluating results
• A large number of mutants exist
• Equivalent mutants
• Mutant is equivalent if it produces same
  input/output response as P
• Between 4 - 10 of mutants are equivalent