Chapter 10 Testing and Debugging

1
Chapter 10 Testing and Debugging
2
Chapter Goals

• Learn techniques to test your code
• Learn to carry out unit tests
• Understand principles of test case selection and
  evaluation
• Learn to use logging
• Become familiar with a debugger

3
(No Transcript)
4
10.1 Unit Tests

• The single most important testing tool
• Checks a single method or a set of cooperating
  methods
• You don't test the complete program that you are
  developing you test the classes in isolation

5
Unit testing

• Avoids confusion of interactions
• More startup cost, but less work in the end
• Analogy Testing car

6
Test Harness

• For each test, you provide a simple class called
  a test harness
• Test harness feeds parameters to the methods
  being tested

7
Example

• To compute the square root of a use a common
  algorithm
• Guess a value x that might be somewhat close to
  the desired square root (x a is ok)
• Actual square root lies between x and a/x
• Take midpoint (x a/x) / 2 as a better guess
• Repeat the procedure. Stop when two successive
  approximations are very close to each other

8

• Method converges rapidly. Square root of 100
  Guess 1 50.5Guess 2 26.24009900990099Guess
  3 15.025530119986813Guess 4
  10.840434673026925Guess 5 10.032578510960604Gu
  ess 6 10.000052895642693Guess 7
  10.000000000139897Guess 8 10.0

9
Testing

• RootApproximator class
• getRoot()
• nextGuess()
• Numeric class
• approxEqual(double, double)

10

• 06public class RootApproximatorTester
• 07
• 08 public static void main(String args)
• 09
• 10 System.out.print("Enter a number ")
• 11 Scanner in new Scanner(System.in)
• 12 double x in.nextDouble()
• 13 RootApproximator r new RootApproximator(x)
  
• 14 final int MAX_TRIES 10
• 15 for (int tries 1 tries lt MAX_TRIES
  tries)
• 16
• 17 double y r.nextGuess()
• 18 System.out.println("Guess "tries " "
  y)
• 19
• 20 System.out.println("Square root "
  r.getRoot())
• 21
• 22

11
Harness

• What is needed to make the testing more robust?
• More tests
• Limitation Hard to replicate tests when bugs are
  fixed
• Solution Use test harness to repeat values

12
10.2 Providing Test Input

• Solution 1 Hardwire series of tests
• No need to memorize series of tests, already
  taken care of. Just run tester class.

13

• public class RootApproximatorHarness1
• public static void main(String args)
• double testInputs 100, 4, 2, 1, 0.25,
  0.01
• for (int x 0 x lt testInputs.length x)
• RootApproximator r new RootApproximator
  (testInputsx)
• double y r.getRoot()
• System.out.println("square root of "
  testInputsx " " y)

14
Generate Test Cases Automatically

• Instead of hardcoding array of values
• Loop through a sample range of values
• for (int x MIN x lt MAX x INCREMENT)
• RootApproximator r new RootApproximator(x)
• double y r.getRoot()
• System.out.println("square root of " x "
  " y)

15
Generate Test Cases Automatically

• Instead of hardcoding array of values
• Use random number generator
• final int SAMPLES 100
• Random generator new Random()
• for (int i 0 i lt SAMPLES i)
• double x 1000 generator.nextDouble()
• RootApproximator r new RootApproximator(x)
• double y r.getRoot()
• System.out.println("square root of " x "
  " y)

16
What to Test for?

• Good testing requires testing good cases
• Reduces debugging time and ensures better product
• Test all of the features of the method
• Positive tests normal behavior of method
• Boundary test cases (e.g. x 0), make sure end
  conditions are correct
• Negative cases program should reject

17
Advanced 10.1

• Common alternative to hardcoding values Read
  them from a text file
• Use redirection of System.in to a file instead of
  console

18
10.3 Test Case Evaluation

• How do you know whether the output is correct?
• Calculate correct values by hand E.g., for a
  payroll program, compute taxes manually
• Supply test inputs for which you know the answer
  E.g., square root of 4 is 2 and square root of
  100 is 10
• Verify that the output values fulfill certain
  properties E.g., square root squared original
  value

19

• for (int i 1 i lt SAMPLES i)
• double x 1000 generator.nextDouble()
• RootApproximator r new RootApproximator(x)
• double y r.getRoot()
• if (Numeric.approxEqual(y y, x))
• System.out.print("Test passed ")
  passcount
• else
• System.out.print("Test failed ")
• failcount
• System.out.println("x " x 37 ", root
  squared " y y)

20

• Use an Oracle a slow but reliable method to
  compute a result for testing purposes E.g., use
  Math.pow to slower calculate x1/2 (equivalent to
  the square root of x)

21

• for (int i 1 i lt SAMPLES i)
• double x 1000 generator.nextDouble()
• RootApproximator r new RootApproximator(x)
• double y r.getRoot()
• double oracleValue Math.pow(x, 0.5)
• if (Numeric.approxEqual(y,oracleValue))
• System.out.print("Test passed ")
  passcount
• else
• System.out.print("Test failed ")
• failcount

22
10.4 Regression Testing and Test Coverage

• Save test cases
• Particularly tests that reveal bugs
• Use saved test cases in subsequent versions
• A test suite is a set of tests for repeated
  testing

23
Why keep a test case?

• Very common for bugs to show up later
• We often think we fixed a bug, but just covered
  it up
• Cycling bug that is fixed but reappears in
  later versions
• Regression testing repeating previous tests to
  ensure that known failures of prior versions do
  not appear in new versions

24
Test Coverage

• Black-box testing test functionality without
  consideration of internal structure of
  implementation
• Useful since this is what the user interacts with
• White-box testing take internal structure into
  account when designing tests
• Unit testing
• Why? Can not prove absence of bugs, only presence

25

• Test coverage measure of how many parts of a
  program have been tested
• Good testing tests all parts of the program
• E.g. every line of code is executed in at least
  one test
• Example Test all possible branches inside of an
  if statement
• In Tax code program, 6 tests (3 brackets 2
  types of status)

26
Testing Levels

• The part of the program that is being tested.
• System
• Unit

27
System Level Testing

• Test interactions of the various parts of the
  application.
• Completed after the unit tests pass.

28
Unit Level Testing

• Test individual units (classes) in isolation.
• Test features of the unit.
• Multiple test cases for each feature.

29
Black Box Testing

• a.k.a. behavioral, functional, closed
• against the specification
• does not need the program source
• internal implementation is unknown

30
Black Box Testing

• Advantages
• designed from the specifications
• user's point of view
• less biased
• can discover if part of the specification has not
  been fulfilled.

31
Black Box Testing

• Disadvantages
• may be redundant
• can be difficult to design.
• testing every possible input stream is unrealistic

32
White Box Testing

• a.k.a. structural, clear box, open
• tests against the implementation
• every line of source code is executed at least
  once.
• tests exception handling code.

33
White Box Testing

• Advantages
• can discover if any part is faulty
• test coverage (test all paths)
• Disadvantages
• will not discover if part is missing
• only if the programmer knows what the program is
  supposed to do
• code must also be visible to the tester.

34
"Fully" Tested

• To fully test a software product,the following
  are required
• unit testing and system level testing
• black and white box testing
• Does this ensure there are no bugs?

35

• Develop cases to take into account end conditions
• Tip write first test cases before program is
  written completely ? gives insight into what
  program should do
• Build up as program goes along and bugs are found

36
10.6 Logging

• To figure out where your program goes, use
  program trace statements
• Help determine the flow of execution
• if (status SINGLE)
• System.out.println("status is SINGLE"). . .

37
Problem

• When done debugging, have to remove all
  System.out.println trace messages
• What if we find a bug again?
• Solution Create a separate class for the purpose
  of logging

38
Logging

• Logging messages can be deactivated when testing
  is complete
• Use global object Logger.global in place of
  System.out
• Log a message
• Logger.global.info("status is SINGLE")

39
Turn off/on

• How does this class help?
• Can easily switch logging on and off
• Default On
• To turn off, insert at top of main()
• Logger.global.setLevel(Level.OFF)

40
What to print out

• Common to need to know enter/exit conditions of
  methods
• public TaxReturn(double anIncome, int
  aStatus) Logger.global.info("Parameters
  anIncome " anIncome " aStatus "
  aStatus) . . .
• //Exit
• public double getTax(). . .Logger.global.info("
  Return value " tax)return tax

41
Logging

• Obviously other output is needed
• Bug is that certain condition isnt true when it
  is supposed to be track values as you go along
• Logging class has many other options (see API)
• Disadvantages
• Time consuming
• Too much/little output

42
10.6 Debugger

• To avoid logging problems, most professionals use
  a debugger
• Programs rarely (never) run perfectly the first
  time
• Think of perfect first draft to paper
• The larger your programs, the harder to debug
  them simply by logging

43

• Most current development environments contain a
  debugger
• Debugger - program to run your program and
  analyze its run-time behavior
• A debugger lets you stop and restart your
  program, see contents of variables, and step
  through it

44

• Debuggers can be part of your IDE (Eclipse,
  BlueJ) or separate programs (JSwat)
• Three key concepts
• Breakpoints
• Single-stepping
• Inspecting variables

45
Breakpoint

• Breakpoint the debugger doesnt stop running
  the program until it hits a defined breakpoint
• Setup by programmer
• Once stopped, you can see the state of all the
  variables

46
(No Transcript)
47
Step-through

• Once a breakpoint is hit, two options
• Step through the next few statements carefully
  inspecting
• Slow, but useful in heavy calculations
• Single-step line by line execution
• Step into doesnt just skip line to line, but
  goes into method calls within each line
• Step over Execute the enter statement and stop
• Run at full speed until the next breakpoint

48
(No Transcript)
49
Example

• Current line
• String input in.next()Word w new
  Word(input)int syllables w.countSyllables()
  System.out.println("Syllables in " input " "
  syllables)

50
Step Over

• Next step
• String input in.next()Word w new
  Word(input)int syllables w.countSyllables()
  System.out.println("Syllables in " input " "
  syllables)

51
Step into

• public int countSyllables()
• int count 0
• int end text.length() - 1
• . . .

52
Which to choose

• If the method is suspect(may be cause of problem)
  Step into
• If you are sure the method works correctly step
  over

53
What if a test fails?

• It's time to debug!
• We will want (need) a strategy.
• Debugging Strategy
• A systematic way to find and fix bugs.

54
Debug Strategy 1 Trial Error

• Write code
• Run program
• Fix bugs
• Advantages
• no planning required
• Disadvantages
• Spend more time trying to find and fix bugs than
  you did writing the program.

55
Debug Strategy 2 Trial Error with a Debugger

• Write code
• Run program
• Use Debugger program to fix bugs
• Advantages
• no planning required
• Disadvantages
• Spend lots of time in the debugger program trying
  to find and fix the bugs

56
Debug Strategy 3Incremental Development with
Testing

• Write small amount of code with testing and
  debugging in mind.
• Document and write tests for each new piece of
  code.
• Run tests on new code
• Fix any bugs before adding new code.
• Disadvantage Sometimes you have to debug code
  that you didn't write.

57
Debug Strategy 3Incremental Development with
Testing

• Disadvantages
• Takes more time (harder) to design the solution
• Must write more code (tests)
• Only possible if you're the one who wrote the
  program
• Advantages
• A small amount of code to review and fix.
• Bug is fixed before it get hidden by some other
  bug.
• If small parts work, the bigger parts will
  usually work too. If not, it's only the public
  interfaces that have to be debugged.

58
Debugging Strategy 4Test and use a Debugger

• Identify the bug. Must be repeatable and
  testable.
• Write test(s) that fails for the bug.
• Locate the source of the bug. HOW?
• Understand the problem before you fix it.
• Try your fix and run all tests.
• Repeat until all tests pass.

59
How do we find the sourceof the bug?

• Pick a point to start looking
• Beginning (main method)
• Divide and conquer
• Trace the code. (Look for variables with
  unexpected values)
• Manual trace
• Trace messages
• System.out.println()
• Java 1.5 Use java.util.logging.Logger class
• Debugger program