Program Correctness and Efficiency

1
Program Correctness and Efficiency

• Chapter 2

2
Chapter Objectives

• To understand the differences between the three
  categories of program errors
• To understand the effect of an uncaught exception
  and why you should catch exceptions
• To become familiar with the Exception hierarchy
  and the difference between checked and unchecked
  exceptions
• To learn how to use the try-catch-finally
  sequence to catch and process exceptions
• To understand what it means to throw an exception
  and how to throw an exception in a method

3
Chapter Objectives (continued)

• To understand the different testing strategies
  and when and how they are performed
• To learn how to write special methods to test
  other methods and classes
• To become familiar with debugging techniques and
  debugger programs
• To be introduced to the process of program
  verification and the use of assertions and loop
  invariants
• To understand the meaning of big-O notation and
  how it is used to analyze an algorithms
  efficiency

4
Program Defects and Bugs

• A program may be efficient, but is worthless if
  it produces a wrong answer
• Defects often appear in software after it is
  delivered
• Testing can never demonstrate the complete
  absence of defects
• In some situations it is very difficult to test a
  software product completely in the environment in
  which it is used
• Debugging removing defects

5
Examples of Bad Software

• Bad software is everywhere!
• NASA Mars Lander (cost 165 million)
• Crashed into Mars
• Error in converting English and metric units of
  measure
• Denver airport
• Buggy baggage handling system
• Delayed airport opening by 11 months
• Cost of delay exceeded 1 million/day
• MV-22 Osprey
• Advanced military aircraft
• Lives have been lost due to faulty software

6
Software and Complexity
LOC
system
Netscape 17,000,000
Space shuttle 10,000,000
Linux 1,500,000
Windows XP 40,000,000
Boeing 777 7,000,000

• A new car contains more lines of code (LOC) than
  was required to land the Apollo astronauts on the
  moon

7
Syntax Errors

• Syntax errors are mistakes in the grammar of a
  language
• The Java compiler detects syntax errors during
  compilation and requires you to correct them
  before successfully compiling the program
• Some common syntax errors include
• Omitting or misplacing braces
• Performing an incorrect type of operation on a
  primitive type value
• Invoking an instance method not defined
• Not declaring a variable before using it
• Providing multiple declarations of a variable

8
Run-time Errors or Exceptions

• Run-time errors
• Occur during program execution
• Occur when the JVM detects an operation that it
  knows to be incorrect
• Cause the JVM to throw an exception
• Examples of run-time errors include
• Division by zero
• Array index out of bounds
• Number format and Input mismatch error
• Null pointer exceptions

9
Run-time Errors or Exceptions (continued)
10
Logic Errors

• A logic error occurs when the programmer or
  analyst
• Made a mistake in the design of a class or method
• Implemented an algorithm incorrectly
• Most logic errors do not cause syntax or run-time
  errors and are thus difficult to find
• Sometimes found through testing
• Sometimes found during real-world operation of
  the program

11
The Exception Class Hierarchy

• When an exception is thrown, one of the Java
  exception classes is instantiated
• Exceptions are defined within a class hierarchy
  that has the class Throwable as its superclass
• Classes Error and Exception are subclasses of
  Throwable
• RuntimeException is a subclass of Exception

12
The Class Throwable

• Throwable is the superclass of all exceptions
• All exception classes inherit the methods of
  throwable

13
The Class Throwable (continued)
14
Checked and Unchecked Exceptions

• Two categories of exceptions checked and
  unchecked
• Checked exception normally not due to programmer
  error and is beyond the control of the programmer
• Unchecked exception may result from
• Programmer error
• Serious external conditions that are
  unrecoverable

15
Checked and Unchecked Exceptions
16
Catching and Handling Exceptions

• When an exception is thrown, the normal sequence
  of execution is interrupted
• Default behavior
• Program stops
• JVM displays an error message
• The programmer may override the default behavior
  by
• Enclosing statements in a try block
• Processing the exception in a catch block

17
Uncaught Exceptions

• When an exception occurs that is not caught, the
  program stops and the JVM displays an error
  message and a stack trace
• The stack trace shows the sequence of method
  calls, starting at the method that threw the
  exception and ending at main

18
The try-catch-finally Sequence

• Avoid uncaught exceptions
• Write a try-catch sequence to catch an exception
• Handle it rather than relying on the JVM
• Catch block is skipped if all statements within
  the try block execute without error

19
Handling Exceptions to Recover from Errors

• Exceptions provide the opportunity to
• Recover from errors
• Report errors
• User error is a common source of error and should
  be recoverable
• Catch block within the first catch clause having
  an appropriate exception class executes, others
  are skipped
• Compiler displays an error message if it
  encounters an unreachable catch clause

20
The finally Block

• When an exception is thrown, the flow of
  execution is suspended and there is no return to
  the try block
• There are situations in which allowing a program
  to continue after an exception could cause
  problems
• The code in the finally block is executed either
  after the try block is exited or after a catch
  clause is exited
• The finally block is optional

21
Throwing Exceptions

• Instead of catching an exception in a lower-level
  method, it can be caught and handled by a
  higher-level method
• Declare that the lower-level method may throw a
  checked exception by adding a throws clause to
  the method header
• Can throw the exception in the lower-level
  method, using a throw statement
• The throws clause is useful if a higher-level
  module already contains a catch clause for this
  exception type

22
Throwing Exceptions (continued)

• Can use a throw statement in a lower-level method
  to indicate that an error condition has been
  detected
• Once the throw statement executes, the
  lower-level method stops executing immediately

23
Catching Exceptions Example
24
Programming Style

• You can always avoid handling exceptions by
  declaring that they are thrown, or throwing them
  and letting them be handled farther back in the
  call chain
• It is usually best to handle the exception
  instead of passing it along
• The following are recommended guidelines
• If an exception is recoverable in the current
  method, handle the exception in the current
  method
• If a checked exception is likely to be caught in
  a higher-level method, declare that it can occur
  using a throws clause
• It is not necessary to use a throws clause with
  unchecked exceptions

25
Testing (Textbook View)

• There is no guarantee that a program that is
  syntax and run-time error free will also be void
  of logic errors
• The best situation is a logic error that occurs
  in a part of the program that always executes
  otherwise, it may be difficult to find the error
• The worst kind of logic error is one that occurs
  in an obscure part of the code (infrequently
  executed)

26
Structured Walkthroughs

• Most logic errors arise during the design phase
  and are the result of an incorrect algorithm
• Logic errors may also result from typographical
  errors that do not cause syntax or run-time
  errors
• One form of testing is hand-tracing the algorithm
  before implementing
• Structured walkthrough designer must explain the
  algorithm to other team members and simulate its
  execution with other team members looking on

27
Levels and Types of Testing

• Testing exercising a program under controlled
  conditions and verifying the results
• Purpose is to detect program defects after all
  syntax errors have been removed and the program
  compiles
• No amount of testing can guarantee the absence of
  defects in sufficiently complex programs
• Unit testing checking the smallest testable
  piece of the software (a method or class)
• Integration testing testing the interactions
  among units

28
Levels and Types of Testing (continued)

• System testing testing the program in context
• Acceptance testing system testing designed to
  show that the program meets its functional
  requirements
• Black-box testing tests the item based on its
  interfaces and functional requirements
• White-box testing tests the software with the
  knowledge of its internal structure

29
Preparations for Testing

• A test plan should be developed early in the
  design phase
• Aspects of a test plan include deciding how the
  software will be tested, when the tests will
  occur, who will do the testing, and what test
  data will be used
• If the test plan is developed early, testing can
  take place concurrently with the design and
  coding
• A good programmer practices defensive programming
  and includes code to detect unexpected or invalid
  data

30
Testing Tips for Program Systems

• Most of the time, you will test program systems
  that contain collections of classes, each with
  several methods
• If a method implements an interface, its
  specification should document input parameters
  and expected results
• Carefully document each method parameter and
  class attribute using comments as you write the
  code
• Leave a trace of execution by displaying the
  method name as you enter it
• Display values of all input parameters upon entry
  to a method

31
Testing Tips for Program Systems (continued)

• Display the values of any class attributes that
  are accessed by this method
• Display the values of all method outputs after
  returning from a method
• Plan for testing as you write each module rather
  than after the fact

32
Developing the Test Data

• Test data should be specified during the analysis
  and design phases for the different levels of
  testing unit, integration, and system
• In black-box testing, we are concerned with the
  relationship between the unit inputs and outputs
• There should be test data to check for all
  expected inputs as well as unanticipated data
• In white-box testing, we are concerned with
  exercising alternative paths through the code
• Test data should ensure that all if statement
  conditions will evaluate to both true and false

33
Testing Boundary Conditions

• When hand-tracing through an algorithm or
  performing white-box testing, you must exercise
  all paths
• Check special cases called boundary conditions

34
Testing

• Normally testing is done by
• The programmer
• Other members of the software team who did not
  code the module being tested
• Final users of the software product
• Do not rely on programmers for testing as they
  are often blind to their own oversights
• Companies also have quality assurance (QA)
  organizations that verify that the testing
  process is performed correctly
• In extreme programming, programmers work in pairs
  where one writes the code and the other writes
  the tests

35
Stubs

• It may be difficult to test a method or class
  that interacts with other methods or classes
• The replacement of a method that has not yet been
  implemented or tested is called a stub
• A stub has the same header as the method it
  replaces, but its body only displays a message
  indicating that the stub was called

36
Drivers

• A driver program declares any necessary object
  instances and variables, assigns values to any of
  the methods inputs, calls the method, and
  displays the values of any outputs returned by
  the method
• You can put a main method in a class to serve as
  the test driver for that classs methods

37
Using a Test Framework

• A test framework is a software product that
  facilitates writing test cases, organizing the
  test cases into test suites, running the test
  suites, and reporting the results
• A test framework often used for Java products is
  JUnit, an open-source product that can be used in
  a stand-alone mode and is available from junit.org

38
Debugging a Program

• Debugging is the major activity performed by
  programmers during the testing phase
• Testing determines if there is an error,
  debugging determines the cause of it
• Debugging is like detective work
• Inspect carefully the information displayed by
  your program
• Insert additional diagnostic output statements in
  the method to determine more information

39
Debugging Example

• What does programmer intend, and what is the
  problem?
• public static String getSentence()
• String sentence
• int count 0
• String word JOptionPane.showInputDialog(Enter
  word, to quit)
• while (word ! word ! null count lt
  10)
• sentence sentence word
• count
• word JOptionPane.showInputDialog(Enter
  word, to quit)
• return sentence

40
Using a Debugger

• Debuggers often are included with IDEs
• A debugger can execute your program incrementally
  rather than all at once
• Single-step execution executes in increments as
  small as one program statement
• Breakpoints are used to traverse large portions
  of code before stopping
• The actual mechanics of using a debugger depend
  on the IDE that you are using

41
Using a Debugger (continued)
42
Reasoning about Programs Assertions and Loop
Invariants

• Assertions logical statements about a program
  that are claimed to be true generally written as
  a comment
• Preconditions and postconditions are assertions
• A loop invariant is an assertion
• Helps prove that a loop meets it specification
• True before loop begins, at the beginning of each
  repetition of the loop body, and just after loop
  exit

43
Assertions and Loop Invariants Example
44
Testing (My View)

• Can be shown that probability of an error
  appearing after t units of testing is about
• E K/t where K is a constant
• This approximation holds over large range of t
• Then the mean time between failures is
• MTBF t/K
• The good news quality improves with testing
• The bad news quality only improves linearly with
  testing!

45
MTBF and Testing

• The mean time between failures is approximately
• MTBF t/K
• To have 1,000,000 hours without failure, must
  test (on the order of) 1,000,000 hours!
• Ordinary users are, in effect, testing the code
• What does this imply about large complex pieces
  of software?
• Consider Windows XP, for example

46
Testing

• The fundamental problem
• Developers (or good guys) must find (almost) all
  flaws/bugs
• Users (or bad guy) only need 1 flaw/bug
• Aside Software reliability far more difficult in
  security than elsewhere
• Why?
• See the next slide

47
Security Testing Do the Math

• Recall that MTBF t/K
• Suppose 106 security flaws in some software
• Say, Windows XP
• Suppose each bug has MTBF of 109 hours
• Expect to find 1 bug for every 103 hours testing
• Good guys spend 107 hours testing find 104 bugs
• Good guys have found 1 of all the bugs
• Bad guy spends 103 hours of testing finds 1 bug
• Chance good guys found bad guys bug is only 1
  !!!

48
Types of Testing

• Function testing ? verify that system functions
  as it is supposed to
• Performance testing ? other requirements such as
  speed, resource use, etc.
• Acceptance testing ? customer involved
• Installation testing ? test at install time
• Regression testing ? test after any change

49
Other Testing Issues

• Active fault detection
• Dont wait for system to fail
• Actively try to make it fail ? users/attackers
  will!
• Fault injection
• Insert faults into the process
• Even if no obvious way for such a fault to occur
• Bug injection
• Insert bugs into code
• See how many of injected bugs are found
• Can use this to estimate number of bugs
• Assumes injected bugs similar to unknown bugs

50
A Testing Case History

• In one system with 184,000 lines of code
• Flaws found
• 17.3 inspecting system design
• 19.1 inspecting component design
• 15.1 code inspection
• 29.4 integration testing
• 16.6 system and regression testing
• Conclusion must do many kinds of testing
• Overlapping testing is necessary
• Provides a form of defense in depth

51
Testing The Bottom Line

• Testing is inherently difficult
• Security testing far more demanding than
  non-security testing
• Non-security testing ? does system do what it is
  supposed to?
• Security testing ? does system do what it is
  supposed to
• and nothing more?
• Usually impossible to do exhaustive testing
• How much testing is enough?

52
Testing The Bottom Line

• How much testing is enough?
• Recall MTBF t/K
• Seems to imply testing is nearly hopeless!
• But there is some hope
• If we can eliminate an entire class of flaws then
  statistical model breaks down
• For example, if we have a single test (or a few
  tests) to eliminate all buffer overflows

53
Efficiency of Algorithms

• Difficult to get a precise measure of the
  performance of an algorithm or program
• Can characterize a program by how the execution
  time or memory requirements increase as a
  function of increasing input size
• Big-O notation
• A simple way to determine the big-O of an
  algorithm or program is to look at the loops and
  to see whether the loops are nested

54
Efficiency of Algorithms

• Consider
• for (int i 0 ilt x.length i)
• if(xi target)
• return i
• Suppose n is x.length
• Then work is T(n) n
• Since 1 statement each time thru the loop

55
Efficiency of Algorithms

• Consider
• for (int i 0 ilt x.length i)
• for (int j 0 jlt x.length j)
• if(i ! j xi xj)
• return false
• Suppose n is x.length
• Then work is T(n) n2

56
Efficiency of Algorithms (continued)

• Consider
• First time through outer loop, inner loop is
  executed n-1 times next time n-2, and the last
  time once.
• So we have
• T(n) 3(n 1) 3(n 2) 3 or
• T(n) 3(n 1 n 2 1)

57
Efficiency of Algorithms (continued)

• We can reduce the expression in parentheses to
• n x (n 1)
• 2
• So, T(n) 1.5n2 1.5n
• This polynomial is zero when n is 1. For values
  greater than 1, 1.5n2 is always greater than
  1.5n2 1.5n
• Therefore, we say that T(n) is O(n2)
• This is big O notation
• Upper bound on performance
• Assumes n is big enough, and up to constant
  factor
• For example, T(n) n2 5n 25 is O(n2)

58
Efficiency of Algorithms (continued)
59
Chapter Review

• Three kinds of defects can occur in programs
• Syntax errors
• Run-time errors
• Logic errors
• All exceptions in the Exception class hierarchy
  are derived from a common superclass called
  Throwable
• The default behavior for exceptions is for the
  JVM to catch them by printing an error message
  and a call stack trace and then terminating the
  program

60
Chapter Review (continued)

• Two categories of exceptions checked and
  unchecked
• A method that can throw a checked exception must
  either catch it or declare that it is thrown
• throw statement throws an unchecked exception
• Program testing is done at several levels
  starting with the smallest testable piece of a
  program called a unit
• Integration testing once units are individually
  tested, they can then be tested together
• System testing once the whole program is put
  together, it is tested as a whole

61
Chapter Review (continued)

• Acceptance programming involves testing in an
  operational manner demonstrating its
  functionality
• Black-box testing tests the item based on its
  functional requirements without knowledge of its
  internal structure
• White-box testing tests the item using knowledge
  of its internal structure
• Test drivers and stubs are tools used in testing
• Test drivers exercise a method or class
• Stubs stand in for called methods
• Big-O notation determines the efficiency of a
  program