ObjectOriented Software Engineering

1
Object-Oriented Software EngineeringPractical
Software Development using UML and Java
Chapter 10 Testing and Inspecting to Ensure Hig
h Quality Part 1 Basic Definitions Effective
and Efficient Testing
Lots of Personal Notes
2

• Testing and Inspections two very important
  concepts / activities in software development.
  
• We often have roles of testers
• individual developers members of development
  teams
  
• independent test groups quality assurance
  groups, etc.
  
• Quality Control Editors
• We have many different kinds / types of tests
  too.
  
• Exist at many different levels for many
  different stakeholders.
  
• Some are automated many are hands-on
• Some test parts of system others whole
• Some at different stages of development .
• Many Many names!!!!
• Note a successful test is one that
  identifies a flaw.
  
• We will present a lot of very important material
  in these last lectures.

3
Tests

• White box testing black box testing
• Verification and validation (VV)
• Branch testing, path testing, statement testing
• Alpha testing, beta testing
• Acceptance testing
• Unit test, subsystem test, system test,
  integrated system test,
  
• Independent test groups
• Environmental system testing
• Operational Field Testing
• Running in parallel running live

4
Organizational Considerations

• DIREP
• Deficiency Report
• Tracked, prioritized, resources allocated...
• Incident Report
• Problem Ticket
• Fix Ticket
• Office of Primary Responsibility (OPR) Office
  of Collateral Responsibility (OCR)
  
• Product Manager..

5
10.1 Basic definitions

• A failure is an unacceptable behaviour exhibited
  by a system
  
• The frequency of failures measures the
  reliability
  
• An important design objective is to achieve a
  very low failure rate and hence high
  reliability.
  
• A failure can result from a violation of an
  explicit or implicit requirement
  
• An extreme and easily understood kind of failure
  is an outright crash!!
  
• We clearly want to reduce the frequency of
  failures (MTBF) and damage caused by failures.
  (Mean Time Between Failures)
  
• We measure failures and time to repair (MTTR) as
  measures of reliability. (Mean Time to Repair)
  
  

6

• A defect is a flaw in any aspect of the system
  that contributes, or may potentially contribute,
  to the occurrence of one or more failures
  
• It might take several defects to cause a
  particular failure
  
• Note also that a defect can occur anywhere
  requirements, design, implementation, testing,
  etc.
  
• True Story Mr. Smith said this couldnt
  happen!
  
• An error is a slip-up or inappropriate decision
  by a software developer that leads to the
  introduction of a defect

7
10.2 Effective and Efficient Testing

• To test effectively, you must use a strategy that
  uncovers as many defects as possible.
  
• Testing really attempts to find defects! Pure
  and simple!
  
• To test efficiently, you must find the largest
  possible number of defects using the fewest
  possible tests.
  
• This is where the problems come in. There are
  many kinds of tests designed to uncover different
  kinds of defects.
  
• Testing must be efficient. We cannot test
  everything. Rather, we design different kinds of
  tests for sufficient coverage.
  
• All kinds of tests Some are lumped into
  categories called Coverage Tests.
  
• statement coverage
• branch coverage
• path coverage.. We will discuss ahead.

8
Black-box testing

• Testers provide the system with inputs and
  observe the outputs
  
• They can see none of
• The source code
• The internal data
• Any of the design documentation describing the
  systems internals
  
• Simply putting in inputs and observing outputs.
  Bouncing these against the requirements.
  
• This is the kind of testing that end-users most
  frequently undertake for obvious reasons.
  Called validation testing / acceptance testing -
  IF done when system is declared finished.
• May be called other things (alpha testing beta
  testing)
  
• Black box testing is designed to show
  functionality and (often to a lesser degree at
  this time but not always) satisfaction of
  non-functional requirements. (performance,
  load) Generally, black box testing is designed
  to verify functionality!!

9
Who does this?

• Clearly, end users.
• BUT developers must do their own verification
  of functionality before turning this loose!
  
• Developers verification
• Customers validation
• Hence VV.

10
Glass-box testing White-box testing

• Also called white-box or structural testing
• Developers and Testers have access to the system
  design
  
• They can
• Examine the design documents
• View the code
• Observe at run time the steps taken by algorithms
  and their internal data
  
• Individual programmers often informally employ
  glass-box testing to verify their own code.

11
More White Box testing ? Unit Testing

• Developers are always responsible for their own
  modules.
  
• Developers verify their products.
• Usually called unit test
• But there are many kinds of unit testing

12
Glass-Box (White Box) Testing - More

• Path Testing exhaustive impossible there is
  an infinite number of paths in a non-trivial
  program.
  
• Branch testing Edge testing the most
  feasible
  
• Design a test so that all of the edges of a node
  are
  
• executed.
• What do we mean by edge?
• Consider a flowgraph of an algorithm.
• (Think of a program flowchart)

13
White box Flow Graphs

• Want a minimum number of tests that assures a
  high degree of reliability.
  
• Know we cannot test all paths in a non-trivial
  program.
  
• Want a representative set.
• Given a flow graph, there are a couple of
  formulas that will provide the minimum number of
  tests (like number of nodes number of edges
  1) (?) and others.
• One famous way is the Cyclomatic Complexity of a
  Flowgraph.
  
• Defined to be number of regions of the graph
  1
  
• The number of regions is called the cyclomatic
  complexity of the routine.
  

14
Equivalence Class Testing

• This is a Bigee!
• It is inappropriate to test by brute force, using
  every possible input value
  
• Discussed Takes a huge amount of time
• Is impractical and is pointless!
• Far better divide the possible inputs into
  groups which you believe will be treated
  similarly by all algorithms.
  
• Such groups are called equivalence classes.
• Recall ramdomizing in File Structures
  (hashing).
  
• If we used an Equivalence mod 4, we essentially
  divided all remainders into one of five sets
  those with remainders 0, 1, 2, 3, and 4.
  
• ALL divisions resulted in one of these
  remainders.
  
• We applied this equivalence relation to the set
  of integers and obtained a set of equivalence
  classes the Union of which constitutes the
  original set of Integers.
• EC1 U EC2 U ECn set of I

15
Examples of equivalence classes

• A tester needs only to run one test per
  equivalence class
  
• The tester has to
• understand the required input,
• appreciate how the software may have been
  designed
  
• Example
• Valid input is a month number (1-12)
• Equivalence classes are -8..0, 1..12, 13..
  8
  
• These are the three equivalence classes.
• In Equivalence Class Testing, we select inputs
  from each of the equivalence classes and include
  a representative value from EACH equivalence
  class.

16
More on Equivalence Class Testing

• So, for valid integers ranging from 1 through 12,
  equivalence classes are as indicated (above).
  
• Integers less than 1
• Integers between 1 and 12 inclusive
• Integers greater than 12.
• (Is this enough?)
• What if there is more than a single value we are
  testing?
  

17
Combinations of equivalence classes

• Combinatorial explosion means that you cannot
  realistically test every possible system-wide
  equivalence class.
  
• If there are 4 inputs with 5 possible values
  there are 54 (i.e.625) possible system-wide
  equivalence classes.
  
• While you should first make sure that at least
  one test is run with every equivalence class of
  every individual input, you should also test all
  combinations where an input is likely to affect
  the interpretation of another.
• Compatibility tests Associativity tests.
• Examples field non-zero rpi, ? rank had to
  be officer (grade O1 through O10)
  
• Range tests If grade O4, salary must fall
  between nnnn and nnnnnn.
  
• Testing multiple fields all white box!!!

18
Testing at boundaries of equivalence classes
Boundary Value Testing

• More errors in software occur at the boundaries
  of equivalence classes than at any other place!!
  AMEN!!!
  
• The idea of equivalence class testing should be
  expanded to specifically test values at the
  extremes of each equivalence class
  
• E.g. The number 0 often causes problems
• ? E.g. If the valid input is a month number
  (1-12)
  
• Test equivalence classes as before
• Test -24 24, and a 6 (representative values)
  
  
• Test boundaries 0, 1, 12 and 13 as well as very
  large positive and negative values

19
Coming

• Defects in ordinary algorithms.
• There are many of these. Read carefully.