Swe

1
Swe

• Testing OO Software Systems

2
Overview

• Testing OO Software Systems
• Test Models
• Black-Box Testing VS White-Box Testing
• Method Scope Coverage Models
• References
• Arthur H. Watson, Thomas J. McCabe, Structured
  Testing A Testing Methodology Using the
  Cyclomatic Complexity Metric, National Institute
  of Standards and Technology Special Publication
  500-235, 1996
• Robert v. Binder, Testing OO Systems, Addison
  Wesley, 2000, Chapter 11
• CS 696 Advanced OO Programming Design, Spring
  Semester, 1997 Lecture Notes, San Diego State
  University

3
Complexity and Testing

• Assumption
• Complexity is a common source of error in
  software.
• The more complex a module is, the more likely it
  is to contain errors.
• Beyond a certain threshold of complexity, the
  likelihood that a module contains errors
  increases sharply.
• Complexity can be used directly to allocate
  testing effort
• Assuming a connection between complexity and
  error to concentrate testing effort on the most
  error-prone software.

4
Cyclomatic Complexity I

• Cyclomatic complexity is defined for each module
  to be
• v(G) e - n 2
• where e and n are the number of edges and nodes
  in the control flow graph.
• Cyclomatic Complexity of euclid method is 3 (
  12 edges - 11 nodes 2)

5
Integration Testing

• Software systems are built from components that
  must intgeroperate.
• Integration Testing is the search for copmponent
  faults that cause intercomponent failures.
• Tests of individual components
• Tests of component interoperation
• Tests of the system running from the federation
  of components.

6
Integration Testing

• Integration Testing in object-oriented
  development begins early. Takes place at all
  scopes.
• Within a class
• Within a class hierarchy
• Between a client and its server
• Within a cluster of related classes
• Within a subsystem
• Within an application system

7
Integration Testing

• Big-Bang Integration
• Attempt to demonstrate system stability by
  testing all components at the same time
• Botton-Up Integration
• Interleave component and integration testing by
  following usage dependencies
• Top-Down Integration
• Interleave component and integration testing by
  following the application control flow
• Collaboration Integration
• Choose an order of integration according of
  collaboration and their dependencies. Exercise
  interfaces by testing one collaboration at a
  time.
• Layer Integration
• Incrementally exerise interfaces and components
  in a layered architecture.

8
Big Bang

• Context
• Brings together all components of a system
  without regard for interconnection dependency
  risks. Big Bang Testing is ok, if
• The SUT is stabilized and only a few components
  have been added or changed since last system
  scope testing.
• The SUT is small and each component has passed
  component scope testing.
• The only feasable approach for monolithic
  systems. Components are tightly coupled and
  cannot be exercised separatly.

9
Big Bang

• Fault Model
• Hope is that the system will run and thereby
  demonstrate that system testing can begin.
• Consequences
• Lets fire it up and see if it works It
  doesnt of course
• The name is somewhat misleading the anticipated
  big bang is all too often a fizzle.
• Disadvantages
• Debugging can be difficult.
• Incremental Integration The last component added
  is likely to be buggy.
• Big Bang Integration Every component is equally
  suspect.
• Advantages
• Few drivers and stube must be developed.
• Possible to demonstrate stability with only a few
  tests.

10
Bottom Up
11
Bottom Up

• Context
• Achieves stepwise verification of the interfaces
  between coupled components.
• Used to support unit scope testing in the
  iterative and incremental development of
  subsystems components.
• Must defer end-to-end, system scope testing the
  components until all components are developed.
• Early assesment of a component that must
  impolement a critical requirement.

12
Bottom Up

• Consequences
• Disadvantages
• High driver/stub development cost (throwaway
  code)
• Advantages
• Early start of integration testing, as soon as
  leaf-level components are ready
• Tests can run in parallel on independent subtrees

13
Top Down
14
Top Down

• Context
• Achieves stepwise verification of the interfaces
  among components that operate under as common
  control strategy, where top level components have
  control responsability.
• Incremental development
• Each increment adds anozther layer under the
  control layer.
• Concurrent software development
• High-level control components can be designed,
  coded, and tested while
• subsystem interfaces are designed and developed
• virtual machines are designed and developed.
• Framework development
• Resuable frameworks often implement a general
  control strategy while using abstract base
  classes to defer application-specific details.
  First increment of a framework can exercise the
  control strategy. Subsequent increments can
  exercise use cases and domain model
  responsabilities.

15
Top Down

• Consequences
• Disadvantages
• High stub development cost (throwaway code)
• It may be difficult to exercise lower-level
  composition sufficiently. The distance between
  the testing interface and the components being
  integrated increases sucessively ?Loss of test
  control over lower-level components.
• Advantages
• Early start of integration testing, as soon as
  top-level components are ready
• There is only one driver to maintain, instead of
  one driver for every subtree.
• If lower-level interfaces arre undefined or
  likely to change, top-down integration can avoid
  commitment to an unstable interface.

16
(No Transcript)
17
Collaboration

• Context
• Exercises interfaces between participants of a
  collaboration and organizes integration according
  to copllaborations.
• The SUT has clearly defined collaborations
• Demonstrates a working collaboration as soon as
  is important (technical risk, performance
  objectives, ...)

18
Collaboration

• Consequences
• Disadvantages
• Intercollaboration dependencies may be subtle and
  may not have been modeled.
• Participants in a collaboration are not exercised
  separately. In effect, colaboration testing is a
  scenario-based Big Bang Integration
• Some initial collaborations may require many
  stubd
• The specified collaborations may be incomplete
• Collaborations do not focus on pecularities of
  component interfaces, environment dependencies,
  .. Therefor, collaborarion testing may miss bugs
  ..

19
Collaboration

• Consequences
• Advantages
• Testing focuses on end-to-end funcctionality.
  Collaboreation tests may be useful for system
  tetsing
• Minimizes driver development cost, probably high
  stub costs

20
Layer Integration
21
Layer Integration
22
Layer

• Context
• Layer integration verifies iinteroperability in a
  system that can be modeled as a hierarchy that
  allows interfaces only between adjacent layers.
• Layer integration combines elements of Top-Down
  and Bottom-Up Integration

23
Layer

• Consequences
• Disadvantage
• If Bottom-Up
• Requires extensive driver development
• Complete behavior cannot be demonstrated until
  the top layer has been integrated.
• If Top-Down
• Requires extensive stub development, but only one
  driver
• Viability of top-level is not demonstrated until
  the lowest layer is integrated (risk for
  performance critical applications)

24
Integration Complexity

• The integration complexity, S1 , is defined for a
  program with n modules (G 1 through G n ) by the
  formula
• S1 (? iv(G i )) - n 1.
• Integration complexity measures the number of
  independent integration tests through an entire
  programs design.

25
Integration Complexity
26
Incremental Integration

• Hierarchical system design limits each stage of
  development to a manageable effort, and it is
  important to limit the corresponding stages of
  testing as well
• The key principle is to test just the interaction
  among components at each integration stage,
  avoiding redundant testing of previously
  integrated sub-components.
• To extend statement coverage to support
  incremental integration, it is required that all
  module call statements from one component into a
  different component be exercised at each
  integration stage.
• To form a completely flexible statement testing
  criterion, it is required that each statement be
  executed during the first phase (which may be
  anything from single modules to the entire
  program), and that at each integration phase all
  call statements that cross the boundaries of
  previously integrated components are tested.

27
Incremental Integration

• Key is to perform design reduction at each
  integration phase using just the module call
  nodes that cross component boundaries, yielding
  component-reduced graphs, and exclude from
  consideration all modules that do not contain any
  cross-component calls.