Integration testing

1
Integration testing

• Objective ensure assembled modules, that work
  fine in isolation, work together
• Attempt to find interface faults
• Need a module call graph.

main
A
B
C
D
E
F
2
Integration Strategy

• How low-level modules are assembled to form
  higher-level program entities
• Strategy impacts on
• the form in which units test-cases are written
• the type of test tools to use
• the order of coding/testing units
• the cost of generating test cases
• the cost of locating and correcting detected
  defects

3
Integration testing strategies

• Several different strategies can be used for
  integration testing.
• Comparison criteria
• fault localization
• effort needed (for stubs and drivers)
• degree of testing of modules achieved
• possibility for parallel development
• Examples of strategies
• Big-bang
• Top-down
• Bottom-up
• Sandwich

4
Big Bang Integration

• Non-incremental strategy
• Unit test each module in isolation
• Integrate as a whole

test main
test A
test B
test main, A, B, C D, E, F
test C
test D
test E
test F
5
Big Bang Integration

• Advantages
• Convenient for small systems
• Disadvantages
• Need driver and stubs for each module
• Integration testing can only begin when all
  modules are ready
• Fault localization difficult
• Easy to miss interface faults

6
Top-down Integration

• Incremental strategy
• Start by including highest level modules in test
  set.
• All other modules replaced by stubs or mock
  objects.
• Integrate (i.e. replace stub by real module)
  modules called by called by modules in test set.
• Repeat until all modules in test set.

test main
test main, A, B,C
test main, A, B, C D, E, F
7
Top-down Integration

• The integration order can be modified to
• include critical modules first
• leave modules not ready to later

test main
test main, A, C
test main, A, C, D, E, F
test main, A, B, C D, E, F
8
Top-down Integration

• Advantages
• Fault localization easier
• Few or no drivers needed
• Possibility to obtain an early prototype
• Different order of testing/implementation
  possible
• Major design flaws found first
• in logic modules on top of the hierarchy
• Disadvantages
• Need lot of stubs / mock objects
• Potentially reusable modules (in bottom of the
  hierarchy) can be inadequately tested

9
Bottom-up Integration

• Incremental strategy
• Test low-level modules, then
• Modules calling them until highest level module

10
Bottom-up Integration

• Advantages
• Fault localization easier (than big-bang)
• No need for stubs / fewer mock objects
• Logic modules tested thoroughly
• Testing can be in parallel with implementation
• Dis-advantages
• Need drivers
• High-level modules (that relate to the solution
  logic) tested in the last (and least)
• No concept of early skeletal system

11
Sandwich Integration

• Combines top-down and bottom-up approaches
• Distinguish 3 layers
• logic (top) - tested top-down
• middle
• operational (bottom) tested bottom-up

12
Modified Sandwich Integration

• Combines top-down and bottom-up approaches
• Modules tested first in isolation before
  integration

13
Risk Driven integration

• Integrate base on criticality
• most critical or complex modules integrated first
  with modules called

14
Function/thread integration

• Integrate modules according to threads/functions
  they belong to

15
Integration of Object-Oriented Systems

• Different scopes
• Intra Class (methods within a class)
• Big-bang
• Bottom-up constructors - accessors - predicates
  - modifiers - iterators destructors
• State based integration
• Cluster Integration
• Big Bang
• Bottom-up
• Top-down
• Scenario based
• Subsystem/System Integration
• Big Bang
• Bottom-up
• Top-down
• Use case based

16
Intra-Class Integration

• Operations in classes (methods)
• black box, white box approaches
• each exception raised at least once
• each interrupt forced to occur at least once
• each attribute set and interogated at least once
• state-based testing
• Integration of methods
• Big-bang indicated for situation where methods
  are tighly coupled
• complex methods can be tested with stubs/mocks

17
Cluster Integration

• Needs a class dependency tree

FinancialService
Transaction
Account
Rate
Money
AcctNum
IntCollection
18
Cluster Integration Big Bang approach

• Recommended only when
• Cluster is stable, only few elements added or
  changed
• Small cluster
• Components are very tighly coupled
• Impossible to test separately in practice

19
Cluster IntegrationBottom-up approach

• Most widely used technique
• Integrate classes
• starting from leaves and
• moving up in
• dependency tree

test IntCollection
test IntCollection, Rate
test Transaction, Rate,Money, IntCollection
test Money
test FinancialService, Transaction, Account,Money
AcctNum,Rate IntCollection
test Account,Money AcctNum
test AcctNum
20
Cluster IntegrationTop-down Integration

• Possible only when there is a head class
• Example cluster with façade class

21
Cluster IntegrationScenario-based approach

• Scenario
• describes how classes interact
• represented using interaction diagrams
• Approach
• Map collaborations onto dependency tree
• Choose a sequence to apply collaborations. For
  example
• simplest first
• fewest stubs first
• etc.
• Develop and run test exercising collaboration

22
Cluster IntegrationScenario based approach

• Collaboration 1
• FinancialService, Transaction, Rate, InCollection
• Stubs for Account, Money, AcctNum
• Collaboration 2
• FinancialService, Account, Money, AcctNum
• Stubs for Transaction, Rate, IntCollection

FinancialService
Transaction
Account
Rate
Money
AcctNum
IntCollection
23
Problems related to inheritance (1)

• Makes understanding code more difficult
• Instance variables similar to global variables
• Test suite for a method almost never adequate for
  overriding methods
• Accidental reuse
• Using an inherited method when it was not
  intended to do so.

24
Problems related to inheritance (2)

• Multiple inheritance
• Multiple test-cases needed
• Possibility of incorrect binding
• Repeated inheritance
• Abstract classes
• Cannot be instantiated directly.
• Generic classes

25
Example

• Coincidental correctness possible
• Necessity to test inherit method in the context
  of inheriting classes

public class Account extends Object
protected Date lastTxDate, today int
quartersSinceLastTx() return
(90/daysSinceLastTx()) int
daysSinceLastTx() return (today.day()
- lastTxDay.txDate 1) // Correct -
today's transaction return 1 //
day elapsed public class
TimeDepositAccount extends Account int
daysSinceLastTx() return
daysSinceLastTx(today.day() -
lastTxDay.txDate) // Incorrect - today's
transactions // return 0 days
elapsed
26
Subsystem/System Integration

• Big Bang integration
• not generally recommended
• Bottom-up integration
• works well for small/medium systems
• Top-down integration
• Use case based integration
• based on high-level (external) use cases

27
Client/Server integrationTwo tier System

• Test each client with stubs for servers
• Test server with stubs for each client types
• Test pairs of client type with the actual server
  (server may have stubs for the other clients)
• Remove all stubs and test server with clients

client 1
client 3
server
client 2
client 4
28
Client/Server integrationThree-tier System

• Test each client with stubs for servers, and the
  middle-tier.
• Test server with stubs for each client types, and
  the middle-tier.
• Test each client with middle-tier and server
  proxy.
• Test server with middle-tier and client proxy.
• Test clients with middle-tier and the actual
  server.

client1
client2
client3
client4
Middle Tier
server