Integration and Higher Level Testing

1
Integration and Higher Level Testing
Software Testing and Verification Lecture 11

• Prepared by
• Stephen M. Thebaut, Ph.D.
• University of Florida

2
Context

• Higher-level testing begins with the integration
  of (already unit-tested) modules to form
  higher-level program entities (e.g., components).
• The primary objective of integration testing is
  to discover interface and blatant higher-level
  design errors associated with the elements being
  integrated. (Somewhat in keeping with the spirit
  of smoke testing.)

3
Context

• Higher-level testing begins with the integration
  of (already unit-tested) modules to form
  higher-level program entities (e.g., components).
• The primary objective of integration testing is
  to discover interface and blatant higher-level
  design errors associated with the elements being
  integrated. (Somewhat in keeping with the spirit
  of smoke testing.)

4
Context (contd)

• Once the elements have been successfully
  integrated (i.e., once they are able to function
  together), the functional and non-functional
  characteristics of the higher-level element can
  be tested thoroughly (via component, product, or
  system testing).

5
Integration Testing

• Integration testing is carried out when
  integrating (i.e., combining)
• Units or modules to form a component
• Components to form a product
• Products to form a system
• The strategy employed can significantly affect
  the time and effort required to yield a working,
  higher-level element.

6
Integration Testing

• Integration testing is carried out when
  integrating (i.e., combining)
• Units or modules to form a component
• Components to form a product
• Products to form a system
• The strategy employed can significantly affect
  the time and effort required to yield a working,
  higher-level element.

7
Integration Testing (contd)

• Note that integration testing is sometimes
  defined as the level of testing between unit and
  system. We use a more general model of the
  testing process.

8
Levels of Testing
A More General View
System Test
A Popular View
Integration Test
System Test
Product Test
Integration Test
Integration Test
Unit Test
Component Test
Integration Test
Unit Test
9
Integration Testing Strategies

• The first (and usually the easiest...) issue to
  address is the choice between instantaneous and
  incremental integration testing.
• The former is sometimes referred to as the big
  bang approach. (Guess why!) Locating subtle
  errors can be very difficult after the bang.
• Integration is a process, not an event.

10
Integration Testing Strategies

• The first (and usually the easiest...) issue to
  address is the choice between instantaneous and
  incremental integration testing.
• The former is sometimes referred to as the big
  bang approach. (Guess why!) Locating subtle
  errors can be very difficult after the bang.
• Integration is a process, not an event.

11
Integration Testing Strategies

• The first (and usually the easiest...) issue to
  address is the choice between instantaneous and
  incremental integration testing.
• The former is sometimes referred to as the big
  bang approach. (Guess why!) Locating subtle
  errors can be very difficult after the bang.
• Integration is a process, not an event.

12
Integration Testing Strategies (contd)

• Incremental integration testing results in some
  additional overhead, but can significantly reduce
  error localization and correction time. (What is
  the overhead?)
• The optimum incremental approach is inherently
  dependent on the individual project and the pros
  and cons of the various alternatives available.

13
Integration Testing Strategies (contd)

• Incremental integration testing results in some
  additional overhead, but can significantly reduce
  error localization and correction time. (What is
  the overhead?)
• The optimum incremental approach is inherently
  dependent on the individual project and the pros
  and cons of the various alternatives available.

14
As an aside

• Heres a slide from an on-line lecture that I
  came
• across recently

Principles of Integration

• Proper policy and plans
• Advocacy
• Manpower training
• Realistic tasks

• Coordination with other
• sectors
• Proper support
• Access to drugs

Integration of Substance Abuse Disorders in
National Rural Health Mission (NRHM), Centre
for Community Medicine, All India Institute
of Medical Sciences, New Delhi
15
Incremental Strategies

• Suppose we are integrating a group of modules to
  form a component, the control structure of which
  will form a calling hierarchy as shown.

16
Incremental Strategies (contd)

• In what order should the modules be integrated?
• From the top (root) module toward the bottom?
• From bottom (leaf) modules toward the top?
• By function?
• Critical or high-risk modules first?
• By availability?

17
Incremental Strategies (contd)

• In what order should the modules be integrated?
• From the top (root) module toward the bottom?
• From bottom (leaf) modules toward the top?
• By function?
• Critical or high-risk modules first?
• By availability?

18
Incremental Strategies (contd)

• In what order should the modules be integrated?
• From the top (root) module toward the bottom?
• From bottom (leaf) modules toward the top?
• By function?
• Critical or high-risk modules first?
• By availability?

19
Incremental Strategies (contd)

• In what order should the modules be integrated?
• From the top (root) module toward the bottom?
• From bottom (leaf) modules toward the top?
• By function?
• Critical or high-risk modules first?
• By availability?

20
Incremental Strategies (contd)

• In what order should the modules be integrated?
• From the top (root) module toward the bottom?
• From bottom (leaf) modules toward the top?
• By function?
• Critical or high-risk modules first?
• By availability?

21
Incremental Strategies (contd)

• In what order should the modules be integrated?
• From the top (root) module toward the bottom?
• From bottom (leaf) modules toward the top?
• By function?
• Critical or high-risk modules first?
• By availability?

22
Incremental Strategies (contd)

• How many should be combined at a time?
• What scaffolding (i.e., drivers and stubs to
  exercise the modules, and oracles to
  interpret/inspect test results) will be required?
• Is scaffolding ever required outside the context
  of integration testing?

23
Incremental Strategies (contd)

• How many should be combined at a time?
• What scaffolding (i.e., drivers and stubs to
  exercise the modules, and oracles to
  interpret/inspect test results) will be required?
• Is scaffolding ever required outside the context
  of integration testing?

24
Incremental Strategies (contd)

• How many should be combined at a time?
• What scaffolding (i.e., drivers and stubs to
  exercise the modules, and oracles to
  interpret/inspect test results) will be required?
• Is scaffolding ever required outside the context
  of integration testing?

25
Top-Down Strategy

• Start with the root and one or more called
  modules.
• Test this group using stubs to take the place of
  missing called modules, and one driver (if
  necessary) to call the root module.
• Add one or more other called modules, replacing
  and providing new stubs as necessary.
• (contd)

26
Top-Down Strategy

• Start with the root and one or more called
  modules.
• Test this group using stubs to take the place of
  missing called modules, and one driver (if
  necessary) to call the root module.
• Add one or more other called modules, replacing
  and providing new stubs as necessary.
• (contd)

27
Top-Down Strategy

• Start with the root and one or more called
  modules.
• Test this group using stubs to take the place of
  missing called modules, and one driver (if
  necessary) to call the root module.
• Add one or more other called modules, replacing
  and providing new stubs as necessary.
• (contd)

28
Top-Down Strategy (contd)

1. Continue the process until all elements have been
   integrated and tested.

29
Top-Down Strategy (contd)
A
B
C
D
C
D
E
F
G
H
I
G
H
I
J
K
L
30
Top-Down Strategy (contd)
driver
A
B
stub
stub
stub
stub
31
Top-Down Strategy (contd)
driver
A
B
stub
C
C
stub
stub
stub
stub
32
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
stub
stub
stub
stub
stub
stub
33
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
stub
stub
stub
stub
stub
34
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
stub
stub
stub
stub
stub
35
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
stub
stub
stub
stub
36
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
stub
stub
stub
stub
37
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
stub
stub
stub
38
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
stub
stub
39
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
stub
40
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
41
Top-Down Strategy (contd)

• Potential Advantages
• Allows early verification of high-level behavior.
• One driver (at most) is required.
• Modules can be added one at a time with each step
  if desired.
• Supports both breadth first and depth
  first approaches.

42
Top-Down Strategy (contd)

• Potential Advantages
• Allows early verification of high-level behavior.
• One driver (at most) is required.
• Modules can be added one at a time with each step
  if desired.
• Supports both breadth first and depth
  first approaches.

43
Top-Down Strategy (contd)

• Potential Advantages
• Allows early verification of high-level behavior.
• One driver (at most) is required.
• Modules can be added one at a time with each step
  if desired.
• Supports both breadth first and depth
  first approaches.

44
Top-Down Strategy (contd)

• Potential Advantages
• Allows early verification of high-level behavior.
• One driver (at most) is required.
• Modules can be added one at a time with each step
  if desired.
• Supports both breadth first and depth
  first approaches.

45
Top-Down Strategy (contd)

• Potential Advantages
• Allows early verification of high-level behavior.
• One driver (at most) is required.
• Modules can be added one at a time with each step
  if desired.
• Supports both breadth first and depth
  first approaches.

46
Top-Down Strategy (contd)

• Potential Disadvantages
• Delays verification of low-level behavior.
• Stubs are required for missing elements.
• Test case inputs may be difficult to formulate.
• Test case outputs may be difficult to interpret.
  (Oracles may be needed to interpret/inspect test
  results.)

47
Top-Down Strategy (contd)

• Potential Disadvantages
• Delays verification of low-level behavior.
• Stubs are required for missing elements.
• Test case inputs may be difficult to formulate.
• Test case outputs may be difficult to interpret.
  (Oracles may be needed to interpret/inspect test
  results.)

48
Top-Down Strategy (contd)

• Potential Disadvantages
• Delays verification of low-level behavior.
• Stubs are required for missing elements.
• Test case inputs may be difficult to formulate.
• Test case outputs may be difficult to interpret.
  (Oracles may be needed to interpret/inspect test
  results.)

49
Top-Down Strategy (contd)

• Potential Disadvantages
• Delays verification of low-level behavior.
• Stubs are required for missing elements.
• Test case inputs may be difficult to formulate.
• Test case outputs may be difficult to interpret.
  (Oracles may be needed to interpret/inspect test
  results.)

50
Top-Down Strategy (contd)

• Potential Disadvantages
• Delays verification of low-level behavior.
• Stubs are required for missing elements.
• Test case inputs may be difficult to formulate.
• Test case outputs may be difficult to interpret.
  (Oracles may be needed to interpret/inspect test
  results.)

51
Bottom-Up Strategy

• Start at the bottom of the hierarchy with two or
  more sibling leaf modules, or an only-child leaf
  module with its parent.
• Test this group using a driver. (No stubs are
  required.)
• Add one or more additional siblings, replacing
  drivers with modules only when all modules they
  call have been integrated.
• (contd)

52
Bottom-Up Strategy

• Start at the bottom of the hierarchy with two or
  more sibling leaf modules, or an only-child leaf
  module with its parent.
• Test this group using a driver. (No stubs are
  required.)
• Add one or more additional siblings, replacing
  drivers with modules only when all modules they
  call have been integrated.
• (contd)

53
Bottom-Up Strategy

• Start at the bottom of the hierarchy with two or
  more sibling leaf modules, or an only-child leaf
  module with its parent.
• Test this group using a driver. (No stubs are
  required.)
• Add one or more additional siblings, replacing
  drivers with modules only when all modules they
  call have been integrated.
• (contd)

54
Bottom-Up Strategy (contd)

• Continue the process until all elements of the
  subtree have been integrated and tested.
• Repeat the steps above for the remaining subtrees
  in the hierarchy (or handle in parallel).
• Incrementally combine the sub-trees and then add
  the root module.

55
Bottom-Up Strategy (contd)

• Continue the process until all elements of the
  subtree have been integrated and tested.
• Repeat the steps above for the remaining subtrees
  in the hierarchy (or handle in parallel).
• Incrementally combine the sub-trees and then add
  the root module.

56
Bottom-Up Strategy (contd)

• Continue the process until all elements of the
  subtree have been integrated and tested.
• Repeat the steps above for the remaining subtrees
  in the hierarchy (or handle in parallel).
• Incrementally combine the sub-trees and then add
  the root module.

57
Bottom-Up Strategy (contd)
A
B
C
D
C
D
E
F
G
H
I
G
H
I
J
K
L
58
Bottom-Up Strategy (contd)
driver
F
J
59
Bottom-Up Strategy (contd)
driver
F
E
J
60
Bottom-Up Strategy (contd)
driver
B
F
E
J
61
Bottom-Up Strategy (contd)
driver
B
F
E
driver
J
K
L
62
Bottom-Up Strategy (contd)
driver
B
driver
F
E
H
H
J
K
L
63
Bottom-Up Strategy (contd)
driver
B
driver
F
E
H
H
I
I
J
K
L
64
Bottom-Up Strategy (contd)
driver
driver
B
D
D
F
E
H
H
I
I
J
K
L
65
Bottom-Up Strategy (contd)
driver
driver
B
D
D
driver
F
E
H
H
I
I
G
G
J
K
L
66
Bottom-Up Strategy (contd)
driver
driver
driver
B
D
D
C
C
F
E
H
H
I
I
G
G
J
K
L
67
Bottom-Up Strategy (contd)
driver
driver
D
D
C
C
B
H
H
I
I
G
G
E
F
K
L
J
68
Bottom-Up Strategy (contd)
driver
driver
B
C
C
D
D
F
E
G
G
H
H
I
I
J
K
K
L
69
Bottom-Up Strategy (contd)
driver
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
70
Bottom-Up Strategy (contd)
driver
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
71
Bottom-Up Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
72
Bottom-Up Strategy (contd)

• Potential Advantages
• Allows early verification of low-level behavior.
• No stubs are required.
• Easier to formulate input data for some subtrees.
  
• Easier to interpret output data for others.

73
Bottom-Up Strategy (contd)

• Potential Advantages
• Allows early verification of low-level behavior.
• No stubs are required.
• Easier to formulate input data for some subtrees.
  
• Easier to interpret output data for others.

74
Bottom-Up Strategy (contd)

• Potential Advantages
• Allows early verification of low-level behavior.
• No stubs are required.
• Easier to formulate input data for some subtrees.
  
• Easier to interpret output data for others.

75
Bottom-Up Strategy (contd)

• Potential Advantages
• Allows early verification of low-level behavior.
• No stubs are required.
• Easier to formulate input data for some subtrees.
  
• Easier to interpret output data for others.

76
Bottom-Up Strategy (contd)

• Potential Advantages
• Allows early verification of low-level behavior.
• No stubs are required.
• Easier to formulate input data for some subtrees.
  
• Easier to interpret output data for others.

77
Bottom-Up Strategy (contd)

• Potential Disadvantages
• Delays verification of high-level behavior.
• Drivers are required for missing elements.
• As subtrees are combined, a large number of
  elements may be integrated at one time.

78
Bottom-Up Strategy (contd)

• Potential Disadvantages
• Delays verification of high-level behavior.
• Drivers are required for missing elements.
• As subtrees are combined, a large number of
  elements may be integrated at one time.

79
Bottom-Up Strategy (contd)

• Potential Disadvantages
• Delays verification of high-level behavior.
• Drivers are required for missing elements.
• As subtrees are combined, a large number of
  elements may be integrated at one time.

80
Bottom-Up Strategy (contd)

• Potential Disadvantages
• Delays verification of high-level behavior.
• Drivers are required for missing elements.
• As subtrees are combined, a large number of
  elements may be integrated at one time.

81
Hybrid Incremental Integration Approaches

• Risk Driven
• Start by integrating the most critical or
  complex modules together with modules they call
  or are called by.
• Schedule Driven
• To the extent possible, integrate modules as
  they become available.
• (contd)

82
Hybrid Incremental Integration Approaches

• Risk Driven
• Start by integrating the most critical or
  complex modules together with modules they call
  or are called by.
• Schedule Driven
• To the extent possible, integrate modules as
  they become available.
• (contd)

83
Hybrid Incremental Integration Approaches

• Risk Driven
• Start by integrating the most critical or
  complex modules together with modules they call
  or are called by.
• Schedule Driven
• To the extent possible, integrate modules as
  they become available.
• (contd)

84
Hybrid Incremental Integration Approaches (contd)

• Function or Thread Driven
• Integrate the modules associated with a key
  function (thread) continue the process by
  selecting another function, etc.

85
Hybrid Incremental Integration Approaches (contd)

• Function or Thread Driven
• Integrate the modules associated with a key
  function (thread) continue the process by
  selecting another function, etc.

86
How about Object-Oriented Systems?

• Just as a calling hierarchy allows design of an
  integration strategy for imperative software,
  use/include relations serve this purpose for
  object-oriented software.
• Since there may be no single root class,
  testing usually proceeds cluster by cluster in a
  bottom-up fashion, starting with leaf classes
  that depend on no others.
• We will come back to this in Lecture 12.

87
How about Object-Oriented Systems?

• Just as a calling hierarchy allows design of an
  integration strategy for imperative software,
  use/include relations serve this purpose for
  object-oriented software.
• Since there may be no single root class,
  testing usually proceeds cluster by cluster in a
  bottom-up fashion, starting with leaf classes
  that depend on no others.
• We will come back to this in Lecture 12.

88
How about Object-Oriented Systems?

• Just as a calling hierarchy allows design of an
  integration strategy for imperative software,
  use/include relations serve this purpose for
  object-oriented software.
• Since there may be no single root class,
  testing usually proceeds cluster by cluster in a
  bottom-up fashion, starting with leaf classes
  that depend on no others.
• We will come back to this in Lecture 12.

89
Higher-Level Testing

• Higher-level tests focus on the core
  functionality specified for higher level
  elements, and on certain emergent properties that
  become more observable as testing progresses
  toward the system level.
• The black-box testing strategies already
  considered (e.g., partition and combinatorial
  approaches) apply to functional testing at any
  level.

90
Higher-Level Testing

• Higher-level tests focus on the core
  functionality specified for higher level
  elements, and on certain emergent properties that
  become more observable as testing progresses
  toward the system level.
• The black-box testing strategies already
  considered (e.g., partition and combinatorial
  approaches) apply to functional testing at any
  level.

91
Higher-Level Testing (contd)

• Higher-level testing typically includes
• A brief overview of each follows.

• Usability test
• Installability test
• Serviceability test
• Performance test
• Stress test
• Security test

• Software compatibility test
• Device and configuration test
• Recovery test
• Reliability test

92
Higher-Level Testing (contd)

• Higher-level testing typically includes
• A brief overview of each follows.

• Usability test
• Installability test
• Serviceability test
• Performance test
• Stress test
• Security test

• Software compatibility test
• Device and configuration test
• Recovery test
• Reliability test

93
Usability Test

• Focus is on factors which influence the ease with
  which potential end-users are able to utilize the
  system to accomplish their goals.
• Specialized and sophisticated HCI experts
  conduct experiments in simulated work
  environments.
• Protocol analysis is utilized to identify
  usability bottlenecks.
• Application-specific metrics related to
  under-standability, learnability, and operability
  may be employed.

94
Usability Test

• Focus is on factors which influence the ease with
  which potential end-users are able to utilize the
  system to accomplish their goals.
• Specialized and sophisticated HCI experts
  conduct experiments in simulated work
  environments.
• Protocol analysis is utilized to identify
  usability bottlenecks.
• Application-specific metrics related to
  under-standability, learnability, and operability
  may be employed.

95
Usability Test

• Focus is on factors which influence the ease with
  which potential end-users are able to utilize the
  system to accomplish their goals.
• Specialized and sophisticated HCI experts
  conduct experiments in simulated work
  environments.
• Protocol analysis is utilized to identify
  usability bottlenecks.
• Application-specific metrics related to
  under-standability, learnability, and operability
  may be employed.

96
Usability Test

• Focus is on factors which influence the ease with
  which potential end-users are able to utilize the
  system to accomplish their goals.
• Specialized and sophisticated HCI experts
  conduct experiments in simulated work
  environments.
• Protocol analysis is utilized to identify
  usability bottlenecks.
• Application-specific metrics related to
  under-standability, learnability, and operability
  may be employed.

97
Installability Test

• Focus is functional and non-functional
  requirements related to the installation of the
  product/system.
• Coverage includes
• Media correctness and fidelity,
• Relevant documentation (including examples), and
• Installation processes and supporting system
  functions.
• (contd)

98
Installability Test

• Focus is functional and non-functional
  requirements related to the installation of the
  product/system.
• Coverage includes
• Media correctness and fidelity,
• Relevant documentation (including examples), and
• Installation processes and supporting system
  functions.
• (contd)

99
Installability Test

• Focus is functional and non-functional
  requirements related to the installation of the
  product/system.
• Coverage includes
• Media correctness and fidelity,
• Relevant documentation (including examples), and
• Installation processes and supporting system
  functions.
• (contd)

100
Installability Test

• Focus is functional and non-functional
  requirements related to the installation of the
  product/system.
• Coverage includes
• Media correctness and fidelity,
• Relevant documentation (including examples), and
• Installation processes and supporting system
  functions.
• (contd)

101
Installability Test

• Focus is functional and non-functional
  requirements related to the installation of the
  product/system.
• Coverage includes
• Media correctness and fidelity,
• Relevant documentation (including examples), and
• Installation processes and supporting system
  functions.
• (contd)

102
Installability Test (contd)

• Functions, procedures, documentation, etc.,
  associated with product/system decommissioning
  must also be tested.

103
Serviceability Test

• Focus is requirements related to upgrading or
  fixing problems after installation.
• Coverage includes
• Change procedures (adaptive, perfective, and
  corrective service scenarios),
• Supporting documentation, and
• System diagnostic tools.

104
Serviceability Test

• Focus is requirements related to upgrading or
  fixing problems after installation.
• Coverage includes
• Change procedures (adaptive, perfective, and
  corrective service scenarios),
• Supporting documentation, and
• System diagnostic tools.

105
Serviceability Test

• Focus is requirements related to upgrading or
  fixing problems after installation.
• Coverage includes
• Change procedures (adaptive, perfective, and
  corrective service scenarios),
• Supporting documentation, and
• System diagnostic tools.

106
Serviceability Test

• Focus is requirements related to upgrading or
  fixing problems after installation.
• Coverage includes
• Change procedures (adaptive, perfective, and
  corrective service scenarios),
• Supporting documentation, and
• System diagnostic tools.

107
Serviceability Test

• Focus is requirements related to upgrading or
  fixing problems after installation.
• Coverage includes
• Change procedures (adaptive, perfective, and
  corrective service scenarios),
• Supporting documentation, and
• System diagnostic tools.

108
Performance Test

• Focus is requirements related to throughput,
  response time, memory utilization, input/ output
  rates, etc.
• Also very specialized in some organizations
  sophisticated test-beds and instrumentation are
  the norm.
• Statistical testing based on an operational
  profile is often employed.
• Requirements must be stated in quantifiable terms.

109
Performance Test

• Focus is requirements related to throughput,
  response time, memory utilization, input/ output
  rates, etc.
• Also very specialized in some organizations
  sophisticated test-beds and instrumentation are
  the norm.
• Statistical testing based on an operational
  profile is often employed.
• Requirements must be stated in quantifiable terms.

110
Performance Test

• Focus is requirements related to throughput,
  response time, memory utilization, input/ output
  rates, etc.
• Also very specialized in some organizations
  sophisticated test-beds and instrumentation are
  the norm.
• Statistical testing based on an operational
  profile is often employed.
• Requirements must be stated in quantifiable terms.

111
Performance Test

• Focus is requirements related to throughput,
  response time, memory utilization, input/ output
  rates, etc.
• Also very specialized in some organizations
  sophisticated test-beds and instrumentation are
  the norm.
• Statistical testing based on an operational
  profile is often employed.
• Requirements must be stated in quantifiable terms.

112
Stress Test

• Focus is system behavior at or near overload
  conditions (i.e., pushing the system to
  failure).
• Often undertaken with performance testing.
• In general, products are required to exhibit
  graceful failures and non-abrupt perfor-mance
  degradation.

113
Stress Test

• Focus is system behavior at or near overload
  conditions (i.e., pushing the system to
  failure).
• Often undertaken with performance testing.
• In general, products are required to exhibit
  graceful failures and non-abrupt perfor-mance
  degradation.

114
Stress Test

• Focus is system behavior at or near overload
  conditions (i.e., pushing the system to
  failure).
• Often undertaken with performance testing.
• In general, products are required to exhibit
  graceful failures and non-abrupt perfor-mance
  degradation.

115
Security Test

• Focus is vulnerability of resources to
  unauthorized access or manipulation.
• Issues include
• Physical security of computer resources, media,
  etc.,
• Login and password procedures/policies,
• Levels of authorization for data or procedural
  access, etc.

116
Security Test

• Focus is vulnerability of resources to
  unauthorized access or manipulation.
• Issues include
• Physical security of computer resources, media,
  etc.,
• Login and password procedures/policies,
• Levels of authorization for data or procedural
  access, etc.

117
Security Test

• Focus is vulnerability of resources to
  unauthorized access or manipulation.
• Issues include
• Physical security of computer resources, media,
  etc.,
• Login and password procedures/policies,
• Levels of authorization for data or procedural
  access, etc.

118
Security Test

• Focus is vulnerability of resources to
  unauthorized access or manipulation.
• Issues include
• Physical security of computer resources, media,
  etc.,
• Login and password procedures/policies,
• Levels of authorization for data or procedural
  access, etc.

119
Security Test

• Focus is vulnerability of resources to
  unauthorized access or manipulation.
• Issues include
• Physical security of computer resources, media,
  etc.,
• Login and password procedures/policies,
• Levels of authorization for data or procedural
  access, etc.

120
Software Compatibility Test

• Focus is compatibility with other
  products/systems in the environment and/or with
  interoperability standards.
• May also concern source- or object-code
  compatibility with different operating
  environment versions.
• AKA compatibility/conversion testing when
  conversion procedures or processes are involved.

121
Software Compatibility Test

• Focus is compatibility with other
  products/systems in the environment and/or with
  interoperability standards.
• May also concern source- or object-code
  compatibility with different operating
  environment versions.
• AKA compatibility/conversion testing when
  conversion procedures or processes are involved.

122
Software Compatibility Test

• Focus is compatibility with other
  products/systems in the environment and/or with
  interoperability standards.
• May also concern source- or object-code
  compatibility with different operating
  environment versions.
• AKA compatibility/conversion testing when
  conversion procedures or processes are involved.

123
Device and Configuration Test

• Focus is configurability for, and/or
  compatibility with, all supported hardware
  configurations.
• Particularly taxing for client/server-based
  applications
• Tests are usually limited to combinations of
  representative devices for each supported
  protocol.

124
Device and Configuration Test

• Focus is configurability for, and/or
  compatibility with, all supported hardware
  configurations.
• Particularly taxing for client/server-based
  applications
• Tests are usually limited to combinations of
  representative devices for each supported
  protocol.

125
Device and Configuration Test

• Focus is configurability for, and/or
  compatibility with, all supported hardware
  configurations.
• Particularly taxing for client/server-based
  applications
• Tests are usually limited to combinations of
  representative devices for each supported
  protocol.

126
Recovery Test

• Focus is ability to recover from exceptional
  conditions associated with hardware, software, or
  people.
• This can involve
• detecting exceptional conditions,
• switch-overs to standby systems,
• recovery of data,
• maintaining audit trails, etc.
• May also involve external procedures such as
  storing backup tapes, etc.

127
Recovery Test

• Focus is ability to recover from exceptional
  conditions associated with hardware, software, or
  people.
• This can involve
• detecting exceptional conditions,
• switch-overs to standby systems,
• recovery of data,
• maintaining audit trails, etc.
• May also involve external procedures such as
  storing backup tapes, etc.

128
Recovery Test

• Focus is ability to recover from exceptional
  conditions associated with hardware, software, or
  people.
• This can involve
• detecting exceptional conditions,
• switch-overs to standby systems,
• recovery of data,
• maintaining audit trails, etc.
• May also involve external procedures such as
  storing backup tapes, etc.

129
Recovery Test

• Focus is ability to recover from exceptional
  conditions associated with hardware, software, or
  people.
• This can involve
• detecting exceptional conditions,
• switch-overs to standby systems,
• recovery of data,
• maintaining audit trails, etc.
• May also involve external procedures such as
  storing backup tapes, etc.

130
Recovery Test

• Focus is ability to recover from exceptional
  conditions associated with hardware, software, or
  people.
• This can involve
• detecting exceptional conditions,
• switch-overs to standby systems,
• recovery of data,
• maintaining audit trails, etc.
• May also involve external procedures such as
  storing backup tapes, etc.

131
Recovery Test

• Focus is ability to recover from exceptional
  conditions associated with hardware, software, or
  people.
• This can involve
• detecting exceptional conditions,
• switch-overs to standby systems,
• recovery of data,
• maintaining audit trails, etc.
• May also involve external procedures such as
  storing backup tapes, etc.

132
Recovery Test

• Focus is ability to recover from exceptional
  conditions associated with hardware, software, or
  people.
• This can involve
• detecting exceptional conditions,
• switch-overs to standby systems,
• recovery of data,
• maintaining audit trails, etc.
• May also involve external procedures such as
  storing backup tapes, etc.

133
Reliability Test

• Requirements may be expressed as
• the probability of no failure in a specified time
  interval, or as
• the expected mean time to failure.
• Appropriate interpretations for failure and time
  are critical.
• Utilizes Statistical testing based on an
  operational profile.

134
Reliability Test

• Requirements may be expressed as
• the probability of no failure in a specified time
  interval, or as
• the expected mean time to failure.
• Appropriate interpretations for failure and time
  are critical.
• Utilizes Statistical testing based on an
  operational profile.

135
Reliability Test

• Requirements may be expressed as
• the probability of no failure in a specified time
  interval, or as
• the expected mean time to failure.
• Appropriate interpretations for failure and time
  are critical.
• Utilizes Statistical testing based on an
  operational profile.

136
Integration and Higher Level Testing
Software Testing and Verification Lecture 11

• Prepared by
• Stephen M. Thebaut, Ph.D.
• University of Florida