Testing Overview

1
Testing Overview

• Overview of Software Testing
• Test Plan Outline

2
Agenda

• Deliverables
• Friday, May 31 _at_ 5 PM - Status Report 3
• Friday, May 31 _at_ noon
• Requirements Document
• Project Plan
• Lecture Testing Overview

3
Software Quality Factors
Portability Reusability Interoperabilty
Maintainability Flexibility Testability
Correctness, Reliablity, Efficiency, Integrity,
Usability
4
Two Key Concepts

• Validation Are we building the right thing?
• Conformance to customer requirements and quality
  attributes.
• Verification Are we building the thing right?
• Process conformance, quality activities.

5
Maturity of Testing Philosophy
PHASE O Testing Debugging
PHASE 1 Testing shows the software works
PHASE 2 Testing shows the software doesnt work
PHASE 3 Testing doesnt prove anything, it
reduces risk of unacceptable
software delivery.
PHASE 4 Testing is not an act, a mental
discipline of quality.
6
Testing Ground Rules

• Importance

The development of software systems involves a
series of production activities where
opportunities for injection of human
fallibilities are enormousBecause of human
inability to perform and communicate with
perfection, software development is accompanied
by a quality assurance activity. Deuts
ch

• Objectives
• Testing is the process of executing a program
  with the intent of finding errors
• A good test case is one that has a high
  probability of finding an error
• A successful test is one that uncovers an error

7
Ground Rules (continued)

• Benefits
• Uncover errors in software
• Demonstrates that software functions appear to be
  working according to specification
• Demonstrate performance requirements have been
  met
• Provide indication of software reliability and
  quality
• However, testing cannot show the absence of
  defects
• Minimal set of test cases needs to be developed
  because exhaustive testing not possible

8
Testing Paradox
Every method you use to prevent or find bugs
leaves a residue of subtler bugs against which
those methods are ineffective.
9
Testing Phases

• Unit
• Integration
• Component
• System
• Usability
• Regression
• Validation
• Alpha/Beta

10
Unit Testing Guidelines

• Primary Goals
• Conformance to specifications
• Determine extent to which processing logic
  satisfies the functions assigned to the module
• Logical and operational requirements taken from
  specifications
• Processing accuracy
• Input
• Process
• Output

11
Integration Testing Guidelines

• Primary goals
• Compatibility
• Calling of modules in an operational environment
• Verify that all modules are called correctly, do
  not cause abnormal ends
• Inter-module processing accuracy
• Check that data transfers between modules operate
  as intended within constraints

12
Testing Strategies

• Dynamic Analysis
• Black Box
• White Box
• Static Analysis
• Code Reviews
• Walkthroughs
• Inspections

13
Black Box Testing

• Used to demonstrate functions are operational
• Proper input produces correct output
• Focuses on the functional requirements
• Goal
• Set of inputs that fully exercises all functional
  requirements

• Kinds of errors targeted
• Incorrect or missing functions
• Interface errors
• Errors in data (structures or databases)
• Behavior or performance errors
• Initialization and termination errors

14
White Box Testing

• Uses structure of program to derive test cases
• Set of test cases case guarantee
• All independent paths have been executed at least
  once
• Exercise all logical decisions, both T and F
• Execute all loops at their boundaries and within
  bounds
• Exercise internal data structures to ensure
  validity

15
Comparison of Test Strategies
Te
s
t
S
t
r
ate
g
y
M
ethod
G
oal
D
is
a
dva
n
tage
s
Wh
i
te-
B
ox
L
o
gi
c
P
ro
v
e
Fu
n
ct
i
on
a
l f
l
a
w
s,

dat
a
pr
o
ce
s
s
i
ng
s
e
n
s
iti
v
e
c
on
d
it
i
on
s
,

err
o
rs
a
cro
s
s

mo
d
ul
e
s
d
i
ffi
c
ul
t
to
t
est
B
la
c
k-
B
ox
D
at
a
P
ro
v
e r
e
su
l
ts
T
y
pe
2
err
o
rs
l
og
i
c
pr
o
bl
e
m
s
dif
f
ic
u
lt
t
o f
i
nd
To
p
-D
o
wn
In
c
re
m
en
t
al
E
x
e
rc
i
se

cri
t
ic
a
l
S
c
a
ffo
l
di
n
g t
a
ke
s
ti
m
e
c
o
de

to i
m
pr
o
ve
C
on
s
ta
n
t c
h
an
g
e
m
ay
re
l
ia
b
il
i
ty
i
n
trod
u
ce

ne
w
e
r
rors
B
otto
m
-U
p
A
l
l
or
P
erf
e
ct

part
s

Fu
n
ct
i
on
a
l f
l
a
w
s
f
oun
d
n
o
thi
n
g
If
p
art
s

w
or
k
,
l
a
te
c
au
s
e
d
ela
y
s
w
ho
l
e
s
ho
u
ld
E
rror
s
a
c
ro
s
s
m
od
u
le
s
w
or
k
d
i
ffi
c
ul
t
to
t
rac
e
an
d
fi
n
d
16
Test Plan Outline

• Introduction
• Purpose
• Program Description
• Test Environment
• Test Case Descriptions
• Coverage Criteria
• Details
• Summary