CIS 4932 Software Testing Testing Concepts

1
CIS 4932Software TestingTesting Concepts
2
Purpose
This module presents the basic concepts of
software testing. We present several aspects of
software testing, and a give a generic lifecycle.
" arbitrarily selected test set ... results in
inefficient testing, leaving some functions
untested while performing redundant testing of
others." Darlene Mackay, Quality Consultants
Unlimited
3
Agenda

• What is Testing

• Testing Lifecycle

• A Framework for Testing

• A Complete Example

4
WHAT IS TESTING?

• Executing software to determine whether it
  meets its specification
• Executing a software element to discover defects
  or anomalies
• Executing software to evaluate its behavior in
  different environments
• Inspecting software element code to discover
  defects or anomalies.

5
TESTING IS HARD

• Exhaustive testing is impossible
• Software is complex
• Specifying requirements is hard
• Testing is a fallible, human process
• Consequences of failure not always understood in
  advance

6
WORKING SMART

• Test small pieces
• Use proven techniques
• Use tools to advantage
• Start early
• Cost of errors grows exponentially with time they
  go undetected

7
Agenda

• What is Testing

• Testing Lifecycle

• A Framework for Testing

• A Complete Example

8
A TESTING LIFECYCLE
Analysis
Design
9
TESTING vs DEVELOPMENT

• Work broken into stages
• Require similar skills
• Require management oversight
• Produce artifacts used to assess quality of the
  work
• Test artifacts are overhead, not part of
  delivered system.

10
TEST ARTIFACTS

• Strategy/Plan (must know techniques)
• Test cases central artifacts
• Test machinery to apply test cases
• Data showing results
• Reports that interpret test results
• List of problems/defects to be fixed.

11
TESTING IS HIERARCHICAL

• Unit testing smallest components
• Aggregate testing assemblies of units
• Integration testing combinations of assemblies
• System testing system built from assemblies

12
TESTING OBJECTIVES

• To demonstrate conformance
• To find defects
• To measure behavior in specific environments
• Load / stress testing
• Performance testing
• To measure trustworthiness/reliability

13
Agenda

• What is Testing

• Testing Lifecycle

• A Framework for Testing

• A Complete Example

14
REQUIRED ELEMENTS OF A TESTING METHODOLOGY?

• Specification for the software
• Process for designing test cases
• Repeatable process for designing, running and
  evaluating tests
• Accountable artifacts of testing activities
• Economical use of human, time and computing
  resources

15
Agenda

• What is Testing

• Testing Lifecycle

• A Framework for Testing

• A Complete Example

16
BLACK-BOX TESTING
Stimuli
Response(s)

• Testing based on the specification rather than
  the implementation.
• Specification defines the expected response(s) to
  stimuli

17
BLACK-BOX TECHNIQUES

• Functional testing -- tests the behavior of the
  software.
• Boundary testing -- tests behavior at the
  lower/upper bounds of input values
• Random testing -- tests using randomly generated
  stimuli
• Intuitive (ad hoc) testing -- error guessing

18
FUNCTIONAL TEST DESIGN METHODOLOGY

• Specification
• Identify behaviors
• Develop test cases
• Write test script

19
EXAMPLE A(1) Specification

• Compute pay for an hourly employee, given the
  number of hours worked and the hourly pay rate.
  Compute overtime at 1.5 times hourly rate for
  hours in excess of 40.

20
EXAMPLE A(2) Identify Behaviors

• Case 1 No overtime (Hours lt 40)
• Expect Pay Hours Rate
• Case 2 Overtime (Hours gt 40)
• Expect Pay 40Rate1.5Rate(Hours - 40)

21
EXAMPLE A(3) Create Test Cases

• Case 1 No overtime (Hours lt 40)
• Use Rate 10, Hours 30
• Expect Pay Hours Rate 300
• Case 2 Overtime (Hours gt 40)
• Use Rate 10, Hours 50
• Expect Pay 40Rate1.5Rate(Hours - 40)
• 550

22
EXAMPLE A(4) Write Test Script
23
A MORE COMPLEX EXAMPLE (B)

• Increased number of behaviors
• Use of decision table to document behaviors
• Test case generation from decision table

24
EXAMPLE B(1) Specification

• Compute pay for employee, given the number of
  hours worked and the hourly pay rate. For hourly
  employees (rate lt 30), compute overtime at 1.5
  times hourly rate for hours in excess of 40.
  Salaried employees (rate gt 30) are paid for
  exactly 40 hours.

25
EXAMPLE B(2) Identify Behaviors

• Case 1 Hourly AND No overtime
• (Rate lt 30) (Hours lt 40)
• Expect Pay Hours Rate
• Case 2 Hourly AND Overtime
• (Rate lt 30) (Hours gt 40)
• Expect Pay 40Rate1.5Rate(Hours - 40)
• Case 3 Salaried (Rate gt 30)
• Expect Pay 40 Rate

26
DECISION TABLE
Columns define Behaviors
27
EXAMPLE B(3) Create Test Cases

• One test case per column of decision table
• Case 1 Hourly, No Overtime
• Case 2 Hourly, Overtime
• Case 3 Salaried, No Extra Hours
• Case 4 Salaried, Extra Hours
• Order the test cases by column

28
EXAMPLE B(4) Write Test Script
29
RULES -- DECISION TABLES
Use 'Y', 'N', '-' or space
30
NEXT TIME

• Test Case Generation Strategies
• Documenting the Test
• Specification Techniques