Testing

1
Testing
2
Agenda

• Defining Software Testing
• Developing a Rapid Testing Strategy
• Test Planning
• Test Design, Implementation, Debugging
• System Test
• Acceptance Test

3
Defining Software Testing

• Process of analyzing or operating software for
  the purpose of finding bugs
• Involves planned, orderly activities
• Static testing code inspections, walkthroughs,
  and desk checks
• Dynamic testing testing that involves operating
  the software itself
• A bug is a flaw that causes a discrepancy between
  expected and actual results

4
Basic Functions of Software Testing

• Verification
• The products of a phase are consistent with the
  requirements of that phase and previous phases
• Validation
• The final product satisfies the system
  requirements
• Each function can be traced back to a user
  requirement

5
Good Quality if

• It has few failures when implemented
• Users dont have problems
• It is reliable
• Crashes
• Surprising and unexpected results
• Satisfies a majority of the users

ex
6
Essential Componentsof Rapid Testing
ex
7
Developing a Rapid Testing Strategy

• In each phase of the SDLC ask yourself
• Can we prevent defects
• Can we help manage risk to the schedule
• What can be taken to speed up planning, test case
  development, or test execution

8
Requirements Analysis and Testing

• Standish Group survey of 350 companies
• 9 of 8,000 projects were on time and within
  budget
• 31 were canceled before completion
• Half were attributed to some aspect of the
  requirements process
• Incomplete requirements (13.1)
• Lack of user involvement (12.4)
• Unrealistic expectations (9.9)
• Changing requirements and specifications (8.7)
• System no longer needed (7.5)

9
Cost to fix a defect

• Requirements - 1
• Design - 5
• Coding - 10
• Unit Test - 20
• Maintenance - 200

10
Test Team Involvement

• Should be included from the start
• End product from the requirements phase is a
  Requirements Traceability Matrix
• Maps each requirement to tests, design
  components, and code.
• Test using inspections, walkthroughs, and peer
  reviews

ex
11
Sample Requirements Traceability Matrix

• Requirement Definition and Specification IDs to
  map the requirements between definition and
  specification documents
• Design and Code Component IDs for tracing the
  design and code components back to the
  requirements
• The following identifiers to map all testing back
  to the requirements Unit Test Case, Integration
  Test Case, System Test, Acceptance Test Case

12
Requirements Testing Comparison
13
Basic Requirements Criteria

• Traceable
• Use unique identifiers to track progress
• Feasible
• Economically feasible, maintainable, reliable,
  usable
• Testable
• measurable

• Complete
• Components are there
• Unambiguous
• Only one interpretation
• Consistent
• No conflict with standards or policy

14
Testing in an Evolutionary Prototyping Life Cycle
15
Test Planning

• Input is the requirements document
• Test Strategy
• Defining the test system
• Estimating the test effort
• Preparing and reviewing the test plan documents
• Output is a test plan, documents reviewed by the
  test team, development team management

16
Test Strategy

• Scope of the testing to include
  installing/upgrading software/hardware
• Types of testing methodologies static and
  dynamic and use of automated tools
• Procedures to report and fix bugs
• Entry/exit criteria for each phase of testing
  that require participation by test team

17
Reasonable Test Coverage(Source Culbertson)

• Highest priority requirements first
• New functionality and modified code
• Areas most likely to have problems such as those
  modules with problems in unit and integration
  testing
• Focus on functions and configurations that will
  most often be used by the end-users

18
Testing Approach

• Examine each phase of the life cycle to determine
  the tests that can be used in that phase
  Requirements, System Design, Program Design,
  Program Coding, Unit Test, Integration Test,
  System Test, Acceptance Test, and Regression Test

ex
19
Testing Criteria and Quality Checkpoints

• Entry criteria what is needed to begin
• Exit criteria what is needed to stop
• Suspension criteria testing is stopped by bugs
• Test pass/fail criteria
• Other criteria compliance with standards

20
Automation Strategy

• False Expectations
• Everything can be automated
• One tool can do it all
• Schedule will be reduced
• Easy to use
• Allows 100 test coverage

21
Estimating the Test Effort

• Identify tasks
• Effort required for each task
• Time required for each task
• Build a detailed schedule
• Assess the risk with mitigation plans
• Sound familiar?

22
Test Plan Documents

• IEEE Standard 829, the IEEE Standard for Software
  Test Documentation
• Has 16 components
• Test Plan Reviews by team, developers, and
  management

23
Automated Testing Benefits

• Improved system reliability performance and
  stress tests as well as repetitive
• Reduction of test schedule - less time to run
  tests, analyze results, determine test status and
  report test results

24
Test Design

• Define test objectives - what will be
  accomplished, not how
• Define input specs input data files, db
  records, or other inputs to prepare test
  environment to run tests
• Define test configurations
• Create test design document which ties back to
  the requirement traceability matrix

25
Developing Test Cases

• Smallest unit of testing and can have at least
  one expected result
• Constructively breaking the product which means a
  high probability of finding a bug
• Should be repeatable
• Should have clear pass/fail criteria
• Should not be redundant

26
System Testing

• Apply entry criteria all previous testing done?
• Run tests follow test plan and report bugs
• Report test results bug review report
• Apply exit criteria all tests done/
• Apply pass/fail criteria
• Readiness review

27
Tracking Bugs

• Must first define whether you will fix never,
  now, or later (bug state)
• Maintain a log of all bugs and bug state
• Bug report should include
• Description of the defect
• Difference in system behavior
• Step-by-step procedure to reproduce the problem
• Bug Reviews

28
In summary

• Each programmer performs unit testing
• Then an Integration test when combining all those
  programs unit tested
• System test on the entire system usually by
  testing team (not programmers)
• Acceptance test usually performed by the end user