Test Planning

About This Presentation

Title:

Test Planning

Description:

Test Case - Test cases are how the testers validate that a software function or ... Programmers may fail to keep a change log, or to maintain backup copies or to ... – PowerPoint PPT presentation

Number of Views:58

Avg rating:3.0/5.0

Slides: 29

Provided by: mattbu4

more less

Transcript and Presenter's Notes

Title: Test Planning

1
Test Planning

Risk Concepts and Vocabulary
Risks Associated with Software Development
Risks Associated with Software Testing
Risk Analysis
Risk Management
Prerequisites to Test Planning
Create the Test Plan

2
Risk Concepts and Vocabulary

Test Case - Test cases are how the testers
validate that a software function or a structural
attribute of software meets the software
specifications
Test Data - Test data is information used to
build a test case
Test Script - Test scripts are an online entry of
test cases in which the sequence of entering
test cases and the structure of the online entry
system must be validated.
Risk - The potential loss to an organization.
Risk can be measured by performing a risk
analysis
Risk Analysis - The process of evaluating risks,
threats, controls, and vulnerabilities.
Threat - A threat is something capable of
exploiting vulnerability in the security of a
computer system or application. Threats include
both hazards and events that can trigger flaws.
Vulnerability - A flaw in the system of control
that will enable a threat to be exploited.
Control - Anything that tends to cause the
reduction of risk.
Damaging Event - The materialization of a risk to
an organizations assets.
To identify the risks in a computerized
environment a tester must
Identify these risks
Estimate the severity of the risk
Develop tests to substantiate the impact of the
risk on the application.

3
Risks Associated with Software Development

Improper use of technology It is a mismatch of
technology and its needs and can result in an
unnecessary expenditure of resources. Conditions
that lead to this are
Systems analysts and systems programmers
improperly skilled in the use of technology
Early user of new hardware technology
Early user of new software technology
Minimal planning for the installation of new
hardware and software technology
Repetition of errors In a manual system errors
are made individually. In a automated system if
a rule is erroneous the processing will always be
erroneous. Conditions that cause repetition of
errors
Inadequate checks on entry of master information
Insufficient program testing
Failure to monitor the results of processing

4
Risks Associated with Software Development

Cascading of errors Errors that trigger other
errors to occur. This often occurs between
applications. Conditions that lead to cascading
errors
Inadequately tested applications
Failure to communicate the type and date of
changes being implemented
Limited testing of program changes
Illogical processing The production of an
automated event that would be highly unlikely to
occur in a manual processing environment.
Conditions that lead to this are
Failure to check for unusually large amounts on
output documents
Fields that are either too small or too large
Failure to scan output documents

5
Risks Associated with Software Development

Inability to translate user needs into technical
requirements
A communication failure between users and the
project personel. Conditions that can lead to
this are
Users without technical IT skills
Technical people without sufficient understanding
of user requirements
Users inability to specify requirements in
sufficient detail
Multi-user systems with no user in charge of
the system
Inability to control technology Controls are
needed over the technological environment.
Conditions that can lead to this are
Selection of vendor-offered hardware and software
without consulting how to control them
Too many control tradeoffs for operational
efficiency
Inadequate restart and recovery procedures
Inadequate control over different versions of
programs

6
Risks Associated with Software Development

Incorrect entry of data - Conditions that can
cause incorrect entry of data
Human errors in keying data
Mechanical failure of hardware devices
Misinterpretation of characters or meaning of
manually recorded input
Misunderstanding of data entry procedures
Concentration of data Conditions that can cause
data concentration issues
Inadequate access controls enabling unauthorized
access to data
Erroneous data and their impact on multiple users
of the data
Impact of hardware and software failures that
make the data available to multiple users
Inability to react quickly The inability to
satisfy a users needs in a timely basis.
Conditions that can cause this are.
The structure of the computer files is
inconsistent with the information requested
The general-purpose extract programs are not
available to satisfy the desired request
Computer time is unavailable to satisfy the
request
The cost of processing exceeds the value of the
information requested

7
Risks Associated with Software Development

Inability to substantiate processing The
inability to reconstruct the processing of a
single transaction and the reconstruction of
control totals. Conditions that lead to this
are.
Evidence is not retained long enough
The cost of substantiating processing exceeds the
benefits derived from the process
The evidence from intermediate processing is not
retained
Concentration of responsibilities The
conditions that lead to this are.
Establishment of large databases
Client-server systems
Web-based systems

8
Risks Associated with Software Development

Erroneous or falsified input data Conditions
that can cause erroneous or falsified input data
are.
Unreasonable or inconsistent source data values
may not be detected
Keying errors may not be detected
Incomplete or poorly formatted transactions may
be accepted and treated as if they were complete
transactions
Records in one format may be interpreted
according to a different format
An employee may fraudulently add, delete, or
modify data (e.g., payment vouchers or claims) to
obtain benefits (e.g., checks or negotiable
coupons) for himself
Lack of document counts and other controls over
source data or input transactions may allow some
of the data or transactions to be lost without
detection or allow extra records to be added
Records about the personnel (e.g., a record of a
personnel action) may be modified during entry
Data that arrives at the last minute (or under
some other special or emergency condition) may
not be verified prior to processing
Transactions in which errors have been detected
may be corrected without verification of the full
record

9
Risks Associated with Software Development

Misuse by authorized end users Conditions which
cause this are.
An employee may convert information to an
unauthorized use for example, she or he may sell
privileged data about an individual to a
prospective employer, credit agency, insurance
company, or competitor or may use statistics for
stock market transactions before their public
release.
A user whose job requires access to individual
records in a database may manage to compile a
complete listing of the database and then make
unauthorized use of it (e.g., sell a listing of
employees home addresses as a mailing list).
Unauthorized altering of information may be
accomplished by an authorized end user (e.g.,
theft of services).
An authorized user may use the system for
personal benefit (e.g., theft of services).
A supervisor may manage to approve and enter a
fraudulent transaction.
A disgruntled or terminated employee may destroy
or modify records possibly in such a way that
backup records are also corrupted and useless.
An authorized user may accept a bribe to modify
or obtain information.

10
Risks Associated with Software Development

Uncontrolled system access Conditions which can
cause this are.
Data or programs may be stolen from the computer
room or other storage areas.
IT facilities may be destroyed or damaged by
either intruders or employees.
Individuals may not be adequately identified
before they are allowed to enter the IT area.
Unauthorized persons may not adequately protect
remote terminals from use.
An unauthorized user may gain access to the
system.
Passwords may be inadvertently revealed to
unauthorized individuals. A user may write
his/her password in some convenient place, or the
password may be obtained from discarded
printouts, or by observing the user as they type
it.
A user may leave a logged-in terminal unattended,
allowing an unauthorized person to use it.
A terminated employee may retain access to an IT
system because his name and password are not
immediately deleted from authorization tables and
control lists.
An unauthorized individual may gain access to the
system for his own purposes (e.g., theft of
computer services, or data, or programs,
modification of data, alteration of programs,
sabotage, and denial of services).
Repeated attempts by the same user or terminal to
gain unauthorized access to the system or to a
file may go undetected.

11
Risks Associated with Software Development

Ineffective security and privacy practices for
the application
Poorly defined criteria for authorized access may
result in employees not knowing what information
they, or others, are permitted to access.
The person responsible for security or privacy
may fail to restrict user access to only those
processes and data that are needed to accomplish
assigned tasks.
Large fund disbursements, unusual price changes,
and unanticipated inventory usage may not be
reviewed for correctness.
Repeated payments to the same party may go
unnoticed because there is no review.
The application staff may carelessly handle
sensitive data, by the mail service, or by other
personnel within the organization.
Post-processing reports analyzing system
operations may not be reviewed to detect security
or privacy violations.
Inadvertent modification or destruction of files
may occur when trainees are allowed to work on
live data.
Appropriate action may not be pursued when a
security variance is reported to the system
security officer or to the perpetrating
individuals supervisor in fact, procedures
covering such occurrences may not exist.

12
Risks Associated with Software Development

Procedural errors during operations This
contains two sections - Procedures and Controls
Storage Media Handling
Procedures and Controls
Files may be destroyed during database
reorganization or during release of disk space.
Operators may ignore operational procedures for
example, allowing programmers to operate computer
equipment.
Job control language parameters may be erroneous.
An installation manager may circumvent
operational controls to obtain information.
Careless or incorrect restarting after shutdown
may cause the state of a transaction update to be
unknown.
Hardware maintenance may be performed while
production data is online and the equipment
undergoing maintenance is not isolated.
An operator may perform unauthorized acts for
personal gain (e.g., make extra copies of
competitive bidding reports, print copies of
unemployment checks, and delete a record from
journal file).
Operations staff may sabotage the computer (e.g.,
drop pieces of metal into a terminal).
The wrong version of a program may be executed.
A program may be executed twice using the same
transactions.
An operator may bypass required controls.

13
Risks Associated with Software Development

Procedures and Controls (contd)
Supervision of operations personnel may not be
adequate during nighttime shifts.
Because of incorrectly learned procedures, an
operator may alter or erase master file
Storage and Media Handling
Inadvertently or intentionally mislabeled
storage media are erased. In a case where they
contain backup files, the erasure may not be
noticed until it is needed.
Backup files with missing or mislabeled
expiration dates may be erased.
Temporary files created during a job step, for
use in subsequent steps, may be erroneously
released or modified through inadequate
protection of the files or because of an abnormal
termination.
Storage media containing sensitive information
may not get adequate protection because the
operations staff is not advised of the nature of
the information content.
Output may be sent to the wrong individual or
terminal.
Improperly operating procedures in post
processing tasks may result in loss ofoutput.
Surplus output material (e.g., duplicates of
output data, used carbon paper) may not be
disposed of properly.

14
Risks Associated with Software Development

Program errors
Records may be deleted from sensitive files
without a guarantee that the deleted records can
be reconstructed.
Programmers may insert special provisions in
programs that manipulate data concerning
themselves (e.g., payroll programmer may alter
own payroll records).
Program changes may not be tested adequately
before being used in a production run.
Changes to a program may result in new errors
because of unanticipated interactions between
program modules.
Program acceptance tests may fail to detect
errors that occur for only unusual combinations
of input (e.g., a program that is supposed to
reject all except a specified range of values,
actually accepts an additional value.)
Programs, the content of which should be
safeguarded, may not be identified and protected.
Test data with associated output, or
documentation for certified programs may not be
retained for future use.
Documentation for vital programs may not be
safeguarded.
Programmers may fail to keep a change log, or to
maintain backup copies or to formalize
record-keeping activities.
An employee may steal programs he or she is
maintaining and use them for personal gain (e.g.,
for sale to a commercial organization, or to hold
another organization for extortion).

15
Risks Associated with Software Development

Program Errors (contd)
Poor program design may result in a critical data
value being initialized to zero.
An error may occur when the program is modified
to change a data value but only change it in one
place.
Production data may be disclosed or destroyed
when used during testing.
Errors may result when the programmer
misunderstands requests for changes to the
program.
Programs may contain routines not compatible with
their intended purpose, which can disable or
bypass security protection mechanisms. For
example, a programmer who anticipates being fired
inserts a code into a program that will cause
vital system files to be deleted as soon as his
or her name no longer appears in the payroll
file.
Inadequate documentation or labeling may result
in the wrong version of a program being modified.

16
Risks Associated with Software Development

Operating system flaws
User jobs may be permitted to read or write
outside assigned storage area.
Inconsistencies may be introduced into data
because of simultaneous processing of the same
file by two jobs.
An operating system design or implementation
error may allow a user to disable controls or to
access all system information.
An operating system may not protect a copy of
information as thoroughly as it protects the
original.
Unauthorized modifications to the operating
system may allow a data entry clerk to enter
programs and thus subvert the system.
An operating system crash may expose valuable
information, such as password lists or
authorization tables.
Maintenance personnel may bypass security
controls while performing maintenance work. At
such times the system is vulnerable to errors or
intentional acts of the maintenance personnel, or
anyone else who might be on the system and
discover the opening (e.g., micro coded sections
of the operating system may be tampered with or
sensitive information from online files may be
disclosed).
An operating system may fail to maintain an
unbroken audit trail.
When restarting after a system crash, the
operating system may fail to ascertain that all
terminal locations previously occupied are still
occupied by the same individuals.
A user may be able to get into a monitor or
supervisory mode.
The operating system may fail to erase all
scratch space assigned to a job after the normal
or abnormal termination of the job.
Files may be allowed to be read or written prior
to being opened by the operating system.

17
Risks Associated with Software Development

Communications system failure
Accidental Failures
Undetected communications errors may result in
incorrect or modified data.
Information may be accidentally misdirected to
the wrong terminal.
Communication nodes may leave unprotected
fragments of messages in memory during
unanticipated interruptions in processing.
Communication protocol may fail to positively
identify the transmitter or receiver of a
message.
Intentional Acts
Unauthorized individuals may monitor
communication lines.
Data or programs may be stolen.
Programs in the network that switch computers may
be modified to compromise security.
Data may be deliberately changed by an
individuals tapping the line (requires some
sophistication, but is applicable to financial
data).
An unauthorized user may take over a computer
communication port as an authorized user
disconnects from it. Many systems cannot detect
the change. This is particularly true in much of
the currently available communication protocols.
If encryption (i.e., use of codes) is used, keys
may be stolen.
A user may be spoofed (i.e., tricked) into
providing sensitive data.
False messages may be inserted into the system.
True messages may be deleted from the system.
Messages may be recorded and replayed into the
system.

18
Risks Associated with Software Testing

When conducting Risk analysis two components are
taken into consideration
The probability that the negative event will
occur.
The potential loss or impact associated with the
event.
It is the test managers responsibility to
determine how to apply the test methodology to
achieve the greatest level of confidence in the
application under development.
The test manager must determine the appropriate
amount of testing to perform based upon the risks
associated with the application.
The test manager is also responsible for
identification of potential risks that might
impact testing.
Not Enough Training/Lack of Test Competency
Us versus Them Mentality
Lack of Test Tools
Lack of Management Understanding and Support of
Testing
Lack of Customer and User Involvement
Not Enough Schedule or Budget for Testing
Over Reliance on Independent Testers
Rapid Change
Testers are in A Lose-Lose Situation
Having to Say No
Test Environment
New technology

19
Risk Analysis

The objective of performing risk analysis as part
of test planning is to help allocate limited test
resources to those software components that pose
the greatest risk to the organization.
Performing risk analysis during test planning is
a four-step process as follows
Form the risk analysis team
Identify risks
Estimate the magnitude of the risk
Select testing priorities
Skills needed for members of a Risk Analysis Team
Knowledge of the user application
Understanding of risk concepts
Ability to identify controls
Familiarity with both application and information
services risks
Understanding of information services concepts
and systems design
Understanding of computer operations procedures
Candidates for the Risk analysis team should come
from the user area any of the following areas
Software testing
Risk consultant
Project leader

20
Risk Analysis(Continued)

The magnitude of a risk can be determined by any
of the following means
Intuition and Judgment - In this process, one or
more individuals state they believe the risk is
of a certain magnitude.
Consensus - A team or group of people agrees to
the severity of magnitude of a risk.
Risk Formula - Using this process, the
probability or frequency of an event occurring
must be determined, and the loss per frequency
must be determined.
Annual Loss Expectation (ALE) Estimation
Make a preliminary assessment of the loss.
Make a preliminary assessment of frequency using
a frequency table of multiples of five as in
Figure 32.
Calculate an ALE using the loss and frequency
tables in Figure 31 and Figure 32.
The Annual Loss Expectation can be lowered by
Using multiple opinions to establish most likely
ALE.
Rank intangible elements (see Figure 33 below).
Assign monetary values to intangibles.
Accumulate additional evidence, if needed.
Make a post-analysis challenge of the decision.
Document the decision reached.
Considerations which may impact the
prioritization of risks
Compliance required to laws and regulations
Impact on competitiveness

21
Risk Management

Risk management is a totality of activities that
are used to minimize both the frequency and the
impact associated with risks.
two activities associated with risk management
which are
Risk Reduction Methods
Contingency Planning
Risk Reduction Methods
The Loss due to risk the frequency of
occurrence multiplied by the loss per occurance
This can then be compared with the estimated cost
of implementing a control to see if it is
economical to implement a control to reduce the
loss.
Contingency Plans Plans established for
activation when a loss is known to occur for a
given risk
The role of testers is to evaluate the adequacy
of the contingency plans associated with risk.

22
Prerequisites to Test Planning

There are five prerequisites to test planning
Test Objectives
Acceptance Criteria
Assumptions
People Issues
Constraints
Test Objectives
Test to assure that the software development
project objectives were met
Test to achieve the mission of the software
testing group (This didnt make a lot of sense)
Testing the functional and structural objectives
of the software - Meeting requirements
Testing to meet the needs of the users Fit for
use
Acceptance Criteria
A method of communicating the characteristics of
a desired software system.
This can be defined by the development staff
This can be defined by the end user
Its purpose is to facilitate good communication
between the IT organization and its users.

23
Prerequisites to Test Planning(Continued)

People Issues
Tend to be both political and personal
Should be resolved before starting development
Typical Issues
Who should run the project
Who can make decisions
Which organizational group has authority to
decide requirements
Its been tried before and didnt work
When identifying issues people can be divided
into four categories, they are
People who will make the software system happen.
People who will hope the software system happens.
People who will let the software system happen.
People who will attempt to make the software
system not happen.
Constraints
Three common constraints Test Staff Size, Test
Schedule, and Budget
Constraints must be integrated into the test plan

24
Create the Test Plan

The test plan describes how the testing will be
accomplished
The test plan should be an evolving document
The test plan should provide background
information on the software being tested, test
objectives and risks, and specific tests to be
performed.
Tests in the test plan should be
Repeatable
Controllable
Ensure adequate test coverage
Understand the characteristics of the software
being developed
Definewhat it means to meet the project
objectives.
Understand the core business areas and processes.
Assess the severity of potential failures.
Identify the components for the system
Links to core business areas or processes
Platform languages, and database management
systems
Operating system software and utilities
Telecommunications
Internal and external interfaces
Owners

25
Create the Test Plan(Continued)

Assure requirements are testable.
Address implementation schedule issues
Implementation checkpoints
Meetings
Identification and selection of conversion
facilities
Time needed to put converted systems into
production
The conversion of backup and archival data
Address interface and data exchange issues
Development of a model showing the internal and
external dependency links among core business
areas, processes, and information systems
Notification of all outside data exchange
entities
Data bridges and filters
Contingency plans if no data is received from an
external source
Validation process for incoming external data
Contingency plans for invalid data

26
Create the Test Plan (Continued)

Evaluate contingency plans for this system and
activities.
Identify vulnerable parts of the system and
processes operating outside the information
resource management area.
Build the Test Plan The development of an
effective test plan involves
Set Test Objectives
Test objectives should restate the project
objectives from the project plan.
When defining test objectives it is best to keep
them at ten or fewer.
To define test objectives testers need to
Define each objective so that you can reference
it by a number.
Write the test objectives in a measurable
statement,
Assign a priority to the objectives (High,
Medium, Low)
Define the acceptance criteria for each
objective.
Develop the test matrix
The test matrix lists which software functions
must be tested and the available tests.
It shows How the software will be tested.
Define tests as required
Define Conceptual Test Cases to be Entered as a
Test Script
A conceptual test script is a high-level
description of the test objectives, not the
specific test cases that will be entered during
online testing.

27
Create the Test Plan(Continued)

Define the test administration
The administrative component of the test plan
identifies the schedule, milestones, and
resources needed to execute the test plan
State Test Plan General Information, this
normally includes
Software Project
Summary
Pretest Background
Test Environment
Test Constraints
References
When to Stop
Define Test Milestones
Test milestones are designed to indicate the
start and completion date of each test.