Performance Test Planning Part 2

1
Performance Test Planning Part 2
CS 4803 EPR
Enterprise Computing Performance
2
Lecture Overview

• Performance Test Planning Part 2
• Scalability
• Test strategy
• Instrumentation
• Test data
• Test cases
• Case Studies and White Papers
• Student Presentations 1

3
Scalability

• Smith definition
• Scalability is the ability of a system to
  continue to meet its response time and/or
  throughput objectives as demand increases.

4
Scalability (contd)

• Connie U. Smith
• To evaluate scalability, gradually increase the
  arrival rate of performance scenarios and look
  for the knee of the scalability curve.

5
Scalability (contd)

• Two ways to evaluate System Scalability
• Increase load while measuring response time
  against baseline response time
• Increase hardware while measuring HW resource
  utilization against baseline resource utilization

6
Scalability (contd)

• Near linear scalability
• Generally .71 or better
• Full Linear scalability
• 11 never achieved with distributed system

7
Scalability (contd)

• 2 Types of Hardware/System Scalability
• Horizontal Scalability (scale out add
  machines)
• Vertical Scalability (scale up add system
  resources)

8
Test Strategy

• Clearly document your intended strategy

2.4 Strategy The following details the specific
strategy for FM in this round of FRP performance
testing. Mercury LoadRunner scripts will simulate
the VB/CODA apps via the Oracle 2-tier
protocol. Journal Entries The Performance Team
will load Journal Entries using LoadRunner with
one user test case against either the VB or CODA
fat client. The load will be a specified
percentage of the total 500 concurrent
users. Browse Details The Performance Team will
load Browse Details using LoadRunner with one
user test case against the CODA fat client. The
load will be a specified percentage of the total
500 concurrent users. Reports per Hour Assuming
a 10-hour day, SQLs will be executed by the FM
team, to verify that, under nominal load, FM can
support 350 reports per hour, or the rate of
3,500 reports per day. Back-end Interfaces The
FM team will write and execute shell scripts to
verify that the Back-end Interfaces that will
yield a minimum of 15 documents per second, while
under nominal load.
9
Instrumentation

• Smith 8.6/9.2

Inserting code (probes) at key points to enable
the measurement of pertinent execution
characteristics.
Instrumentation Principle
Instrument systems as you build them to enable
measurement and analysis of workload scenarios,
resource requirements, and performance objective
compliance.
10
Test Data

• Types of Test Data
• DB Resident
• Data physically residing in the db
• Parameterization Data
• Data derived from system db
• Synthesized data

11
Test Data

• Data is Mission Critical!
• Test Planning
• 40
• Simulation scripting
• 30
• Data Development
• 30

12
Test Cases

• Shares a many to one relationship to Use Cases
• Much more granular than Use Cases
• Contains steps quantifiable actions
• Must be traceable to a Use Case which is in turn
  traceable to a requirement (traceability)