Load Testing Approach

1
Load Testing Approach

• Quick Guide to Plan and Execute Load Test Using
  Load Runner!
• -Ranjit
• jcrvs_at_hotmail.com

2
Why Is a Approach Needed?

• Quality approach work smarter to be faster.
• Do it right first time, every time.
• Detailed approach that prevent from getting lost
  in the load testing sea.
• Use of smart tools load runner.

3
Conventional Approach

• Random requirements from management fro load test
• In-experienced staff
• No requirements analysis performed for load test
• Easy to get carried away in load testing

4
The Approach

• Perform requirement analysis to get information
  as
• Type of load test
• User load estimation
• User load distribution
• User activity analysis
• Production environment analysis
• Database size
• Report generation
• Simple metric collection

5
Type of Load Test

• Determine the goal of the test
• Do we intent to find the synchronization
  problems?
• Do we intend to perform a load test? Or a stress
  test? Or a performance test?

6
Synchronization Problem Tracing Load Test

• Goal of this test is to determine the synchronous
  issues and trouble some areas.
• This test may need vu-generated scripts with lots
  of rendezvous points or rendezvous points at each
  post/action.

7
Load/stress/performance

• Different type of tests
• Load test determines how is the performance of
  application under the concurrent user sessions
  for typical user scenario. The think time taken
  into consideration in these test scripts.
• Stress test examines how application behaves
  under maximum load. In simple terms find the
  upper threshold for the application below which
  it can work normally. Think time ignored in these
  tests.
• Performance test indicates response time for the
  entire application from the users perspective.

8
What Do U Want?

• Do determine what kind of test do you want.
• Then plan ahead.

9
User Load Estimation

• A detailed feed back form the marketing/business
  development will give a idea of the user load or
  the number of users using the product. This will
  determine the load to be used against the product
  in testing.
• An inexperienced staff may configure a load test
  to simulate 1000 v-user but user base for the
  application may be not more than 400. This may
  result in licenses being lost for the v-user and
  time and efforts as well.

10
User Load Distribution

• A detailed feed back form the marketing/business
  development will give a idea of the user load
  distribution.
• The user load may be peak in the morning or
  afternoon and very less in the evening.
• This may also help determine the concurrent users
  and simultaneous users.

11
User Load Distribution

• This factor will input to the scenarios to be
  used and configured in the load test.
• E.G. BD may come up with a user load of 1000
  users per day and each grouped to perform certain
  activity as say 30 in section A of application
  rest in section B of the application etc.

12
User Activity Analysis

• A detailed discussion with marketing/BD may
  reveal the user activity details on which
  detailed scripts can be written.
• E.G. In morning there may be 200 forms submitted
  or in evening most of the users may login and
  perform results generation activity so in morning
  scenario out of 1000 concurrent user, 70 may be
  performing this activity.

13
Production Environment Analysis

• A discussion with the I.T. Department will throw
  light on deployment environment.
• This may input valuable information as does the
  environment have enough hardware? Is it running
  in a clustered environment?

14
Production Environment Analysis

• Mirror the production environment into a test
  bed.
• Alternatively create test bed and after load test
  suggest production environment to management.

15
Database Size

• The database size does matter in the load test.
• Bulky the database, more the latency.
• Define the database size prior to load test e.G.
  Conducting first set of test on a 50K sized DB
  then a 100K sized DB and so forth.

16
Report Generation

• Generate two types of reports
• Load runner detailed reports for engineering
  department
• Generic reports for the activities performed in
  the scenario and response time details with other
  observation and conclusions for the management
• Generating graphs also helps
• Helps in tracking per load cycle results
• Easy interpretation for people

17
Simple Metric Collection

• Basic metrics can be collected as follows
• CPU utilization should not exceed 60.
• Per page size. This is a indicator of the
  bulky-ness of the page. Response time for each
  transaction.
• Configuration details.
• Test bed details.

18
Simple Metric Collection

• Total hits and hits/second, should not be greater
  than 20 or request queue details need to be
  collected.
• Failed transactions, should not be greater than
  5.
• Number of processes running on server.
• Memory details.

19
Simple Metric Collection

• Load runner can be used to collect the above
  metrics.
• Other option is to use O.S. Specific tools.

20
Plan on the Above Analysis Clubbing All Together

• After the requirement analysis, feed all the data
  into a test plan.
• Test plan will contain the type of test to
  perform, test bed details, load/user details and
  activity scenarios.

21
Plan on Scenarios

• Plan scenarios for execution based on the above
  input and different daytime for the day as
  morning scenario, afternoon scenario etc.
• Each scenario may scale to 3 to 4 hours. So
  activity analysis as number of forms submitted
  per day etc and time may help to set the number
  of iteration for each scenario.

22
Follow up Actionscripts

• Create test scripts with vu-generator for the
  interactive activities to be performed with
  application.
• Unit test in vu-generator debug mode.
• Also run the same in 10 user 10 iteration load
  runner scenario mode. Use the option show
  v-user to check for the activities being
  performed.
• Parameterize scripts and co-related them.
• Make scripts dynamic, more efforts to be put in
  for making dynamic than just co-relating the
  scripts.

23
More on Scripts

• Add comments and queries to fetch the
  parameterized data from database to make it user
  independent.
• Logs may reveal error which at time may be
  hidden.
• In runtime settings, use option as mark each step
  as transaction to prevent adding manually
  transaction function for each step.

24
Configuring Scenarios

• Recommended to unique create groups, each group
  is represented by one script.
• Use ramp-up and ramp-down features to prevent
  instantaneous loads on load test server. Load
  runner gives one click access to pre-configured
  slow ramp-up and ramp up scenario.
• If scripts are dynamic then recommended to use
  duration based scenario.
• More refined control can be achieved for scenario
  by scheduling by group rather than scheduling by
  scenario.

25
Configuring Scenarios

• Enable standard logging.
• One click generator configuration can be
  performed using load runner.
• In runtime settings, use option as mark each step
  as transaction to prevent adding manually
  transaction function for each step.
• The think time in-between two iterations is
  recommended in load tests.
• Proxy may or may not be needed. Recommended not
  to use proxy to get a benchmark results first,
  then run the same scripts though proxy and
  compare results.

26
Configuring Scenarios

• Simulating browser cache option may be turned off
  to make sure each request is fetched form the
  server.
• Bandwidth throttling may or may not be used as
  per requirements.
• Enable web performance graphs option in load
  runner to monitor at test runtime the results.

27
Configuring Scenarios

• Disable or enable continue on error option.
  Recommended to enable it for cluster fail-over
  tests and also regular load test.
• The option to run v-user as a process or thread
  may be used as per requirements. Recommended to
  use option is thread to prevent overhead of the
  memory and system resources. But also more
  threads per process makes the process unstable.
  Load runner allows configuration of number of
  thread per process for stability.

28
Execute Tests and Report

• Prepare test bed and execute planned scenarios.
• Recommended server restart before each test
  cycle.
• Generate reports for each cycle.

29
In a Nutshell

• Planned approach make life simple and makes load
  test simple too.
• Requirement analysis is important activity before
  starting load test.
• Metrics are the end results so save results for
  each cycle and analyze.
• FLY HIGH with simple minds and simple practices
  !!

30
Glossary

• LR- load runner
• O.S. Operating system
• BD business development

31
Q A