Measurement Based Performance Characterization of Software Servers'

1
Measurement Based Performance Characterization of
Software Servers.

• Under the guidance of
• Prof. Varsha Apte.

By B.Nagaprabhanjan (03329010)
2
Agenda

• Introduction
• The process of performance measurement
• Problem Statement
• Work Done
• Survey of tools
• Conclusion
• Future Work
• Issues

3
Introduction

• Performance
• - An important Attribute of any computer system.
• - Has got commercial importance.
• Three methods of performance evaluation
• Analytical modelling.
• Simulation.
• Load Testing and Capacity Analysis.
• -- The last gives near accurate results than the
  other two.

4
The process of Performance Measurement

• Load generation
• Resource Monitoring
• Capacity Analysis

5
Load Generation

• Basic Requirement
• Need to simulate the real load as accurately as
  possible.
• Characteristics of real workloads
• Timeouts
• Retries
• Thinktime
• Sessions
• Issues
• Scalability
• Flexibility
• Interoperability

6
Resource monitoring

• -- Helps in finding resource bottlenecks of
  system.
• Some parameters that can be monitored
• Cpu utilization
• Incoming/outgoing packet rate
• Disk I/O rate
• Memory consumption
• Needs some help from the underlying operating
  system.

7
Capacity Analysis

• -- Analyzing the data collected from the
  previous phases. Still done manually.
• Some problems with manual intervention
• Synchronizing load generation and resource
  monitoring can be error prone
• Repeating the same steps a number of times
• Many man hours are wasted.
• The problems increase with the complexity of the
  system to be tested.
• Hence there is a need to automate the capacity
  Analysis.
• This is the main focus of our project.

8
Performance Measurement in a Distributed
Environment

• - In an enterprise environment, a request is
  serviced by many nodes in the network.
• - The performance of such a system depends on the
  performance of all the nodes in the network.
• - Two things are necessary to measure the
  performance of the system
• The topology of application.
• Resource requirements of components.
• Finding the resource requirements is the initial
  focus of our project.

9
Problem Statement

• Initial Focus
• To find the resource requirements of the
  components of applications in a distributed
  environment by writing a generic tool.
• Main Focus
• Automated Capacity Analysis.
• Objectives
• To integrate the load testing and performance
  measurement and automate capacity analysis.
• Reduce user participation and hence sources of
  errors.
• Minimize test cyles and hence accelerate
  deployment.

10
Work Done

• 1. Since Load testing and resource monitoring are
  two fundamental things to be done, a survey of
  load generators and resource monitoring tools is
  done.
• 2. Setup a test environment with a sample bank
  database, and measured the performance of point,
  multipoint, range and join queries.

11
Survey of Load Generators

• Torture
• - Simple, Scalable, Tests for buffer overflow
• Pure Load
• - HTTP Recorder, Web crawler
• eValid
• - Inbuilt scheduler
• OpenSTA
• - Distributed Architecture, Server Monitoring
  using SMTP
• Web Load
• - Supports a variety of protocols, can work as a
  permanent monitor.

12
Survey of Load Generators (Contd..)

• Silk Performer
• - Enterprise Class software. Simulates multiple
  connection speeds, Very accurate timers
• Load Runner
• - Can test entire enterprise infrstructure,
  highly scalable
• Web Avalanche
• - A hardware, software combiation. Can simulate 2
  million connections.
• WAPT
• - Nice User Interface, Ability to test multiple
  servers simultaneously
• Httperf
• - Its own timers, Can support sessions, Good
  design.

13
Resource Monitoring Tools
Linux Resource Monitoring tools top - can show
percent CPU busy, total memory occupied, total
idle time etc. iostat, vmstat - monitors system
input/output device loading, their average
transfer rates, paging activity on a system
basis. ps - displays information about processes,
owner information, process state, size etc. sar -
reads /var/log/pacct and displays information
about processes that are terminated.
14
Resource Monitoring Tools(contd..)

• Some Projects
• CSA- Comprehensive System Accounting
• Patches the kernel, and makes it write more
  information about processes.
• CKRM - Class Based Kernel Resource Management
• - Provides differnetiated services to processes
  by classifying processes, acts like resource
  manager.
• ELSA Enhanced Linux System Accounting
• - Similar to CSA. Uses a structure called banks
  to get resource usage information about a group
  of processes and not for a single process. Adds a
  system call called elsa()

15
Resource Monitoring Tools(contd..)

• LTT- Linux Trace ToolKit
• A tool to extract minute program execution
  details.
• Central concept is 'event'
• Disk Scheduling is an event, program scheduling
  is an event, start of network activity is an
  event...
• Writes a detailed log of the events of various
  processes.
• Logs can be analyzed to get minute details of
  processes.
• Modifies kernel, only supports a couple of kernel
  versions.
• Suitable for our work.
• Problems
• Supports only a couple of kernels( 2.5.7 and
  2.4.16)
• Difficult to install.

16
Conclusion

• Most of the load generators are targeted to test
  the web servers.
• There are very few load generators to generate
  load for any server like IMAP server, database
  server etc.
• Most of the resource monitoring tools are not
  capable of providing finer details.
• Among the tools studied, The Linux Trace Tool Kit
  can provide minute measurement details which are
  suitable for our purpose.
• Inspite of sophisticated tools, capacity analysis
  is still done manually.

17
Future Work

• For the initial work
• Write a generic tool to find the resource
  requirements of components in distributed
  environment.
• For capacity Analysis
• Writing a load generator with open and closed
  arrivals.
• Determining steady state and thresholds of
  parameters like throughput, maximum number of
  users etc.
• Finding knee points of curves.
• Finding the distributions of parameters.

18
Issues

• For Resource Requirements in a distributed
  environment
• The logs found in different systems have to be
  synchronized w.r.t a global clock.
• Synchronizing the load generator and resource
  monitoring tool is necessary to get good results.
• For Automated Capacity Analysis
• Finding knee points require complex algorithms.
• Synchronizing the two phases has to be accurate.
• Determining steady state is dependent on number
  of users.