P1253553574XRBUH

1
Evaluating Transaction System Performance Using
Real Workloads
Ashutosh Tiwary Co-Founder and CTO www.performant.
com
2
Performance is a real problem

• IT projects fail due to multiple reasons
• Performance is an important component
• Application complexity is growing dramatically
• Many layers of opaque 3rd party components
• More and more IT developers know about less and
  less
• Deploying scalable J2EE applications is hard
• 12 SUN CPUs, 12 GB J2EE App server performance?
• Packaged apps have their own challenges
• 10M - 1B spent on deployments

3
Performance problem examples

• Inability to recreate real world problems
• Our frustration is that we cannot recreate the
  production problems in our test environment
  (Director of
  Performance Testing)
• Inability to anticipate problems with current
  tools
• Why did we not find this problem with our load
  testing before we deployed the application?
  
  (VP Engineering)
• Inability to predict future performance
• Will our current system be able to handle the
  anticipated load for next 24 months?
  (VP Information Systems)

4
Real Workload Approach

• A new approach
• Real workload instead of synthetic
• Diagnose and fix performance problems in
  production systems
• A view inside the application while under load
  detailed analysis
• Automated application instrumentation

5
Current implementation OptiBench

• HTTP Instrumentation using plug-ins
• Support major web servers with minimal overhead
  (1-5)
• Automated byte code instrumentation of Java
  interfaces for J2EE applications
• JSP and JDBC on major application servers
• Insert byte-code into Jar files, minimal
  execution overheads (1-7)
• General purpose automated instrumentation
  technique that can instrument J2EE and
  potentially MS .NET interfaces
• Multi-user and distributed capture, analysis and
  playback of workloads
• Test management, data analysis
• Correlation of system and application metrics
  with workloads

6
Sample Usage Scenarios

1. Solving production performance problems
2. Load testing with a view inside the application
3. Scaling and capacity planning
4. Benchmarking

7
Production Problem-Solving
TEST SYSTEM
EXISTING SYSTEM
DEVELOPMENT SYSTEM
PERFORMANCE SERVER
Workload Performance
WORKLOAD
CODE FIXES
PERFORMANCE
Triggers

• Process
• Instrument application and deploy to production
• Set workload capture thresholds using monitoring
  systems. Capture workload and state when problem
  occurs.
• Playback the workload to recreate the problem in
  the test environment to diagnose and fix it.
• Replay the workload with the code fixes to
  validate that the problem is solved
• Deploy the fixed code to production

Monitoring System
OptiBench Console
Monitoring Console
8
Conclusions

• Innovative technology and shipping product
• Recording, playback, and correlation of real
  production workload with low overheads
• Addresses real performance problems for customers
• Product currently being deployed at major
  customer sites

9
(No Transcript)