Title: the ebusiness management company
1- the e-business management company
SPIN March 21, 2000
2Program Mission
eConfidence
Unmatched expertise to deliver e-business
reliability solutions
3The Risk Site Availability and Performance
Threatened by Constant Change
- New Functionality
- Web Front-end
- Web and App Server Logic
- Front- and Back-office Systems
- Inter-business Systems
- Business Conditions
- Traffic Spikes from Planned Promotional/Seasonal
Activities or Unplanned Market Conditions
- Technology
- Browser Versions
- Web/App Server Versions
- Load Balancing Introduction/Changes
- Middleware Versions
- Security Tightening
- ERP Versions
- DBMS Versions
- Front-office Development Tool Versions
- OS Version Changes
- New Hardware
4The Result
- Delayed Rollout of Valuable Improvements
- Loss of Revenue
- Loss of Credibility
- Business Consequences of Poor Site Availability/
Performance - Loss of Market Share
- Missed/Delayed Cost Savings
- Sanctions from Regulators
5e-Business Reliability Testing Needs
Web Clients
. . .
Observing
CORBA
EJB
Monitoring
RMI
Tuxedo
Metering
. . .
Performance
6The Solution Run your e-business with
eConfidence
- Regression Tools
- Configuration of Functional Testing for Browsers
- Configuration of Functional Testing for Browser
UI Components - Creation of Test Harness/Test Suite
- Implementation of Transactions
- Performance Tools
- Customization and Configuration of Load Testing
Solution and interaction with Web Server - Implementation of Load Tests to Determine Site
Capacity and Performance Degradation Points
7The Solution Run your e-business with
eConfidence
- Benefits of Test Automation
- Cost Savings
- Time to Market Savings
- Repeatability
- Testing Breadth and Depth
8Real World Solutions
To be successful, you need
- A modular approach to component and functional
testing - connect the dots - Tools that meet the diverse demands of a mix of
technologies for web and client/server
applications - Support for test planning and management - too
many chefs in the kitchen?
9Functional Testing ToolsKey Feature - Support
for a Mix of Technologies
Feature - support for all of the major
platforms, browsers, and development
environments. Benefit - use one testing tool for
functional and regression testing
of many components
Windows NT Windows 2000 Windows 95,
98 UNIX 3270/5250
Internet Explorer Navigator
C/C Java Visual Basic ActiveX
10Functional Testing ToolsKey Feature - Automated
Testing
Feature - automated testing Benefit - maximize
productivity by letting your testing tool do
the work for you
- Choose recording, scripting, or both to create
tests - Reuse testcases with a data driven approach
- Run unattended tests overnight using a recovery
system and still get meaningful results - Schedule and Drive your scripts from a central
point of control, even on different platforms
11Functional Testing ToolsKey Feature - Test
Planning and Management
Feature - test planning and management Benefit -
plan, coordinate, control and report on
group testing efforts
-
- Centralize and modularize your test plan with a
master plan and sub-plans - Include manual tests for progress and completion
reports - Query your plan to flag and run only those
testcases that meet specified criteria - Prepare reports on testing progress, including
graphs
12Functional TestingKey Feature - Direct Database
Verification
Feature - direct database verification Benefit -
test the complexity of your database without
relying on the user interface
- Begin database and black box testing before the
user interface is ready - Validate database components that have no UI,
such as stored procedures, Remote Procedure
Calls, triggers, and data replication - Connect to any ODBC compliant database
- Connect to a local database host or to a
networked server database
13Functional Testing
Feature - The more the tool can do, the
better Benefit - Spend less time managing tools,
more time interpreting results
-
- Repository for storing and managing results
- Defect tracking integration or communication
- Link checking
- Environment should be user friendly
14The Technical Challenge of E-BusinessHow To Load
Test This?
Simulate Impact of Multi-Network Components
Simulate Traffic of Client/Server Components
Simulate Impact of Web Components
Simulate Integration of Packaged Applications
Traffic
Load Testing Tool
15Load Testing Feature - Multiple
Protocol/Environment Testing
Feature - Support Multiple Protocols and
Environments Benefit - Enables testing web,
client- server, database, and ERP systems
- Test the integration of multiple applications on
one common resource (i.e. Database) - Support all major Web Protocols (HTTP, HTTPS,
FTP, POP3, SMTP, SOCKS, LDAP, TCP) - Support Database Load-Testing for ODBC and
popular Databases (Oracle, SQL Server, etc.) - Support Popular Middleware Systems and Protocols
(CORBA, Tuxedo, MQ Series)
16Load TestingKey Feature - Results Reporting
Feature - Allow you to extract from your load
testing accurate, useful information Benefit -
Use this information to understand current
scalability and plan for future needs
Display your results in multiple formats and
compare run-to-run results.
Display only the system and results information
you want
17Load Testing Key Feature - Ease of Use
Feature - Ease of Use Benefit - Comprehensive
creation, execution and customization
settings are available
-
- Record tests quickly and easily
- Scripting language allows customization
- Monitor script, system and results information in
real-time - Setup tests that make use of error limits, custom
timeout settings and external data - Customize users to emulate real-world settings
of connection speeds, browser-types and think
times
18Load Testing Key Feature - Realistic load
generation
Feature - Load generation as real as
possible Benefit - Accurate loads create accurate
statistics for intelligent analysis and
conclusions
-
- Create multi-user/multi-transaction load tests
- Include IP address simulation
- Modem speed simulation
- Replay secure traffic
- Real simulation of browser cache
- Full integration of server side statistics and
custom statistics
19eConfidence in Production
- Reliability Plan Specific to Business and
Technology Plans for the Site - Metrics for the Site's Current Functionality and
Performance - Implementation of Functionality and Performance
Tests Specified in the Reliability Plan - Periodic reports on reliability and change
metrics - Recommendations for Ongoing Production Reliability
20eConfidence in Production
- Baseline and Plan Development
- Establish a health status on the sites current
functionality, uptime and performance - How much of the site currently works
- Response times for typical activity and
transactions - Site performance break points
- Develop a specific reliability plan
- Reports baseline findings
- Identify risks of your business and technology
plans - Point-by-point plans for addressing risk
21eConfidence in Production
- Reliability Plan Implementation
- Develop and run scenarios
- Diagnose findings
- Share results with technology owners and users
- Monthly/quarterly reports on reliability and
change metrics
22Market Drivers
Significant investments in e-Business applications
demand results
- The best website in your space, driven by market
conditions and technology - Reliability across multi-tiered, technologically
complex system - Maximum return on your QA investment
2324 Hour Site Monitor
What does this tell me about the overall
performance of my web site?
2424 Hour Site Monitor With Competitors
How does my web site compare to other sites?
25Transaction Comparison
How was the time spent?
26Where To Now? Testing...
How do I get accurate results?
- Browser Simulation
- Threads, Browser Specific Content (DHTML,
ActiveX, etc.) - Connection Speed Simulation
- Business Users (T1) versus Home Users (Modems,
ISDN) - Workload Type
- Steady-State Performance Metrics
- Increasing Workload Server Scalability, Breaking
Point - Multiple Machines/Multiple Addresses Per Machine
- Distributed Testing, IP Address Simulation
- Randomization Of Data
- Uniform, Individual, Binomial Distributions
- In Script or From File
27How do I find problems with my site?
Errors as number of users crosses over 3200
users
error rate
28Server Busy Time
How is response time affected by number of users?
29Another View Of Response Time
What does this tell us about our servers
scalability?
30Run Some Tests, Make Some Changes...
How did tuning my server affect its performance?
31Real World Solutions
To be successful, you need
- To ensure your applications scalability -
Downtime and Limited Capacity mean lost business - Support for the e-Business development model -
You MUST validate that newly introduced
applications DO NOT create bottlenecks for
existing applications - Tools which meet the diverse demands of varied
users responsible for Web application development
and deployment
32Summary Run your e-business with eConfidence
- Faster site improvement and change adoption with
minimized risk - Improved decision-making on sites ability to
meet business plans - Ingrained reliability awareness and methodology
- Improved team skills and intra-team processes
- Focus on business goals
- Time/Cost savings