Performance Testing

1
Performance Testing

• Back to Basics
• May 2013

2
Who Am I?
3
Who Am I?
4
Who Are You?
5
Who Are You?

• Performance testers?

6
Who Are You?

• Performance testers
• Testers/QA?

7
Who Are You?

• Performance testers
• Testers/QA
• Developers?

8
Who Are You?

• Performance testers
• Testers/QA
• Developers
• CFR Cluj Supporters?

9
Today

• 4 sessions about an hour each
• Some practical work in groups
• Any questions/comments speak up
• Please share your thoughts/experiences

10
Conversation NOT lecture
11
Performance Testing

• Back to Basics

12
What is Performance Testing?

• ?

13
What is Performance Testing?

• performance testing is in general testing
  performed to determine how a system performs in
  terms of responsiveness and stability under a
  particular workload?

Wikipedia 2013
14
What is Performance Testing?

• Load Testing
• Stress Testing
• Soak Testing
• Spike Testing
• Configuration Testing
• Isolation Testing

15
Some History
16
Traditional Performance Testing

• understand the volumes
• Model the business processes
• Run experiments (tests)
• Measure performance indicators (response time,
  cpu etc.)
• Tune if needed
• Sign off report

17
Some History
18
Some History
19
Some History
20
From This
21
To This
22
Applications have changed

• More complex functionality
• Minimal defined business processes
• Diverse Public Internet
• Use of services (homegrown and third party)
• Cacheing
• Increased Client functionality (javascript etc.)

23
Applications have changed

• Cloud computing
• Multiple client platforms (tablet, mobile etc.)
• Push technology

24
Traditional Performance Testing
Problems

• Focus on IT/IS requirements
• Too difficult to model the entire application
• Length of time it takes to run a performance test
• Models have too many variables to be reliable

25
New Approaches to Performance

• As applications have increased in complexity and
  application usage has become more diverse we have
  developed new tools and approaches to help us
  predict and manage the performance of
  applications

26
New Approaches to Performance

• Synthetic testing (traditional perf testing)
• A/B testing
• Instrumentation/Analytics (RUM)
• React Fast
• Over_Provision
• Third Party tools

27
Performance Testing

• Back To Basics

28
Rank Sites by Performance

• News related websites
• www.nytimes.com
• news.bbc.co.uk
• www.huffingtonpost.co.uk
• tv.msnbc.com
• www.cnn.com
• www.dailymail.co.uk

29
Rank Sites by Performance

• News related websites by response time
• news.bbc.co.uk (0.7)
• www.dailymail.co.uk (3.3)
• www.cnn.com (3.5)
• www.nytimes.com (3.9)
• tv.msnbc.com (5.5)
• www.huffingtonpost.co.uk (6.3)

Gomez April 2013
30
Rank Sites by Performance

• News related websites by popularity
• www.cnn.com
• tv.msnbc.com
• www.nytimes.com
• www.huffingtonpost.co.uk
• www.dailymail.co.uk
• news.bbc.co.uk

31
The Five W's
32
The Five W's

• WHO?

33
The Five W's

• WHO?
• WHAT?

34
The Five W's

• WHO?
• WHAT?
• WHY?

35
The Five W's

• WHO?
• WHAT?
• WHY?
• WHEN?

36
The Five W's

• WHO?
• WHAT?
• WHY?
• WHEN?
• WHERE?

37
Who is interested in performance?

• IS/IT Department

38
Who is interested in performance?

• IS/IT Department
• Protection of production
• Stability
• Capacity
• Resource

39
Who is interested in performance?

• Customer

40
Who is interested in performance?

• Customer
• Stability (of Browser)
• Freezing
• Consistency
• Ease of use (fewest steps to get job done)

41
Who is interested in performance?

• Business

42
Who is interested in performance?

• Business
• Revenue
• Unique monthly visitors
• Conversion rates
• Site Abandonment rate

43
Who is interested in performance?
As Application complexity has increased so too
has the amount of performance related information
available As a performance tester it is
important to be able to interpret the data and be
able to present information that is useful and
relevant to people.
44
Test models

• Choose 2 key business processes for each site
• www.amazon.com
• www.lastminute.com
• www.digitalspy.co.uk

45
Test models

• Choose 2 key business processes for each site
• www.amazon.com

46
Test models

• Choose 2 key business processes for each site
• www.lastminute.com

47
Test models

• Choose 2 key business processes for each site
• www.digitalspy.co.uk

48
What to Test?

• Information used to prioritize testing

49
What to Test?

• Information used to prioritize testing
• New technology
• Heavy usage
• Spikes in usage
• Key revenue generators
• Customer goals (or hot buttons)

50
What to Test?
As it has become harder to model and test an
entire application, being able to identify which
areas of an application are key for performance
is now more important than ever
51
Test models

• Choose 2 key business processes for each site
• www.facebook.com
• www.youtube.com

52
Why Do we Test?

• ?

53
Why Do we Test?

• Prevent problems
• Validate improvements
• Reproduce problems
• Aid in decision making

54
Why Do we Test?

• What Does a Successful Test look like?
• Setting goals
• Convert business goals into technical model
• Volumes (Average, Peak)

55
Understanding business process

• Using Webpagetest
• Free tool to allow you to see underlying calls
  made when you execute a business process
• Identify Key calls for performance

56
Understanding business process

• Using Webpagetest
• http//www.webperformancetoday.com/2010/07/09/wate
  rfalls-101/

57
Using Webpagetest

• News related websites
• www.nytimes.com
• news.bbc.co.uk
• www.huffingtonpost.co.uk
• tv.msnbc.com
• www.cnn.com
• www.dailymail.co.uk

58
Understanding business process

• Identify Key calls for performance
• Based on who is Interested
• Based on what the key business processes are
• Based on why you are testing

59
Why do we Test?
Integrate business goals with technical and
customer goals Be wary of using
design/developers as way of understanding what
the application really does
60
When to Test?

• Traditionally performance testing was done just
  prior to release. Reasons for this
• Models take time to build
• Need for stable baseline to compare against
• Site must be functional before perf testing

61
When to Test?

• Rise of Agile requires new approaches
• Split up into smaller component models to allow
  testing during development
• Clustering production allows testing in
  production with incremental rollout
• Test in design

62
Web Sequence diagrams

• Example for Facebook Login
• http//www.uml-diagrams.org/sequence-diagrams-exam
  ples.html

63
Where to Test?

• Dedicated Performance environment
• Prod
• Disaster recovery
• QA/Dev Environment
• Preprod
• Isolated components

64
Where to Test?

• Dont forget the surrounding infrastructure
• Network
• Load Balancers
• Caches
• CDN
• Third Party services

65
How?

• Lots of Vendors promising magic. Get demos and
  proof of concept for your site
• Only once you have a good understanding of the
  basics for your site.

66
How?

• Dont forget the basic free tools available
• WebPageTest
• Browser developer tools
• Curl
• SoapUI/LoadUI
• Many others

67
Questions