Unobtrusive%20Performance%20Analysis%20

1
Unobtrusive Performance Analysis Where is the
QoS in TAPAS?

• University College London
• James Skene j.skene_at_cs.ucl.ac.uk

2
Overview

• What are we modelling?
• Why is performance modelling important?
• Why is it important for TAPAS?
• What is the state of the art
• In performance modelling?
• In system modelling?
• What problems must be addressed?
• An example approach.

3
What are we modelling?
Beans
User
Beans
Container
Container
Database
Network
Network
4
Why is performance modelling important?

• During development
• Wed like to be able to predict when the design
  will cause performance problems
• Wed like to be able to combine services and
  components and predict the overall performance
• Wed like to be able to plan the capabilities of
  our services.
• During operation
• Wed like to know what the capacities of our
  services are in order determine our ability to
  provide for new and existing customers.

5
Example performance scenario 1

• Web service modelled as simple queuing network,
  infinite population.

• Risks Outages due to bursty traffic,
• under-provisioning.

6
Example performance scenario 2

• Web service with multiple servers modelled as
  finite queue with multiple servers, infinite
  population, fixed arrival rate.

• Risks Under- or over-provisioning.

7
What properties are of interest?

• Generic properties
• Response time time to process a single request.
• Throughput rate of processing.
• Utilisation of resources normally a percentage.
• Reliability expected time to failure.
• Availability percentage of time spent in an
  operational state.
• Specific properties SLA parameters.
• Different properties require different techniques.

8
Why is performance modelling important to TAPAS?

• It will allow application service providers to
  predict what service level agreements that they
  can enter into, and what agreements they require.
  This will increase the feasibility of the SLA
  approach, with all its concomitant benefits.
• Modelling will reduce development risk for these
  organisations and hasten time to market by
  reducing rework.

9
What is the state of the art In performance
modelling?

• Briefly consider
• Formalisms
• Methodologies
• Standards

10
Formalisms

• Markov Chains

• Queuing Networks

?
0
SCPU
?0,1
?2,0
?1,0
?2,0
CPU
?1,2
1
2
SD1
?2,1
Disk 1
11
More Formalisms

• Stochastic Timed Petri nets

• Stochastic Process Algebras
• CLIENT (think, thinkRate) . (request, T) .
  (return, T) . CLIENT
• SERVICE (request, requestRate) .
• (serviceA, serviceARate) .
• (serviceB, serviceBRate) .
• (serviceC, serviceCRate) .
• (return, returnRate) . SERVICE
• (CLIENT ltgt CLIENT ltgt CLIENT)
• lt request, return gt SERVICE

?Disk
?CPU
12
And what else

• Differential equations (control theory)
• Execution graphs
• Finite state machines (genebralised Markov
  chains)
• Generalised queuing theory, LQNs
• Probability theory, Bayesian networks
• Simulation

13
Methodologies

• Software Performance Engineering (Smith)
• Uses a combination of execution graphs and
  queuing networks.
• Advocates a repeated cycle of
• Modelling
• Implementation
• Monitoring
• Model validation and parameterisation.
• Advocates separate performance engineer role.

14
Standards UML Real-time

• Properly called UMLTM Profile for
  Schedulability, Performance, and Time
  Specification.
• Defines a domain model for performance analysis.
• Defines a set of UML stereotypes and tagged
  values for annotating designs with performance
  information.
• Relies on the UML action semantics.

15
What is the state of the art In system modelling

• Pertinent technologies are
• SOAP, WSDL, UDDI
• UML Profile for Enterprise Java Beans (JCP draft)
• Ultimately we will also consider modelling the
  network layer.

16
What are the problems that have to be addressed?

• The software performance engineering problem.
• The information hiding problem.
• Creating feasible models.
• Creating valid models.

17
The software performance engineering problem

• Why does the discipline Software Performance
  Engineering even exist? (Menasce)
• It is a cost/benefit problem.
• Increasing the benefit SLAs.
• Decreasing the cost Automation and integration
  Unobtrusive Performance Analysis.

18
The information hiding problem

• Middleware obscures the details of distribution.
• Application servers obscure the details of
  persistence and transactions.
• Interfaces obscure the details of functionality
  (polymorphism).
• Good from an engineering standpoint, but makes
  performance analysis much harder.

19
Creating feasible models

• E.g. state space explosion is a big problem.
  Consider the number of states in a system with
  concurrent processes and pre-emption
• n processes, each sliced up m ways leads to a
  state space of size mn
• My PC at the time of writing is running 47
  distinct processes concurrently, including
  database and webserver.
• E.g. the form of a queuing model can prohibit an
  exact solution, an analytical solution, or indeed
  any stable solution.

20
Creating valid models

• Feasible models are not guaranteed to accurately
  reflect the behaviour of the system under
  consideration.
• Model making requires possibly invalid
  assumptions.
• Formal models may be automatically generated and
  not inspected by modelling experts.
• An error margin of up to 30 may be considered
  acceptable in performance analysis (Menasce).

21
An example approach

• Using the EJB profile
• Model the application.
• Using the real-time profile
• Provide deployment information.
• Define the workload.
• Using a-priori knowledge of the architecture
• Refine the model to include container actions.
• Determine the resources and the resource demands.
• Create and solve a queuing network model.

22
Model the application
23
Model the deployment
24
Define a workload
The create user use-case
25
Add container actions
26
Define a queuing network model
Sthink
Workload derived from sequence
Client
Class link1 user client
A 2 1 1
Snet
link1Network
Stable
userTable
27
Deliver the results

• In this example the results will be throughput,
  response time, and utilisation for the two
  components modelled.
• The results will be delivered into the
  development environment somehow. Possibly they
  will be incorporated as an annotation to the
  model (UML real-time).
• Model parameters could be functions, resulting in
  a range of analyses, the results delivered as a
  graph.

28
Where were the numbers in this example?

• This example used very few specific quantities,
  such as resource demands.
• Performance characteristics differ by several
  orders of magnitude between local operations and
  remote operations.
• Similarly between operations requiring
  persistence and those not.
• This may lead to simplifications in the models.
• The specific timings may be irrelevant early on.
• Using fewer quantities simplifies the technique.

29
Integration with the SLA language

• The SLA language will associate components,
  assemblies or interfaces with performance
  constraints or capabilities.
• A tool could translate XML structures into UML
  structures and annotate them according to the
  performance profile.
• The modelling environment would check that the
  constraints were met and identify spare capacity.

30
Concluding remarks

• Perhaps federated services will provide the
  motivation for better performance analysis.
• Perhaps performance analysis techniques can be
  incorporated into existing modelling techniques
  and automated.
• This research is particularly useful in the
  specific context of TAPAS as it supports the
  construction of service level agreements.