Performance Prediction of Parallel Applications based on Historical Data

1
Performance Prediction of Parallel Applications
based on Historical Data

• S D Penurkar
• Jay Yagnik

2
Topics Covered

• What is performance prediction and why we require
  it?
• Approaches to performance prediction
• Performance contract approach
• Prophesy approach
• Our ideas
• References

3
Performance Prediction Why?

• Performance is the one which drives use of
  Parallel Applications (Vs sequential)
• Since grids are basically widely distributed
  collection of computers, there is
  unpredictability of resource avaibility and their
  performance . Hence certain guarantee of stated
  performance is required.

4
Performance PredictionTypes and Parameters

• Types-
• Compute intensive applications
• Data intensive applications
• Parameters-
• No of computations
• Communication delays
• Processor load

5
Performance Prediction Approaches

• Model based
• Prediction based on historical data

6
Performance Prediction Model Based

• Mapping of application on to the computing
  resources (network and processors)
• Should include aspects of uncertainity inherent
  in grid applications
• Dynamic monitoring of resource performance
• Models help us to choose the mapping to produce
  best throughput of the application

7
Performance Prediction Model based

• Trace from actual run
• Emulator
• Simulator

8
Performance Prediction Model based

• Trace obtained from single instance of
  application and machine configuration
• Static and can not reflect dynamic nature
• Emulator is a program which emulates the
  application, hence dyanamic.
• Simulator models the subsystems of parallel
  machine at sufficient level of accuracy

9
Performance predictionComparision

• Analytical / Statistical
• Emulator
• Simulator
• Higher accuracy and more expensive as you
  move from top to bottom.
• More manual intervention down the line

10
Performance Prediction Performance Contracts

• Evaluation of expected performance in different
  load conditions
• 3 steps
• Application signature
• Executation signature
• Signature match and fuzzy membership

11
Performance ContractsApplication Signature

• Application intrinsic metrics
• solely depends on the application code and
  problem parameters
• expresses demands that application places on the
  resources, independent of execution environment
• Ex msg/ byte, flops
• Multidimentional with each application metric

12
Performance ContractsExecution Signature

• It reflects both the application demands on the
  resources and response of resources to those
  demands
• Eg. instr/second , msg/sec

13
Performance Contracts Signature Match

• Project the application signature in execution
  metric space using scaling factor for each
  dimension and analyse

14
Performance prediction Prophesy Framework

• Main goal to automatically generate performance
  models to aid in performance analysis
  evaluation of given application
• Data collection- using profiling and
  instrumentation
• Central databases- to collect performance info
  generated during execution, to store model
  templates and to store system charatreistics
• Data analysis
• model builder produces analytical performance
  models
• Symbolic prediction

15
Performance Prediction Prophesy

• Uses 3 methods for performance prediction
• Curve Fitting
• Parameterization
• Composition method (analysis of kernel
  interactions)

16
Performance prediction Prophesy Modelling

• Curve fitting
• Constructing functions whose graphs are the best
  approx to collected emperical data points. Eg.
  Linear, Exponential or polynomials
• It uses only the emperical data
• It uses the least square fit when performing the
  fit.

17
Performance Prediction Our Approach and
Required Infrastructure

• Use of Maximum available machines(20)
• Use standard benchmark scaLAPACK application
  (eigen value)
• Use of standard available profiling tool to
  collect data ( PAPI)
• Analysis of collected data and build prediction
  model

18
Performance Prediction Some Ideas

• Use of uncertainty information in prediction
• Eg. Prediction Variance in Gaussian Process based
  prediction
• Selecting the curve type based on the observed
  data points, and some criteria
• Support Vector Regression may be useful

19
Performance Prediction References

• Valerie Taylor, Xingfu Wu, and Rick Stevens,
  PROPHESY A Web-based Performance Analysis and
  Modelling System for Parallel and Distributed
  Applications, Performance TOOLS 2003 (Tool
  demonstrations), Urbana, Illinois, September 2-5,
  2003.
• F. Vraalsen, R.A. Aydt, C.L. Mendes, and D.A.
  Reed,Performance Contracts Predicting and
  Monitoring Grid Application Behaviour, Grid
  Computing - GRID 2001  Proceedings of the 2nd
  International Workshop on Grid Computing, Denver,
  CO,  November 12, 2001,Springer-Verlag

20
Performance Prediction

• We believe that the most important open
  question in performance today is how to assess
  and verify the accuracy of performance models.
  Without the means to assure the accuracy of
  models , it is difficult to put them in
  production use
• ---------------- Toledo

21
Thank You