Evaluating the Parallel Performance of a Heterogeneous System

1
Evaluating the Parallel Performance of a
Heterogeneous System

• Elizabeth Post
• Hendrik Goosen
• formerly of
• Department of Computer Science
• University of Cape Town, South Africa

2
Introduction

• Why measure and evaluate parallel performance?
• What to measure
• How to evaluate
• Why speedup and efficiency are not appropriate
• Alternate methods of performance evaluation
• Power weight
• Linear speed
• Linear efficiency
• Conclusions

3
Why measure parallel performance?

• Increasing use of parallel processing
• Clusters, such as Beowulfs
• Networks of Workstations (NOWs)
• Scalability
• Does performance improve as more processors are
  added?
• Will performance continue to improve as more
  processors are added?
• Efficiency
• Is the best possible performance being achieved?
• Where and how can performance be improved?
• Is the best algorithm being used?

4
What makes systems heterogeneous?

• Different architectures
• Different memory and cache capabilities
• Different operating systems
• Communications
• Non-dedicated systems

5
Why use a real application to measure performance?

• Real users want to know how long real
  applications take
• Measures such as
• MIPS
• MFLOPS
• Kernels
• Vendor-tuned benchmarks
• do not take into account factors such as
• input/output,
• communication time,
• memory needs and usage,
• idle time etc.

6
Background

• Climatology application
• Cloud radiation simulation to model and measure
• reflectivity,
• transmissivity, and
• absorptivity
• of a heterogeneous strato-cumulus cloud deck.
• Environment
• Network of Unix workstations,
• five different models of Silicon Graphics and Sun
  workstations
• with varying CPU performance and memory capacity,
  
• connected by a 10Mbit Ethernet network.

7
Serial times for processorsCPU, System and
Elapsed times
8
Parallel performance -What to measure?

• Elapsed time
• CPU time
• System time
• Communication time
• Idle time

9
Parallel performance -How to evaluate?

• Elapsed time graphs
• Speedup
• Efficiency
• Power weight
• Linear speed
• Linear efficiency

10
Heterogeneous Parallel groupCPU, System and
Elapsed times
11
Speedup

• Speedup is
• the ratio of the serial time taken on one
  processor compared to the parallel time on all
  processors
• Elapsed time on 1 processor
• Elapsed time on n processors

12
Speedup how to calculate it?

• Which single processor elapsed time should be
  used?
• Elapsed time of fastest processor?
• Elapsed time of slowest processor?
• Mean elapsed time of all processors used?

13
Speedup vs Perfect Speedup(calculated with means)
14
Efficiency

• Efficiency is
• the ratio of the speedup divided by the number
  of processors used
• Speedup for n processors
• n

15
Efficiency(calculated with means)
16
Speedup and efficiency not appropriate

• Conclusion that speedup and efficiency are not
  appropriate for evaluating parallel performance
  on a heterogeneous system, and even have their
  limitations on a homogeneous system.

17
Power weight(Zhang et al)

• Power weight is
• ratio of performance of each processor as
  compared to performance of the fastest processor
• Elapsed time for fastest processor
• Elapsed time for nth processor

18
Linear Speed (Crowl)

• Linear Speed is
• the amount of work done in unit time
• 1 Total elapsed time

19
Linear Speed
20
Linear Efficiency (Post and Goosen)

• Linear Efficiency is
• ratio of work done in parallel application
  compared to potential amount of work that could
  be done by all processors
• Linear speed for parallel application on n
  processorsSum of single processor linear speeds
  for n processors

21
Linear Efficiency
22
Advantages of linear speed and linear efficiency

• Linear speed for each machine is independent of
  all other machines
• Linear speeds can be recalculated dynamically as
  workloads vary and used to determine each
  machines current performance compared to its
  serial capacity.
• Dynamic calculation of linear speeds and linear
  efficiency can be used for dynamic load-balancing
  algorithms.

23
Conclusions

• Important to measure overall elapsed time, as
  well as components of CPU, communication, idle
  time etc.
• Speedup and efficiency are not appropriate for
  evaluating parallel performance, especially for
  heterogeneous systems
• Linear speed and linear efficiency provide useful
  ways of evaluating parallel performance for both
  heterogeneous and homogeneous systems.
• Linear speed and linear efficiency can be
  calculated dynamically and be used in dynamic
  load balancing algorithms.

24
(No Transcript)