Design and Implementation of GridBench

1
Design and Implementation of GridBench
Marios Dikaiakos, George Tsouloupas mdd,georget
_at_ucy.ac.cy High Performance Computer systems
Laboratory University of CyprusSupported in
part by the European Union through the CrossGrid
Project (IST-2001-32243)

2
Goal and Motivation

• Provide a tool for the performance evaluation of
  Computational Grid Resources.
• End-users
• Ease of use was a primary concern.
• Perform resource selection based on site
  micro-benchmarks or application computational
  kernel performance
• Evaluate the functional status of Grid Resources
• VO/site Administrators
• Evaluate the functional status/performance of
  Grid sites belonging to a VO.
• Easily perform end-to-end on-demand tests from an
  intuitive user interface.
• Schedulers

3
GridBench The Benchmark Suite

• Layered approach
• Worker-node
• Site
• and VO level
• Micro-benchmarks
• Micro-kernel Benchmarks
• Application-kernel Benchmarks

4
GridBench Key Components

• Benchmarks
• User Interface
• Orchestrator
• Archiver
• Middleware Plugin
• Monitoring Clients

5
The GridBench Definition Language

• Define benchmark executions (in XML)
• benchmark
• parameters
• Optionally define
• monitoring
• Definition of workflow benchmarks.
• After execution the document includes
• metrics

6
The GridBench Definition Language

• "type"mpi" name"cluster.ui.sav.sk" cpucount"16"
  wncount"16"
  jobmanager"jobmanager-pbs-workq"/
  type"system" dataType"0"epwhets
  tone type"system"
  dataType"0"/opt/cg/gb/bin
  type"system" dataType"0"manualeter dataType"1"10000 name"mips" unit"whetstone_mips" name"hostname"n15 n1 n2 n3
  690 692 692 691
  

7
Trusting measurements

• Result filtering through monitoring
• Pre-defined and user-customizable
• Support for easy integration of existing
  monitoring tools
• Currently supports the CrossGrid JIMS monitoring
  tool.
• Monitoring data is encapsulated in the GBDL
  document

8
Web-services Orchestrator

• Manage Benchmark execution
• Invokes Middleware plugins

9
Web-services Orchestrator

• Simple Interface
• GDBL input, GBDL (including metrics) output.
• Interactive execution or fire-and-forget.
• Results automatically archived
• Middleware plugins
• Carry the burden of interfacing to the middleware
• Plugin implementations for
• Globus RSL generation, direct job execution
• CrossGrid/EDG/LCG JDL generation, job execution
  via Resource Broker

10
Web-services Archiver

• Archiver
• Store/Retrieve benchmark results
• Transformation of XMLRelational data

11
GridBench User Interface

• Easy benchmark invocation (drag-n-drop)
• Benchmark Models/Templates
• real-time Infrastructure status information
• Interactive benchmarking environment

12
GridBench User Interface

• Execution monitoring
• Result Browsing
• Query and analyse results
• Analysis tools
• Charting
• result comparison

13
Sample results
MemoryBandwidth
CPUPerformance
14
Sample results
MPIPerformance
Application-kernel Performance
15
Summary

• A portable set of benchmarks to characterize the
  performance of a computational Grid resource
• Mechanisms for the easy execution of benchmarks,
  and management of the experiments.
• Mechanisms for the easy inclusion of new
  benchmarks.
• A portable user-space tool that is already in use
  by other Grid projects with minimal poring
  effort. (Grid-Ireland)

16
Future Work

• More Benchmarks
• Availability
• Performability
• Derivation of higher-level metrics for quality
  features of Grid resources
• homogeneity
• GIS trustworthiness
• Infrastructure health
• Automated/periodic execution
• automatic benchmark parameter selection
• Additional Middleware plugins.

17
Thanks!Questions?more information
athttp//grid.ucy.ac.cy/GridBench
18
New benchmark integrationcase-study

• Integrating an application kernel into GridBench
• 1.Write a Model GBDL
• model"true"
• bstre
  am1.1
• 40rameter
• 20meter
• tube38x40
  x40
• tube3
  8x40x40.bs

19
New benchmark integrationcase-study

• 2. Create wrapper script/ Instrument the code to
  output metrics
• step"20" period"200" name"time"0.079617 0.079529 0.079511 0.079498
  ... 0.094326
• unit"s"639.633215

20
New benchmark integrationcase-study

• 3. Add to the database
• 4. Use the GUI Run benchmarks.