GRID monitoring

1
GRID monitoring

• Gennaro Tortone (INFN Napoli) gennaro.tortone_at_na
  .infn.it

GRID.it meeting Bologna 14.2.2003
2
Summary

• monitoring of Grid elements
• GLUE schema
• EDG-WP4 monitoring framework
• tasks
• future activities

3
Monitoring of grid elements (1/2)
Computing Element
Resource Broker
Storage Element
Worker Node
Worker Node
Worker Node
Information Index
Replica Catalog
Worker Node
Replica Manager

• LOW LEVEL measurements
• CPU load
• memory usage
• disk usage (per partition)
• network activity
• number of processes
• number of users (UI)

• SERVICE checks
• gatekeeper
• gsiftp
• gris
• gdmp
• RB/LB

• GRID measurements
• number of total CPUs
• number of free CPUs
• number of running jobs
• number of waiting jobs
• SE free disk space

4
Monitoring of grid elements (2/2)

• sources of information
• LOW LEVEL measurements -gt plugins/sensors
  installed on each machine
• SERVICE checks -gt sensors installed on monitoring
  server
• GRID measurements -gt sensors installed on
  monitoring server
• aggregate information
• per VO
• per site

5
GLUE schema

• Conceptual model of grid resources to be used as
  a base schema of the GIS(Grid Information
  Service) for discovery and monitoring purposes
• model of computing resources (CE)
• model of storage resources (SE)
• model of relationships among them (close CE/SE)
• Implementation status (v. 1.0) (for Globus MDS)
• LDAP schema (DataTAG WP4.1)
• information providers (CE/SE)
• GLUE schema extension to include all monitoring
  metricsdone host level added to GLUE schema

6
(No Transcript)
7
EDG-WP4 monitoring framework

• It provides a client (Monitoring Sensor Agent -
  MSA) running sensors (Monitoring Sensors - MS) on
  each node to monitor, and a central server
  (Fabric Monitoring Server - fmonServer) to
  collect data.
• The server receives samples as they are measured
  by MSA, and stores them in a flat file / Oracle
  database
• The client is provided with a sensor
  (sensorLinuxProc) which uses /proc file system to
  measure various basic quantities on Linux (CPU
  load, network, etc).

8
EDG-WP4 monitoring framework
local farm element
computing element
9
ldap query
information index
ldap query
monitoring server
write
run
ldif output
farm monitoringarchive
read
computing element
10
Tasks (1/3)

• identify the requirements for Grid monitoring
• done Grid monitoring analysis draft with some
  LCG inputs(available on http//gridmon.na.infn.i
  t/lcg-edt)
• evaluation of existing monitoring tools (sensors)
  to use as first monitoring layer on each
  grid-element
• done tools evaluated
• EDG-WP4 fabric-monitoring tool (fmon)
• client-server model
• very easy to use
• very easy to install (one RPM without
  dependencies)
• highly customizable (time interval for each
  metric, )
• it is very easy to add a new metric
• historical archive
• database in Oracle/plain-text format

11
Tasks (2/3)

• extension of the WP4 fabric-monitoring tool
  (fmon) to include other monitoring metrics
• done (all metrics added are available on
  http//gridmon.na.infn.it/lcg-edt)
• GLUE schema extension to include all monitoring
  metrics
• done host level added to GLUE schema
• development of information-providers to fill
  the GLUE host level extension done
• definition of database structure to store
  snapshot/historical monitoring data done

12
Tasks (3/3)

• automatic resource discovery using MDS
  infrastructure and GLUE schema in progress
• development of a web interface to display various
  grid-views (per VO, per site, etc.) in
  progress

13
Future activities

• personal Grid-monitoring integration with
  VOMS
• job monitoring
• evaluation of OGSA as monitoring service
• development of a Grid monitoring tool
• scalability
• very low intrusivity
• automatic resource discovery
• fault detection and notification
• metrics graphs
• web interface

14
(No Transcript)
15
(No Transcript)