CrossGrid: Interactive Applications, Tool Environment, New Grid Services, and Testbed

1
CrossGrid Interactive Applications, Tool
Environment, New Grid Services, and Testbed
Marian Bubak X TAT Institute of Computer
Science ACC CYFRONET AGH, Cracow,
Poland www.eu-crossgrid.org

2
Overview

• Applications and their requirements
• X architecture
• Tools for X applications development
• New grid services
• Structure of the X Project
• Status and future

3
CrossGrid in a Nutshell
Interactive and Data Intensive Applications

• Interactive simulation and visualization of
• a biomedical system
• Flooding crisis team support
• Distributed data analysis in HEP
• Weather forecast and air pollution modeling

Grid Application Programming Environment

• MPI code debugging and
• verification
• Metrics and benchmarks
• Interactive and semiautomatic
• performance evaluation tools

Grid Visualization Kernel
Data Mining
HLA
New CrossGrid Services
DataGrid ...

• Portals and roaming access
• Grid resource management
• Grid monitoring
• Optimization of data access

Services
Globus Middleware
Fabric
4
Biomedical Application

• Input 3-D model of arteries
• Simulation LB of blood flow
• Results in a virtual reality
• User analyses results in near real-time,
  interacts, changes the structure of arteries

5
VR-Interaction
6
Steering in the Biomedical Application
CT / MRI scan
Segmentation
Visualization
LB flow
simulation
Medical
Medical
HDB
VE
DB
DB
WD
PC
PDA
10 simulations/day 60 GB 20 MB/s
Interaction
7
Modules of the Biomedical Application

1. Medical scanners - data acquisition system
2. Software for segmentation to get 3-D images
3. Database with medical images and metadata
4. Blood flow simulator with interaction capability
5. History database
6. Visualization for several interactive 3-D
   platforms
7. Interactive measurement module
8. Interaction module
9. User interface for coupling visualization,
   simulation, steering

8
Interactive Steering in the Biomedical Application
CT / MRI scan
Segmentation
Visualization
LB flow
simulation
Medical
Medical
HDB
VE
DB
DB
WD
PC
PDA
Interaction
9
Biomedical Application Use Case (1/3)
CT / MRI scan
Segmentation
LB flow
simulation
Medical
Medical
DB
DB

• Obtaining an MRI scan for the patient
• Image segmentation (clear picture of important
  blood vessels, location of aneurisms and
  blockages)
• Generation of a computational mesh for a LB
  simulation
• Start of a simulation of normal blood flow in the
  vessels

10
Biomedical Application Use Case (2/3)

• Generation of alternative computational meshes
  (several bypass designs) based on results from
  the previous step
• Allocation of appropriate Grid resources (one
  cluster for each computational mesh)
• Initialization of the blood flow simulations for
  the bypasses
• The physician can monitor the progress of the
  simulations through his portal
• Automatic completion notification, (e.g. through
  SMS messages.

11
Biomedical Application Use Case (3/3)

• Online presentation of simulation results via a
  3D environment
• Adding small modifications to the proposed
  structure (i.e. changes in angles or positions)
• Immediate initiation of the resulting changes in
  the blood flow
• The progress of the simulation and the estimated
  time of convergence should be available for
  inspection.

Visualization
LB flow
simulation
VE
WD
PC
PDA
Interaction
12
Asynchronous Execution of Biomedical Application
13
Flooding Crisis Team Support
14
Cascade of Flood Simulations
Data sources
Meteorological simulations
Hydrological simulations
Users
Hydraulic simulations
Output visualization
15
Basic Characteristics of Flood Simulation

• Meteorological
• intensive simulation (1.5 h/simulation) maybe
  HPC
• large input/output data sets (50MB150MB /event)
• high availability of resources (24/365)
• Hydrological
• Parametric simulations - HTC
• Each sub-catchment may require different models
  (heterogeneous simulation)
• Hydraulic
• Many 1-D simulations - HTC
• 2-D hydraulic simulations need HPC

16
Váh River Pilot Site
Nosice
(Outflow point)
Strecno
(Inflow point)
Pilot Site Catchment Area 2500km2 (above
Strecno 5500km2)
Váh River Catchment Area 19700km2, 1/3 of
Slovakia
17
Typical Results - Flow and Water Depth
18
Distributed Data Analysis in HEP

• Objectives
• Distributed data access
• Distributed data mining techniques with neural
  networks
• Issues
• Typical interactive requests will run on o(TB)
  distributed data
• Transfer/replication times for the whole data of
  order of one hour
• Data transfers once and in advance of the
  interactive session.
• Allocation, installation and set up the
  corresponding database servers before the
  interactive session starts

19
Weather Forecast and Air Pollution Modeling

• Distributed/parallel codes on Grid
• Coupled Ocean/Atmosphere Mesoscale Prediction
  System
• STEM-II Air Pollution Code
• Integration of distributed databases
• Data mining applied to downscaling weather
  forecast

20
COAMPS Coupled Ocean/Atmosphere Mesoscale
Prediction System Atmospheric Components

• Complex Data Quality Control
• Analysis
• Multivariate Optimum Interpolation Analysis
  (MVOI) of Winds and Heights
• Univariate Analyses of Temperature and Moisture
• OI Analysis of Sea Surface Temperature
• Initialization
• Variational Hydrostatic Constraint on Analysis
  Increments
• Digital Filter
• Atmospheric Model
• Numerics Nonhydrostatic, Scheme C, Nested Grids,
  Sigma-z, Flexible Lateral BCs
• Physics PBL, Convection, Explicit Moist Physics,
  Radiation, Surface Layer
• Features
• Globally Relocatable (5 Map Projections)
• User-Defined Grid Resolutions, Dimensions, and
  Number of Nested Grids
• 6 or 12 Hour Incremental Data Assimilation Cycle
• Can be Used for Idealized or Real-Time
  Applications
• Single Configuration Managed System for All
  Applications
• Operational at FNMOC
• 7 Areas, Twice Daily, using 81/27/9 km or 81/27
  km grids

21
Air Pollution Model STEM-II

• Species 56 chemical, 16 long-lived, 40
  short-lived, 28 radicals (OH, HO2 )
• Chemical mechanisms
• 176 gas-phase reactions
• 31 aqueous-phase reactions.
• 12 aqueous-phase solution equilibria.
• Equations are integrated with locally 1-D finite
  element method (LOD-FEM)
• Transport equations are solved with
  Petrov-Crank-Nicolson-Galerkin (FEM)
• Chemistry mass transfer terms are integrated
  with semi-implicit Euler and pseudo-analytic
  methods

22
Key Features of X Applications

• Data
• Data generators and data bases geographically
  distributed
• Selected on demand
• Processing
• Needs large processing capacity both HPC HTC
• Interactive
• Presentation
• Complex data require versatile 3D visualisation
• Support interaction and feedback to other
  components

23
Overview of the CrossGrid Architecture
1.4 Meteo Pollution
1.3 Data Mining on Grid (NN)
1.3 Interactive Distributed Data Access
1.2 Flooding
1.1 BioMed
Applications
3.1 Portal Migrating Desktop
2.4 Performance Analysis
2.2 MPI Verification
2.3 Metrics and Benchmarks
Supporting Tools
Applications Development Support
MPICH-G
1.1, 1.2 HLA and others
App. Spec Services
1.1 Grid Visualisation Kernel
1.1 User Interaction Services
3.1 Roaming Access
3.2 Scheduling Agents
3.3 Grid Monitoring
3.4 Optimization of Grid Data Access
DataGrid Replica Manager
Globus Replica Manager
Generic Services
GRAM
GSI
Replica Catalog
GIS / MDS
GridFTP
Globus-IO
DataGrid Job Submission Service
Replica Catalog
Fabric
Resource Manager (CE)
Resource Manager
Resource Manager (SE)
Resource Manager
3.4 Optimization of Local Data Access
CPU
Secondary Storage
Instruments ( Satelites, Radars)
Tertiary Storage
24
Tool Environment
manual information transfer
25
MPI Verification

• A tool that verifies the correctness of parallel,
  distributed Grid applications using the MPI
  paradigm.
• To make end-user applications
• portable,
• reproducible,
• reliable on any platform of the Grid.
• The technical basis MPI profiling interface
  which allows a detailed analysis of the MPI
  application

26
Benchmark Categories

• Micro-benchmarks
• For identifying basic performance properties of
  Grid services, sites, and constellations
• To test a single performance aspect, through
  stress testing of a simple operation invoked in
  isolation
• The metrics captured represent computing power
  (flops), memory capacity and throughput, I/O
  performance, network ...
• Micro-kernels
• Stress-test several performance aspects of a
  system at once
• Generic HPC/HTC kernels, including general and
  often-used kernels in Grid environments
• Application kernels
• Characteristic of representative CG applications
• Capturing higher-level metrics, e.g. completion
  time, throughput, speedup.

27
Performance Measurement Tool G-PM

• Components
• performance measurement component (PMC),
• component for high level analysis (HLAC),
• component for performance prediction (PPC) based
  on analytical performance models of application
  kernels,
• user interface and visualization component UIVC.

28
For Interactive X Applications ...

• Resource allocation should be done in near-real
  time (a challenge for the resource broker
  scheduling agents).
• The resource reservation (i.e. by prioritizing
  jobs)
• Network bandwidth reservation (?)
• Near-real time synchronization between
  visualization and simulation should be achieved
  in both directions user to simulation and
  simulation to user (rollback etc)
• Fault tolerance
• Post-execution cleanup

29
User Interaction Service
user site
Service Factory
User Interaction
User Interaction
User Interaction
CM for Sim 1
Services
Services
Services
User Interaction
User Interaction
User Interaction
User Interaction
Services
Service
Services
Services
CM for Sim 2
CM
CM
CM
CM
CM for Sim 3
CM
Visualisation In VE
Running Simulation 3
Running Simulation 1
Running Simulation 2
CM
UIS connections
Pure module
UIS service
Other connections
Control module
30
Tools Environment and Grid Monitoring
Applications
Portals (3.1)
G-PM Performance Measurement Tools (2.4)
MPI Debugging and Verification (2.2)
Metrics and Benchmarks (2.4)
Grid Monitoring (3.3) (OCM-G, RGMA)
Application programming environment
requires information from the Grid about current
status of applications and it should be able to
manipulate them
31
Monitoring of Grid Applications

• Monitor obtain information on or manipulate
  target application
• e.g. read status of applications processes,
  suspend application, read / write memory, etc.
• Monitoring module needed by tools
• Debuggers
• Performance analyzers
• Visualizers
• ...

32
CrossGrid Monitoring System
33
Very Short Overview of OMIS

• Target system view
• hierarchical set of objects
• nodes, processes, threads
• For the Grid new objects sites
• objects identified by tokens, e.g. n_1, p_1, etc.
• Three types of services
• information services
• manipulation services
• event services

34
OMIS Services

• Information services
• obtain information on target system
• e.g. node_get_info obtain information on nodes
  in the target system
• Manipulation services
• perform manipulations on the target system
• e.g. thread_stop stop specified threads
• Event services
• detect events in the target system
• e.g. thread_started_libcall detect invocations
  of specified functions
• Information manipulation services actions

35
Components of OCM-G

• Service Managers
• one per site in the system
• permanent
• request distribution
• reply collection
• Local Monitors
• one per node, user pair
• transient (created or destroyed when needed)
• handle local objects
• actual execution of requests

36
Monitoring Environment

• OCM-G Components
• Service Managers
• Local Monitors
• Application processes
• Tool(s)
• External name service
• Component discovery

37
Security Issues

• OCM-G components handle multiple users, tools and
  applications
• possibility to issue a fake request (e.g., posing
  as a different user)
• authentication and authorization needed
• LMs are allowed for manipulations
• unauthorized user can do anything

38
Portals and Roaming Access
Applications
Portals (3.1)

• Remote Access Server
• user profiles
• authentication, authorization
• job submission
• Migrating Desktop
• Application portal

Roaming Access Server (3.1)
Scheduler (3.2)
GIS / MDS (Globus)
Grid Monitoring (3.3)

• Allow access user environment from remote
  computers
• Independent of the system version and hardware
• Run applications, manage data files, store
  personal settings

39
Optimization of Grid Data Access

• Different storage systems and applications
  requirements
• Optimization by selection of data handlers

Applications
Portals (3.1)
Optimization of Grid Data Access (3.4)
Scheduling Agents (3.2)

• Service consists of
• Component-expert system
• Data-access estimator
• GridFTP plugin

Replica Manager (DataGrid / Globus)
Grid Monitoring (3.3)
GridFTP
40
CrossGrid Collaboration
Ireland TCD Dublin
Poland Cyfronet INP Cracow PSNC Poznan ICM
IPJ Warsaw
Germany FZK Karlsruhe TUM Munich USTU Stuttgart
Netherlands UvA Amsterdam
Slovakia II SAS Bratislava
Austria U.Linz
Spain CSIC Santander Valencia RedIris UAB
Barcelona USC Santiago CESGA
Greece Algosystems Demo Athens AuTh Thessaloniki
Portugal LIP Lisbon
Italy DATAMAT
Cyprus UCY Nikosia
41
WP1 CrossGrid Application Development
Tasks 1.0 Co-ordination and management (Peter
M.A. Sloot, UvA) 1.1 Interactive simulation and
visualisation of a biomedical system (G.
Dick van Albada, Uva) 1.2 Flooding crisis team
support (Ladislav Hluchy, II SAS) 1.3
Distributed data analysis in HEP (C.
Martinez-Rivero, CSIC) 1.4 Weather forecast and
air pollution modelling (Bogumil Jakubiak, ICM)
42
WP2 - Grid Application Programming Environments
Tasks 2.0 Co-ordination and management (Holger
Marten, FZK) 2.1 Tools requirement definition
(Roland Wismueller, TUM) 2.2 MPI code debugging
and verification (Matthias Mueller, USTUTT) 2.3
Metrics and benchmarks (Marios Dikaiakos,
UCY) 2.4 Interactive and semiautomatic
performance evaluation tools (Wlodek
Funika, Cyfronet) 2.5 Integration, testing and
refinement (Roland Wismueller, TUM)
43
WP3 New Grid Services and Tools
Tasks 3.0 Co-ordination and management (Norbert
Meyer, PSNC) 3.1 Portals and roaming access
(Miroslaw Kupczyk, PSNC) 3.2 Grid resource
management (Miquel A. Senar, UAB) 3.3 Grid
monitoring (Brian Coghlan, TCD) 3.4 Optimisation
of data access (Jacek Kitowski, Cyfronet) 3.5
Tests and integration (Santiago Gonzalez, CSIC)
44
WP4 - International Testbed Organization

• Tasks
• 4.0 Coordination and management
• (Jesus Marco, CSIC, Santander)
• Coordination with WP1,2,3
• Collaborative tools
• Integration Team
• 4.1 Testbed setup incremental evolution
• (Rafael Marco, CSIC, Santander)
• Define installation
• Deploy testbed releases
• Trace security issues

• Testbed site responsibles
• CYFRONET (Krakow) A.Ozieblo
• ICM(Warsaw) W.Wislicki
• IPJ (Warsaw) K.Nawrocki
• UvA (Amsterdam) D.van Albada
• FZK (Karlsruhe) M.Kunze
• IISAS (Bratislava) J.Astalos
• PSNC(Poznan) P.Wolniewicz
• UCY (Cyprus) M.Dikaiakos
• TCD (Dublin) B.Coghlan
• CSIC (Santander/Valencia) S.Gonzalez
• UAB (Barcelona) G.Merino
• USC (Santiago) A.Gomez
• UAM (Madrid) J.del Peso
• Demo (Athenas) C.Markou
• AuTh (Thessaloniki) D.Sampsonidis
• LIP (Lisbon) J.Martins

45
WP4 - International Testbed Organization

• Tasks
• 4.2 Integration with DataGrid (Marcel Kunze,
  FZK)
• Coordination of testbed setup
• Exchange knowledge
• Participate in WP meetings
• 4.3 Infrastructure support (Josep Salt, CSIC,
  Valencia)
• Fabric management
• HelpDesk
• Provide Installation Kit
• Network support
• 4.4 Verification quality control (Jorge Gomes,
  LIP)
• Feedback
• Improve stability of the testbed

46
CrossGrid Testbed Map
Géant
TCD Dublin
PSNC Poznan
UvA Amsterdam
ICM IPJ Warsaw
FZK Karlsruhe
CYFRONET Cracow
CSIC-UC IFCA Santander
II SAS Bratislava
USC Santiago
LIP Lisbon
Auth Thessaloniki
UAB Barcelona
CSIC RedIris Madrid
CSIC IFIC Valencia
UCY Nikosia
DEMO Athens
47
WP5 Project Management
Tasks 5.1 Project coordination and
administration (Michal Turala, INP) 5.2
CrossGrid Architecture Team (Marian Bubak,
Cyfronet) 5.3 Central dissemination (Yannis
Perros, ALGO)
48
EU Funded Grid Project Space (Kyriakos
Baxevanidis)
49
Project Phases
M 4 - 12 first development phase design, 1st
prototypes, refinement of requirements
M 25 -32 third development phase complete
integration, final code versions
M 33 -36 final phase demonstration and
documentation
M 1 - 3 requirements definition and merging
M 13 -24 second development phase integration
of components, 2nd prototypes
50
Rules for X SW Development

• Iterative improvement
• development, testing on testbed, evaluation,
  improvement
• Modularity
• Open source approach
• SW well documented
• Collaboration with other projects

51
Collaboration with other Projects

• Objective exchange of
• information
• software components
• Partners
• DataGrid
• DataTag
• Others from GRIDSTART (of course, with GridLab)
• Participation in GGF

52
Status after M6

• Software Requirements Specifications together
  with use cases
• CrossGrid Architecture defined
• Detailed Design documents for tools and the new
  Grid services (OO approach, UML)
• Analysis of security issues and the first
  proposal of solutions
• Detailed description of the test and integration
  procedures
• Testbed first experience
• Sites LIP, FZK, CSICUSC, PSNC, AuTHDemo
• Basic EDG release 1.2
• Applications
• EDG HEP simulations (Atlas,CMS)
• first distributed prototypes using MPI
• NN distributed training
• Evolutionary Algorithms

53
Near Future

• Participation in production testbed with DataGrid
• All sites will be ready to join by end of
  September
• Common DEMO at IST 2002, Copenhagen, November
  4th-6th
• Collaboration with DataGrid in specific points
  (e.g. user support and helpdesk software)
• CrossGrid Workshop, Linz (w/ EuroPVM/MPI 2002),
  September 28th-29th
• Conference Across Grids together with RI Forum
• Santiago de Compostella, Spain, February
  9th-14th,2003
• With Proceedings (reviewed papers)

54
Linz CrossGrid Workshop Sep 28th-29th

• Evaluate the current status of all tasks
• Discuss interfaces and functionality
• Understand what we may expect as first prototypes
• Coordinate the operation of the X testbed
• Agree about common rules for software development
  (SOP)
• Start to organize the first CrossGrid EU review
• Meet with EU DataGrid representatives
• Discuss the technology for the future (OGSA)
• Details at
• http//www.gup.uni-linz.ac.at/cros
  sgrid/workshop/

55
Summary

• Layered structure of the all X applications
• Reuse of SW from DataGrid and other projects
• Globus as the bottom layer of the middleware
• Heterogeneous computer and storage systems
• Distributed development and testing of SW
• 12 partners in applications
• 14 partners in middleware
• 15 partners in testbeds
• In total 21 partnes
• First 6 months successful

56
Thanks to

• Michal Turala
• Kasia Zajac
• Maciek Malawski
• Marek Garbacz
• Peter M.A. Sloot
• Roland Wismueller

• Wlodek Funika
• Ladislav Hluchy
• Bartosz Balis
• Jacek Kitowski
• Norbert Meyer
• Jesus Marco
• Marcel Kunze

57
www.eu-crossgrid.org