Interactive European Grid

1
Interactive European Grid

• Grid Job Management and Control in Earth Science
• DEGREE Workshop
• Bratislava 1-2 March 2007
• Prepared by Isabel Campos (CSIC), Branislav Simo
  (UISAV)
• and Jesús Marco (CSIC) Instituto de Física de
  Cantabria SPAIN

2
Interactive European Grid (http//www.interactive-
grid.eu)
Project acronym int.eu.grid Contract
number 031857 Instrument I3 Duration 2
years may 06-april 08
KEEPING EGEE INTEROPERABILITY!
providing transparently the researchers
desktop with the power of a supercomputer,
using distributed resources http//www.interact
ive-grid.eu
Coordinator CSIC, Jesús Marco, IFCA, Santander,
SPAIN marco_at_ifca.unican.es
User Friendly Access Grid Interactive Desktop
Distributed Parallel (MPI) Interactive Computing
Storage at the Tera level
3
Key points

• Established interest from research communities
• Fusion, medicine, environment, HEP, astrophysics
• MPI Interactivity support integration
• MPI
• Open MPI (incl. Infiniband support)
• PACX-MPI
• Collaboration with EGEE (Dublin meeting)
• Interactivity
• SDJ collaboration/consideration of requirements
• Architecture
• Validation and Deployment
• CrossBroker
• MD/RAS
• Infrastructure deployment
• Resources clusters storage
• Security
• Network support
• AccessGrid integration/availability/usage
• Prepare for further dissemination to research
  communities

4
Where are we?
5
Background of int.eu.grid CROSSGRID

• Crossgrid EU 5FP Project 2002-2005
• Challenge demonstrate that interactive and
  parallel applications can execute in a Grid
  framework interoperable with EDG/EGEE
• International testbed to support the development
  of middleware oriented to interactivity and MPI
• Partners of CrossGrid joined to propose
  int.eu.grid

6
CrossGrid developed middleware

• Development of a user friendly graphical
  interface to the GRID
• MIGRATING DESKTOP
• Implementation of a Grid scheduler
• Able to execute interactive jobs
• Submission of MPI jobs (mpich and mpich-g2)
• Supports the interaction of the user with the job
  at execution time
• Prototype applications were tested
• Flooding
• High Energy Physics
• Medicine
• Meteorology/Air Pollution

7
Beyond CrossGrid the challenges of int.eu.grid

• From the Infrastructure point of view
• Operate a production level infrastructure 24/7
• Support Virtual Organizations at all levels
• Running the VO (user support)

• From the middleware point of view
• Parallel Computing (MPI)
• Support intracluster Jobs with OpenMPI
• Support intercluster Jobs with PACX-MPI
• Advanced visualization tools allowing simulation
  steering
• GVid, glogin
• A Job scheduler that supports it all
• User friendly interface to the grid supporting
  all this features
• Integrating in the Migrating Desktop all the
  features

Create an interoperable production infrastructure
supporting demanding interactive applications at
all levels

• From the Applications point of view
• Analyze requirements of reference applications
• Ensure that middleware copes the reference
  applications demands
• Application Porting Support
• Promote collaborative environments like
  AccessGrid

8
This is our infrastructure
9
Prototype Application

• Description
• Visualize the behaviour of plasma in fusion
  devices
• From the computational point of view
• The plasma is analyzed like a system of NP
  particles
• Every particle interacts with the magnetic field
  created by the device (Stellerator) and with the
  total background field created by the rest of the
  particles
• The NP particles are distributed among a number
  of processors N
• Each CPU computes the trajectory of the particles
  it has been assigned (independent calculation)
• Each particle stores its position and velocity at
  every step
• That information is used for visualization
  purposes
• Graphical Interface design with OpenGL
• Towards a Virtual Interactive Session ( couple
  of hours) gathering a large amount of resources
  instantly.

10
Working Schema
Video Streaming
11
There is pattern of workUltra Sound Computer
Tomography
12
All requirements for Middleware

• Distributing the NP particles among N processors
• MPI support
• The Job should be started inmediately on the user
  
• desktop
• MPI Interactive job scheduling
• The graphical interface should be forwarded to
  the user desktop
• Graphical interface to the grid ? Migrating
  Desktop
• Supporting Visualization ? GVid
• The user should be able to steer the simulation
• Real Time steering ? glogin

13
Click on movie interactive steering
14
IMS Model Suite

• Environmental Dispersion Modeling
• Lagrangian Trajectory Model
• Complex software system for prediction of the
  consequences of environmental pollution, nuclear
  accident or other emergence

• Numerical modeling of phenomena
• Continuous or short-term releases of multiple
  simultaneous pollutant species
• Dispersion of multiple pollutants
• Atmospheric transport and diffusion under
  changing weather conditions
• Deposition (wet, dry)
• Radioactive decay
• Pre-defined scenarios (Examples industrial
  leakages and emissions in Slovakian towns,
  nuclear accident at reactor in France, pollutant
  releases at the major Saudi towns. Provides
  pollution assessment for regulation and quick
  help for decision makers in case of real
  accident)

15
IMS Model SuiteMass Concentration 33h after
Release Bordeaux
T 33h ng/m3
Volumetric concen- tration in ng/m3
16
Flood forecasting problem
Data sources
Data sources
Meteorological Simulation
Hydrological Simulation
Hydraulic Simulation
Portal
Portal
17
Flood forecasting - Models used

• MM5 (meteorological model)
• limited-area, non hydrostatic, terrain-following
  sigma-coordinate model designed to simulate or
  predict mesoscale atmospheric circulation.
• Input data boundary conditions
• Output data quantitative precipitation forecast,
  temperature, humidity, wind field, etc.
• Parallel MPI application
• HSPF (hydrological model)
• simulates for extended periods of time the
  hydrologic, and associated water quality,
  processes on pervious and impervious land
  surfaces and in streams and well-mixed
  impoundments
• Input data quantitative precipitation,
  temperature, topographical data
• Output data inflow, hydrograph
• Sequential application
• DaveF (hydraulic model)
• a time-explicit finite-volume model
• best suitable for unsteady state with critical or
  super-critical flow (dam-breaking, flash flood,
  flood with wetting/drying in large expanses)
• Input data inflow, topographical data
• Output data water levels and velocities
• Parallel MPI application

18
MPI Support

• Why MPI Support ?
• The standard API for distributed memory
  parallelization
• Write once, run everywhere
• This is what applications are

There are many issues about handling MPI jobs
types already worked out for Linux Clusters,
Supercomputers, etc which have to be addressed
when running MPI on the Grid in a particular
way.

• What is MPI
• Is an API
• Description of the semantics, but
• NOT the implementation
• Almost platform independent
• (modulo problems with MPI-IO)

• What is NOT MPI
• There is no implementation
• No specification of how to start
• the processes
• How to the get the binary on the remote sites
• How to start the binaries on the remote sites
  (ssh, PBS,)

19
Problems of MPI Support on the Grid

• There is no standard way how to start a MPI
  program
• No common Syntax for mpirun
• MPI-2 defines mpiexec as starting mechanism, but
  support for mpiexec is only optional
• Resource Brokers should handle different MPI
  implementations
• Different Schedulers and different MPI
  implementations at each site have different ways
  to specify the machine file

• Non-shared filesystems (Oh!)
• Many Grid sites dont have support for a shared
  home directory
• Many MPI implementations expect that the
  executable is available in the nodes where the
  process is started
• Mixed setup in general some sites have shared
  file systems, some not

20
Problems of MPI Support on the Grid

• Our Solution an intermediate layer
• mpi-start

RESOURCE BROKER
MPI-START
MPI Implement.
Scheduler
21
mpi-start

• Goals
• Hide differences between MPI implementations
• Hide differences between local schedulers
  implementations
• Supports simple file distribution
• ? Hides from the user the filesystem details
• (shared or non-shared)
• Providing a simple but powerful enough unique
• interface for the Resource Broker to specify MPI
  Jobs
• ?The Resource Broker does not have to contain
• hardcoded the MPI support

22
mpi-start

• mpi-start design

mpi-start
portable (bash scripting)
schedulers
MPI
hooks
PBS
SGE
filesystem
Openmpi
PACX-MPI
MPICH
23
MPI Support in Grid Environments

• In Grid Environments there are two possible cases
• Intra Cluster Jobs
• All processes run on the same cluster
• Inter Cluster Jobs
• Processes are distributed across several
  clusters/sites

1
2
SIZE
. . .
P2P
MPI_COMM_WORLD
Collective Communication
24
MPI Support in Grid Environments

• For IntraCluster Jobs we support OpenMPI

• RTE environments SSH, PBS, SGE, Slurm,
• OS Linux, Solaris, MacOS
• Tested gcc, icc, ifc, pgcc, pathc

25
MPI Support in Grid Environments

• For IntraCluster Jobs we support OpenMPI

MPI-Layer
PML
BML
OpenIB
GM
SM
TCP

• Very attractive modular network components for
• Heterogeneous environments as is the Grid

26
MPI Support in Grid Environments

• For Inter Cluster Jobs we support PACX-MPI

• A middleware for seamlessly run a MPI-application
  on a network of parallel computers
• originally dev. in 1995 to connect VectorMPP
• PACX-MPI is an optimized standard-conforming MPI-
  implementation, application just needs to be
  recompiled(!)
• PACX-MPI uses locally installed, optimized vendor
  implementations for cluster inter communication

27
PACX MPI Design

• PACX-MPI start a MPI job in each cluster
• PACX-MPI merge/manage these MPI jobs internally
  and emulate transparently a bigger MPI job to the
  application

Application
PACX-MPI (job)
Open MPI (job)
Open MPI (job)
Cluster 2
Cluster 1
28
MPI Support on the Grid

• A grid site has in general the following topology
• CE Computing Element (head node) public IP
• WN Worker Nodes, private IP

• Requirements
• Connectivity of CE to the clusters and start-up
  daemons
• Files Application Input files
• Start on daemons on the CE. Connectivity of ssh
  to CE

A MPI Job requesting N processes per cluster
spawns N2 processes, Two of them in the CE
running as Daemons, making the bridge between
clusters
CE
WN
29
PACX MPI Design

• Internal Communication
• communication between processes that resides in
  the same cluster is performed via the local
  OpenMPI

CE
CE
30
PACX MPI Design

• External Communication
• Handled via the Computing Element, the only one
  with public IP
• TCP/IP daemons do the job

31
Middleware for Visualization Steering

• Glogin
• Lightweight tool for support of interactivity on
  the grid
• Grid authenticated shell access glogin host
• No dedicated daemon needed such as sshd
• TCP Port Forwarding enables access to grid worker
  nodes with private IPs.
• X11 Forwarding

• GVid
• Grid Video Service
• Visualization can be executed remotely on a grid
  resource
• Transmits the visualization output to the user
  desktop
• Communication of the interaction events back to
  the remote rendering machine
• Uses Glogin as bi-directional communication
  channel

32
Glogin Traffic Forwarding
33
Gvid and Glogin working together
USER
GRID
Steering Visualization
Application Plugin
MPI MASTER
Java Gvid Decoder
Gvid Encoder
Event Interception
Key, mouse
Event reception
GUI
Migrating Desktop
CE
Interactive GSS tunnel
glogin
RAS
glogin
https
34
Summarizing
35
The Migrating Desktop

• Migrating Desktop/Roaming Access Server
• User friendly interface for grid users
• Developed as a Java based GUI
• Independent of any platform (MS Windows, Linux,
  Solaris,)
• Simplifies data management, job submission and
  monitoring
• Glogin has been integrated in
• Gvid specific Plugin has been developed

server
client
36
The Migrating Desktop at work
37
Bringing everything together
Roaming Access Server
CrossBroker
Job Submission Services
Migrating Desktop
gLogin
Gatekeeper
LRMS
Logging Bookkeeping
WorkerNode
Computing Element
WorkerNode
010011000
Application GVid

vtk App
SE
Simul. Res. 30GB
38
My Panel contribution

• All this GRID technology is very interesting but
• Does this works now and will be your tool for
  the future? YES
• Do we need a sustainable model for all this
  effort? YES
• So, how to proceed?
• Infrastructure
• KEEP EGEE INTEROPERABILITY
• Consolidate (towards EGI?)
• APPLICATIONS/RESEARCH COMMUNITIES
• Initiatives like DEGREE are KEY
• We need feedback, challenges, share experiences,
  discussion motivation
• We need a business model
• Will we use National Grid Initiatives as the way
  to go?
• Is this enough/adequate? Are our collaborations
  national or international?
• Would we support from our national projects the
  use of an e-Infrastructure?
• We request this support for the network for
  example
• Do we need an extra support to integrate
  applications at European level?
• Via new projects? In application fields? In ICT?
• More consolidated?
• Through specialized research centers?

39
Thanks and Keep in touch!

• Have a look to our web page
• http//www.interactive-grid.eu
• Try our infrastructure!
• Send us your interest through the questionnaire
• Paper copy
• On-line
• Contact by e-mail
• marco at ifca.unican.es (project coordinator)
• iscampos at ifca.unican.es (application
  coordinator)
• Come to Santander and visit us at IFCA!
• Do Surf (where others come to do research!)