Title: Introduction to Grid Technologies in EGEE
1Introduction to Grid Technologies in EGEE
- Emanouil Atanassov,
- Aneta Karaivanova and Todor Gurov
- Institute for Parallel Processing - BAS
2Overview
- Evolvement of Grids
- What is Grid?
- Grid Services
- Goals of the EGEE project
- Building a production Grid for e-Science
- Grid applications in EGEE and SEE-GRID
- The Grid Challenges
3Evolvement of Grids
- Historical perspective
- Local Computing
- All computing resources at single site.
- People move to resources to work.
- Remote Computing
- Resources accessible from distance.
- All significant resources still centralized.
- Distributed Computing
- Resources geographically distributed.
- Specialized access largely data transfers.
- Grid Computing
- Resources and services geographically
distributed. - Standard interfaces transfers of computations
and data. - Web Services and Grid Computing Grid Services
- Industry adopts Grid technology
4What is GRID?
- Coordinated resource sharing and problem solving
in dynamic, multi-institutional virtual
organizations (I.Foster) - Resources are controlled by their owners
- The Grid infrastructure provides access to
collaborators - A Virtual Organization is
- People from different institutions working to
solve a common goal - Sharing distributed processing and data resources
- Enabling People to Work Together on Challenging
Projects - Science, Engineering, Medicine - e-Science,
e-Health - Public service, commerce - e-Government,
e-Business - The Grid could be the new age Internet
- The Grid intends to make access to computing
power, scientific data repositories and
experimental facilities as easy as the Web makes
access to information., UK PM, 2002
5The GRID vision
- On one hand
- Researchers/employees perform their activities
regardless of geographical location, interact
with colleagues, share and access data - On the other hand
- Scientific instruments and experiments provide
huge amount of data, incl. national databases - And in the middle
- The Grid networked data, processing centres and
grid middleware as the glue of resources.
6Grid Services
- Basic unit of computation job
- Basic unit of storage file
- Information systems BDII, Globus-mds, R-GMA,
file catalogues, metadata catalogues - Authorization, authentication, accounting (AAA)
based on PKI (Public key infrastructure) - Every Grid site provides basic Grid services
- Advanced Grid Services MPI jobs, Mass Storage
Facilities accessed via SRM, Fine grained AAA
(VOMS, DGAS).
7Grid Services - schema
8Grid Services in gLite
9EGEE Partner Federations
- All work in EGEE will be carried out by the 70
partners grouped in 12 federations.
10Goals of the EGEE project
- Goal in one sentence
- Allow scientists from multiple domains to use,
share, and manage geographically distributed
resources transparently. - The EGEE project brings together experts from
over 27 countries with the common aim of building
on recent advances in Grid technology and
developing a service Grid infrastructure,
available to scientists 24 hours-a-day. - The project aims to provide researchers in
academia and industry with access to major
computing resources, independent of their
geographic location. The EGEE project will also
focus on attracting a wide range of new users to
the Grid.
11Scientific disciplines to run Grid applications
- EGEE aims to establish production quality
sustained Grid services - 3000 users from at least 5 disciplines
- integrate 50 sites into a common infrastructure
- offer 5 Petabytes (1015) storage
- Demonstrate a viable general process to bring
other scientific communities on board
12EGEE building a production Grid for e-Science
- Operations Management Centre (OMC)
- At CERN coordination etc
- Core Infrastructure Centres (CIC)
- Manage daily grid operations oversight,
troubleshooting - Run essential infrastructure services
- Provide 2nd level support to ROCs
- UK/I, Fr, It, CERN, Russia (M12)
- Taipei also run a CIC
- Regional Operations Centres (ROC)
- Act as front-line support for user and operations
issues - Provide local knowledge and adaptations
- One in each region many distributed
- User Support Centre (GGUS)
- In FZK manage PTS provide single point of
contact (service desk) - Not foreseen as such in TA, but need is clear
13Components of a production Grid
- A production Grid consists of stable
interoperating Grid sites (Resource centres),
which enable access to Grid users from various
Virtual Organizations - Every Grid site provides basic Grid services and
follows strict operational procedures. - Monitoring allows fast detection of problems and
their resolution or isolation.
14BG01-IPP setup
Terminals
SE
BDII
- PKI X.509 certificate keys - JDL files
RB/II
15Structure of EGEE operations
- The grid is flat, but
- Hierarchy of responsibility
- Essential to scale the operation
- CICs act as a single Operations Centre
- Operational oversight (grid operator)
responsibility - rotates weekly between CICs
- Report problems to ROC/RC
- ROC is responsible for ensuring problem is
resolved - ROC oversees regional RCs
- ROCs responsible for organising the operations in
a region - Coordinate deployment of middleware, etc
- CERN coordinates sites not associated with a ROC
RC Resource Centre
16Operations monitoring maps
- In LCG-2
- 137 sites, 34 countries
- gt12,000 cpu
- 5 PB storage
- Includes non-EGEE sites
- 9 countries, 18 sites
17Selection of Monitoring tools
GIIS Monitor
GIIS Monitor graphs
Sites Functional Tests
GOC Data Base
Scheduled Downtimes
Live Job Monitor
GridIce VO view
GridIce fabric view
Certificate Lifetime Monitor
Note Those thumbnails are links and are
clickable.
18Example LHC at CERN
19CMS LHC Experiment
20Example biomedical app gPTM3D
- One data set is
- DICOM files 100MB 1GB
- One radiological image 20MB 500MB
- Complex interface optimized graphics and
medically-oriented interactions - Physician interaction is required at and inside
all steps - Poorly discriminant data, pathologies, medical
windowing
Interaction
Render
Explore
Analyse
Interpret
Acquire
21Figures
Dataset 87MB 210MB 346MB 87MB
Input data 3MB 18KB/slice 9.6
MB 25KB/slice 15MB 22KB/sclice 410KB 4KB/slice
Output data 6MB 106KB/slice 57MB 151KB/slice
86MB 131KB/slice 2.3MB 24KB/slice
Tasks 169 378 676 95
StandaloneExecution 5min15s 1min54s 33min 11min
5s 18min 36s
EGEE Execution 14 procs. 37s 18s 2min30s 1min15s
2min03 24s
Small body Medium body Large body Lungs
22Example The MAGIC Telescope
- Ground based Air Cerenkov Telescope
- Gamma ray 30 GeV - TeV
- LaPalma, Canary Islands (28 North, 18 West)
- 17 m diameter
- operation since autumn 2003(still in
commissioning) - Collaborators
IFAE Barcelona, UAB Barcelona, Humboldt U.
Berlin, UC Davis, U. Lodz, UC Madrid, MPI
München, INFN / U. Padova, U. Potchefstrom, INFN
/ U. Siena, Tuorla Observatory, INFN / U. Udine,
U. Würzburg, Yerevan Physics Inst., ETH Zürich
Physics Goals Origin of VHE Gamma rays Active
Galactic Nuclei Supernova Remnants Unidentified
EGRET sources Gamma Ray Burst
23Ground based ?-ray astronomy
24MAGIC Hadron rejection
- Based on extensive Monte Carlo Simulation
- air shower simulation program CORSIKA
- Simulation of hadronic background is very CPU
consuming - to simulate the background of one night, 70 CPUs
(P4 2GHz) needs to run 19200 days - to simulate the gamma events of one night for a
Crab like source takes 288 days. - At higher energies (gt 70 GeV) observations are
possible already by On-Off method (This reduces
the On-time by a factor of two) - Lowering the threshold of the MAGIC telescope
requires new methods based on Monte Carlo
Simulations
25BG application in SEE-GRID VO - SALUTE
- The Problem ultra-fast semiconductor carrier
transport - femtosecond relaxation of hot electrons by
phonon emission in presence of electric field. - Barker-Ferry equation and Monte Carlo approach
- Application in nanotechnologies innovative
results for GaAs - collision broadening and memory effects of
quantum kinetic model - Intra-collision field effect quantum scattering
- retarding and accelerating field. - NP-hard problem concerning the evolution time
- Parallel and Grid implementation
26Wigner function
800 x 260 points 150 fs
27Energy relaxation processcollisional broadening
Accumulation From 10 fs up to 250 fs
28 BG application in ESR VO air pollution
prediction
- Under development by Tzvetan Ostromsky from IPP
- Transition from HPC to Grid computing
29Challenges before new sites
- Install middleware and follow security and
middleware upgrades in a timely fashion - Present valuable resource to the Virtual
Organizations that the site supports - Participate in the various challenges. So far we
have seen the HEP and the Biomed VO challenges,
and the security challenges - Participate in innovation efforts development
of middleware and/or grid applications - Attract new users
- The Grid is about people
30BG Grid support centre contact information
- Contact persons
- Emanouil Atanassov, SA1 Activity Leader,
- emanouil_at_parallel.bas.bg
- Aneta Karaivanova,
- NA2 Activity Leader,
- anet_at_parallel.bas.bg
- Todor Gurov,
- Alternate EGEE SEE-ROC and SEE-GRID
manager, gurov_at_parallel.bas.bg - Ivan Dimov,
- EGEE SEE-GRID Project manager for BG
- ivdimov_at_bas.bg