Grid and VOs

1
Grid and VOs
2
Grid from 10 000 feet
Researchers perform their activities regardless
geographical location, interact with colleagues,
share and access data
Scientific instruments, libraries and experiments
provide huge amounts of data
The GRID networked data processing centres and
middleware software as the glue of resources.
based on material from Federico.Carminati_at_cern.ch
and the 3D RTM map by Gidon Moont, IC and GridPP
3
What is Grid?

• The word grid has been used in many ways
• cluster computing
• cycle scavenging
• cross-domain resource, data and information
  sharing
• A definition for what we mean with grid
• Coordinates resources not subject to centralised
  control
• Using standard, open and generic protocols
  interfaces
• Provides non-trivial qualities of collective
  service

Definition source Ian Foster in Grid Today, July
22, 2002 Vol. 1 No. 6, see http//www-fp.mcs.anl
.gov/foster/Articles/WhatIstheGrid.pdf
4
Grid Computing More Than One

• More than one machine
• More than one user
• More than one research community
• More than one administrative domain
• More than one geographical location

General case more than one of each!!!
5
Consequences of Plurality

• More than one user / research community
• Partitioning of resources, authentication,
  authorization, accounting
• More than one machine
• Software engineering, distributions
• More than one administrative domain / research
  community
• Authentication / authorization, non-invasive
  installations, genericity
• More than one admin domain, geographical location
• Worldwide operations coordination

6
Grid characteristics

• Things in e-Science grids that may contrast with
  other distributed efforts
• collaboration of individuals from different
  organisations
• most of the scientific grid communities today
  consist of people scattered over many home
  organisations in many cases internationally
• Virtual organisations but thats what we
  are used to as scientific collaborations!
• delegation services acting on your behalf are
  an integral part of the architecture
• for service and data brokering
• integrating compute, data access, and databases
  in the same task
• unattended work flows

7
Virtual Organisations

• A set of individuals or organisations, not under
  single hierarchical control, (temporarily)
  joining forces to solve a particular problem at
  hand, bringing to the collaboration a subset of
  their resources, sharing those at their
  discretion and each under their own conditions.

• Users are usually a member of more than one VO
• Any large VO will have an internal structure,
  with groups, subgroups, and various roles

8
Virtual organisation structure

• Lots of overlapping groups and communities

graphic OGSA Architecture 1.0, OGF GFD-I.030
9
Virtual vs. Organic structure

• Virtual communities (virtual organisations) are
  many
• An individual will typically be part of many
  communities
• has different roles in different VOs (distinct
  from organisational role)
• all at the same time, at the same set of
  resources
• but will need single sign-on across all these
  communities

graphic OGSA Architecture 1.0, OGF GFD-I.030
10
Expressing collaboration

• provide the means to express collaboration
• membership
• groups and roles
• organisation management tools
• support access control as function of VOs
• access control as a function of VO, group, and
  role
• both at the service and at the content level
• maintain autonomy
• sharing defined by access controls at the source
• no need to hand off the actual data to a third
  party

11
VL-e PoC
12
PoC Position in the VL-e structure
Application specific service
App1
App 2
App 3
Application Potential Generic service
Virtual Lab. services
Virtual Lab. rapid prototyping (interactive
simulation)
Virtual Laboratory
Additional Grid Services (OGSA services)
Grid Middleware
Grid Network Services
Network Service (lambda networking)
Networking
VL-e Experimental Environment
VL-e Proof of concept Environment
13
The VL-e PoC Proof-of-Concept

• What is the PoC Environment?
• A shared, common environment,
• where different tools and services are
• both used and
• provided by the VL-e community
• basis for subsequent application development

14
Elements in the PoC

• The PoC refers to three distinct elements
• PoC Software Distribution
• set of software that is the basis for the
  applications
• both grid middleware and virtual lab generic
  software
• PoC Environment
• the ensemble of systems running the Distribution
• including user desktops or local clusters and
  storage
• PoC Central Facilities
• those systems with the PoC Distribution centrally
  managed for the entire collaboration
• large-scale computing, storage and hosting
  resources

15
PoC Distribution

• The PoC distribution contains components to
• enable service-oriented development
• enable application development
• provide access to data, computing, and storage,
  distributed geographically
• driven by specific VL-e application scenarios
• Work flow to be the integrative layer of VL-e
• functionality should be invocable as a service
• work flow (graphical) systems help in
  compositionbut are not the only way to interact
  with services

16
The PoC software distribution

• The PoC software suite. the following elements
  of this suite can be distinguished
• Grid foundation middleware the basic software
  that is based on interfaces and concepts that are
  internationally adopted. This includes elements
  such as the security model, resource allocation
  interface, based on EGEE middleware suite
• Generic Virtual Laboratory software the software
  developed within the project for the PoC.
• Services imported from outside given that not
  all services are necessarily developed within
  VL-e, components have been imported.
• Associated installation and deployment tools the
  PoC suite is installed on the central facilities
  and (where applicable) also available for
  distributed installation.

17
The PoC software distribution

• Software environment
• geared towards application software developers
• enables cross-leveraging VL-e developments
  between applications
• predictable lifecycle management
• Primary metric is the effectiveness in
  addressing real cross-application needs
• PoC is liberal in including software
• as long as it is useful for multiple domains
• does not compromise integrity
• can be supported and safely deployed

18
Defining content of the PoC Distribution
Tagged Release Candidates
Download RepositoryPoC Installer
Common repositoryIntegration tests
stable, tested releases
External software
VLeIT Recommendation Point
19
Working with the application developers

• Each generic component has an expert on VLeIT
• to work on its optimal use or deployment and
• coordinate enhancement requests
• Latest developments from within the VL-e project
• availability via a fast-lane contrib trajectory
• same installation mechanism
• but supported directly by the developers
• addressing the chicken-and-egg dead lock

20
The VL-e PoC Distribution

• What is the VL-e PoC Distribution?
• The PoC distribution is
• meant to be installed on a RedHat Enterprise
  Linux 3 compatible system
• a stable base environment, with managed releases
• The PoC distribution contains components to
• enable service-oriented development
• enable application development
• provide access to computing, storage, and
  information systems

21
The VL-e PoC Distribution
VL-e PoC Release 1.0 Contents gLite 3.0 Sun
Java2SDK 1.4.2_12 Plus JavaGAT-1.5 MatlabMPI-1.2 M
esa3D-6.4.1 R-2.2.0 Rmpi-0.5 SRB-client-3.4.0 SRB-
devel-3.4.0 fsl-bin-3.2 fsl-devel-3.2 gat-adaptors
-1.8.2 gat-cpp-wrapper-1.8.2 gat-engine-1.8.2 gat-
python-wrapper-1.8.2 globus-toolkit-4.0.1 graphviz
-2.8 ibis-1.2.1 itk-2.4.1 kepler-1.0.0alpha7 lam-d
evel-7.1.2 lam-docs-7.1.2 lam-extras-7.1.2 lam-run
time-7.1.2 libRmath-2.2.0 libRmath-devel-2.2.0 med
line-1.0 mpitb-2.1.72 mricro-1.3.9-4 nimrod-3.0.1
octave-2.1.72 ogsadai-wsrf-2.1 paraview-2.4.2 pl-5
.6.4-200 sesame-client-1.2.3 taverna-workbench-1.3
.1 triana-3.2 vtk-4.4
22
The VL-e PoC Distribution

• Distribution formats
• Network-based installation
• http access
• http proxy access
• DVD-based installation the PoC DVD
• Pre-installed VMware image (present on PoC DVD)
• CentOS 3 with GNOME GUI
• gLite UI
• VL-e Release 1.0 UI packages
• Works with free VMware Player on both Linux and
  Windows

23
PoC Environment

• All systems can be used to perform the
  application scenarios, using the PoC distribution
• Installed both
• at specific central facilities
• on desktops, remote clusters, data servers

24
PoC Central Facilities

• For applications in the Netherlands
• both applications within VL-e and others
• shared common infrastructure
• accessible via grid middleware
• has of course PoC distribution installed
• Location and capacity
• SARA tape (1.2 TB), disk storage (100 TB),
  clusters (1400 cores Debian, 60
  RHEL3),database servicesuser interface gateway
  catch-all
• NIKHEF disk storage (25 TB), clusters (550
  cores RHEL3)

25
PoC Central Facility Usage Today
PoC (NDPF) shared between various applications
SARA LISA Occupancy
grey VLEIBU, VLEMED green ATLAS, blue LHCb
PHICOS production jobs on the PoC (NDPF) at NIKHEF