The Globus Toolkit and OMIIEurope

1
The Globus Toolkit and OMII-Europe

• Neil Chue Hong
• EPCC, University of Edinburgh
• Thanks to Ian Foster and the Globus Team for
  slides

2
What specific problem is the Globus
Toolkitdesigned to address?
3

• Ultimately, the Globus Toolkit
• is designed to enable the
• creation and maintenance of
• Virtual Organizations

4
Virtual Organizations

• Distributed resources and people
• Linked by networks, crossing admin domains
• Sharing resources, common goals
• Dynamic
• Fault tolerant

R
R
R
R
R
R
R
R
R
R
R
R
VO-A
VO-B
5
Layers in the Grid
6
The Globus ToolkitStandard Plumbing for the
Grid

• Not turnkey solutions, but building blocks
  tools for application developers system
  integrators
• Some components (e.g., file transfer) go farther
  than others (e.g., remote job submission) toward
  end-user relevance
• Easier to reuse than to reinvent
• Compatibility with other Grid systems comes for
  free
• Today the majority of the GT public interfaces
  are usable by application developers and system
  integrators
• Relatively few end-user interfaces
• In general, not intended for direct use by end
  users (scientists, engineers, marketing
  specialists)

7
A Typical eScience Use of GlobusNetwork for
Earthquake Eng. Simulation
Links instruments, data, computers, people
8
Without the Globus Toolkit
ComputeServer
A
SimulationTool
ComputeServer
B
WebBrowser
WebPortal
RegistrationService
Camera
TelepresenceMonitor
DataViewerTool
Camera
Database service
C
ChatTool
DataCatalog
Database service
D
CredentialRepository
Database service
E
Certificate authority
Resources implement standard access management
interfaces
Collective services aggregate /or virtualize
resources
Users work with client applications
Application services organize VOs enable access
to other services
9
With the Globus Toolkit
ComputeServer
GlobusGRAM
SimulationTool
ComputeServer
GlobusGRAM
WebBrowser
CHEF
Globus IndexService
Camera
TelepresenceMonitor
DataViewerTool
Camera
Database service
OGSADAI
CHEF ChatTeamlet
GlobusMCS/RLS
Database service
OGSADAI
MyProxy
Database service
OGSADAI
CertificateAuthority
Resources implement standard access management
interfaces
Collective services aggregate /or virtualize
resources
Users work with client applications
Application services organize VOs enable access
to other services
10
The Globus Toolkit is a Collection of Components

• A set of loosely-coupled components, with
• Services and clients
• Libraries
• Development tools
• GT components are used to build Grid-based
  applications and services
• GT can be viewed as a Grid SDK
• GT components can be categorized across two
  different dimensions
• By broad domain area
• By protocol support

11
GT Domain Areas

• Core runtime
• Infrastructure for building new services
• Security
• Apply uniform policy across distinct systems
• Execution management
• Provision, deploy, manage services
• Data management
• Discover, transfer, access large data
• Monitoring
• Discover monitor dynamic services

12
GT Protocols

• Web service protocols
• WSDL, SOAP
• WS Addressing, WSRF, WSN
• WS Security, SAML, XACML
• WS-Interoperability profile
• Non Web service protocols
• Standards-based, such as GridFTP
• Custom

13
Stateless vs. Stateful Services
FileTransferService
Client
move (A to B)
move

• Without state, how does client
• Determine what happened (success/failure)?
• Find out how many files completed?
• Receive updates when interesting events arise?
• Terminate a request?
• Few useful services are truly stateless, but WS
  interfaces alone do not provide built-in support
  for state

14
FileTransferService (without WSRF)
FileTransferService
Client
move (A to B) transferID
move
whatHappen
state
tellMeWhen
cancel

• Developer reinvents wheel for each new service
• Custom management and identification of state
  transferID
• Custom operations to inspect state synchronously
  (whatHappen) and asynchronously (tellMeWhen)
• Custom lifetime operation (cancel)

15
WSRF in a Nutshell

• Service
• State representation
• Resource
• Resource Property
• State identification
• Endpoint Reference
• State Interfaces
• GetRP, QueryRPs, GetMultipleRPs, SetRP
• Lifetime Interfaces
• SetTerminationTime
• ImmediateDestruction
• Notification Interfaces
• Subscribe
• Notify
• ServiceGroups

Service
GetRP
GetMultRPs
EPR
EPR
EPR
SetRP
QueryRPs
Subscribe
SetTermTime
Destroy
16
FileTransferService (w/ WSRF)
FileTransferService
Client
createResource (A to B) EPR
createResource
getRP
queryRPs
destroy

• Developer specifies custom method to
  createResource and leaves the rest to WSRF
  standards
• State exposed as Resource Resource Properties
  and identified by Endpoint Reference (EPR)
• State inspected by standard interfaces (GetRP,
  QueryRPs)
• Lifetime management by standard interfaces
  (Destroy)

17
Globus Toolkit version 2 (GT2)
Web ServicesComponents
Pre-WS Authentication Authorization
GridFTP
C Common Libraries
Grid Resource Alloc. Mgmt (GRAM)
Monitoring Discovery (MDS)
Non-WS Components
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
18
Globus Toolkit version 3 (GT3)
Data Access Integration
CommunityAuthorization
Web ServicesComponents
WS Authentication Authorization
Reliable File Transfer
Grid Resource Alloc. Mgmt (WS GRAM)
MDS3
Java WS Core
Pre-WS Authentication Authorization
GridFTP
C Common Libraries
Grid Resource Alloc. Mgmt (GRAM)
Monitoring Discovery (MDS)
Non-WS Components
Replica Location
eXtensible IO (XIO)
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
19
Core
Globus Toolkit version 4 (GT4)
Contrib/Preview
Grid Telecontrol Protocol
Depre-cated
Community Scheduling Framework
Delegation
Data Replication
Python WS Core
WebMDS
Data Access Integration
CommunityAuthorization
Trigger
C WS Core
Workspace Management
Web ServicesComponents
Authentication Authorization
Reliable File Transfer
Grid Resource Allocation Management
Index
Java WS Core
Pre-WS Authentication Authorization
GridFTP
Pre-WS Grid Resource Alloc. Mgmt
Pre-WSMonitoring Discovery
C Common Libraries
Non-WS Components
Replica Location
eXtensible IO (XIO)
Credential Mgmt
www.globus.org
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
20
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
21
GT4 Components
Your C Client
Your Python Client
Your Java Client
Your Python Client
Your Python Client
Your C Client
Your C Client
CLIENT
Your Java Client
Your Java Client
Your Python Client
Your C Client
Your Java Client
Interoperable WS-I-compliant SOAP messaging
X.509 credentials common authentication
RFT
GRAM
Delegation
Index
Trigger
Archiver
Your C Service
CAS
OGSA-DAI
Your Python Service
GTCP
Your Java Service
Your Java Service
RLS
Pre-WS MDS
SimpleCA
MyProxy
GridFTP
Pre-WS GRAM
C WS Core
pyGlobus WS Core
Java Services in Apache Axis Plus GT Libraries
and Handlers
C Services using GT Libraries and Handlers
Python hosting, GT Libraries
SERVER
22
OMII-Europe / Globus activities
Port OGSA-DAI
Security
Common Accounting
WS-GRAM w/ BES and JSDL
23
GT4 Security
Users
24
GT4 Security

• Public-key-based authentication
• Extensible authorization framework based on Web
  services standards
• SAML-based authorization callout
• As specified in GGF OGSA-Authz WG
• Integrated policy decision engine
• XACML policy language, per-operation policies,
  pluggable
• Credential management service
• MyProxy (One time password support)
• Community Authorization Service
• Standalone delegation service

25
GT4s Use of Security Standards
Supported, Supported, Fastest,
but slow but insecure so default
26
GT-XACML Integration

• eXtensible Access Control Markup Language
• OASIS standard, open source implementations
• XACML sophisticated policy language
• Globus Toolkit ships with XACML runtime
• Included in every client and server built on GT
• Turned-on through configuration
• that can be called transparently from runtime
  and/or explicitly from application
• and we use the XACML-model for our Authz
  Processing Framework

27
GT Authorization Framework
28
Other Security Services Include

• MyProxy
• Simplified credential management
• Web portal integration
• Single-sign-on support
• KCA kx.509
• Bridging into/out-of Kerberos domains
• SimpleCA
• Online credential generation
• PERMIS
• Authorization service callout

29
GT4 WS GRAM

• 2nd-generation WS implementation optimized for
  performance, flexibility, stability, scalability
• Streamlined critical path
• Use only what you need
• Flexible credential management
• Credential cache delegation service
• GridFTP RFT used for data operations
• Data staging streaming output
• Eliminates redundant GASS code

30
GT4 WS GRAM Architecture
Service host(s) and compute element(s)
SEG
Job events
GT4 Java Container
Compute element
GRAM services
Local job control
GRAM services
Local scheduler
Job functions
sudo
GRAM adapter
Delegate
Transfer request
Client
Delegation
Delegate
GridFTP
User job
RFT File Transfer
FTP control
FTP data
Remote storage element(s)
GridFTP
31
Summary

• The Globus Toolkit is a collection of reuseable
  components to help application builders harness
  the Grid
• In OMII-Europe, the challenge is to allow
  components from different middleware stacks to
  interact and interoperate well
• For Globus, the main areas are
• Job Submission
• Data Services
• Accounting and Security

32
Thank you for listening

• Any questions?
• http//www.omii-europe.com
• http//www.globus.org