Particle Physics Data Grid Request Management working group

1
Particle Physics Data GridRequest
Managementworking group
2
Site Manager
parallel disk system
Site Manager
Network
Site Manager
parallel disk system
parallel disk system
Schematic of the distributed data system
3
Request Interpreter
Request Manager
Cache Manager
4
Scenarios

• Simple
• All files for a local application are cached to
  a local disk-cache
• General
• Files for an application can be cached to any
  disk-cache on any site

5
(No Transcript)
6
Functionality of modules (1)

• Request Interpreter
• accepts estimate requests based on property
  predicates
• accepts execute requests for
• property predicates
• set of events
• set of files
• sends request inquiry to Logical Index Service
• consults Matchmaker Resource Planner for
  estimation
• passes entire logical file request to Request
  Manager

7
Functionality of modules (2)

• Request Manager
• accepts logical file request and notifies
  Application module ready to process files
• consults Matchmaker Resource Planner to
  generate request plan
• accepts get next events (files) requests from
  Client module
• coordinates multi-application requests to share
  files
• issues cache file(s) request to Cache Manager
• notifies Client module when a (bundle of) files
  is available in some cache (can be on remote
  sites)
• notifies Cache Manager when a Client is finished
  with files

8
Functionality of modules (3)

• Cache Manager
• accepts cache a file requests from the local
  Request Manager
• accepts reserve space requests from remote
  reservation calls
• allocates space on local cache as well as tape
  resourcesaccording to a local resource
  management policy
• issues lock a file service requests to remote
  sites
• issues reserve space service requests to remote
  sites
• issues move a file service request
• issues release a file service request to remote
  sites
• notifies Matchmaker of any local cache changes

9
Functionality of modules (4)

• Local Resource Manager
• accepts requests for multi-file caching
• Determines the best way to allocate local tape
  and disk cache resources
• notifies cache manager to reserve space
• notifies tape resource manager to reserve tape
  drives
• ...

10
Local Site Manager
Remote Services
logical request (property predicates / event set)
Properties, Events, Files Index
Request Interpreter
files to be retrieved fileevents
Request Manager
File Replica Catalog
Request to move files file from,to
Request to reserve space cache_location
bytes
To Local Cache
GLOBUS Services Layer
To Network
Fig 1 Architecture for the simple scenario
11
(No Transcript)
12
Sequence of API calls

• 1. Application issues an estimate
• 2. Consult Logical index Service
  (properties-events-files) for estimate
• 3. Consult Matchmaker Resource Planner for
  estimate
• 4. Issue execute request command
• 5. Consult Logical index Service for precise
  request evaluation
• 6. Sent filesevents request to Request Planner
• 7. Client issues a request for next events to
  process (files are implied)
• 8. Request Planner Consults Matchmaker Resource
  Planner
• 9. Request Planner issues get file(s), source,
  destination request
• 10. Cache Manager issues reserve space request
  if destination is remote site
• 11. Cache Manager issues move file(s) request
• 12. Cache Manager notifies Matchmaker of any
  changes to local cache
• 13. Client issues commands to read events out of
  files

13
People assigned to define API interfaces(first
name responsible to contact others)

• 1 2 4 5 -- Alex, Dave, Vicky (Igor)
• 3 8 -- Vicky (Igor) to coordinate with Mirons
  working group
• 6 -- Alex, Arie
• 7 -- Dave, Vicky (Igor), Andy
• 9 -- Luis, Andy (for bundle at a time request,
  not just a single file)
• 10 11 -- Luis, Andy
• 12 -- Andy, Luis
• 13 -- Andy, Dave

14
Local Site Manager
Remote Services
logical request (property predicates / event set)
Properties, Events, Files Index
Request Interpreter
Objectivity
files to be retrieved fileevents
AMS
Request Manager
File Replica Catalog
Event (File) in Cache?
Request to move files file from,to
Request to reserve space cache_location
bytes
OOFS
Reserve / make space
Move file HPSS-DISK
To Local Cache
GLOBUS Services Layer
Where OOFS functionality corresponds to
architecture components
To Network
Fig 3 OOFS integration