Middleware WG Status Report

1
Middleware WG Status Report

• ILDG 09
• Presented by
• Balint Joo

2
Overview

• Metadata Catalogue (MDC)
• File Catalogue (Replica Catalogue) Services
• Authentication and Virtual Organisation
  Membership Services (VOMS)
• File Transfer Services
• Outlook for the future

3
Metadata Catalogue (MDC)

• First version of the interface has been defined
• WSDL Interface Definition
• Allows generation of software implementation
  stubs (eg with wsdl2java or other tool)
• Behavioural Definition
• Specifies behaviour of implementation in terms of
• allowed input/output values
• expected return values / failure modes

4
MDC Implementations/Clients

• Implementations of clients
• CSSM Web Portal
• LDG Web Portal
• JLDG Web Portal
• UKQCD (GUI Application in Java)
• Allows Construction of Xpath Queries graphically
• Download it from NeSC Forge
• USQCD Web Portal
• Implementations of services listed in
• http//www.lqcd.org/ildg/Services.xml

5
Replica / File Catalogue

• Prototype Service Interface Definition Developed
  by LDG
• 'Interface Service' Software developed by LDG
• This is a web service implementation that
• Provides Web Service Interfaces on one end
• Allows user to slot in 'back-end' interface on
  the other
• user supplied 'geturl.sh' program (script/app)
• this takes care of all the Web Service Details
• has hooks for dealing with Certficates for
  authentication
• Interface software can be written once and shared
• hope for speedy roll-out
• Software in use by LDG and USQCD (others to
  follow)

6
Authentication/Authorization

• Using Virtual Organization Membership Service
  (VOMS)
• VOMS server currently hosted at DESY
• There is an ILDG V/O
• ILDG Users can request to register certificates
  in the ILDG V/O
• Requests to be approved in a decentralized way
• Two nominated people per sub-collaboration to
  approve requests from that collaboration
• Once users are registered they can be assigned to
  VOMS groups by the collaboration administrator
• VOMS Server can be queried (like a DNS server)
• use results in local infrastructures if desired

7
Joining the VO

• To join the ILDG V/O go to the VO registration
  page
• You need to have your certificate imported into
  your browser
• There is a simple form to fill out on the
  registration page
• Then click 'I have read and agree with the VO's
  usage rules'
• You will be sent a confirmation email with a
  'secret link' which you need to click to confirm
  your registration.
• VO administrators will receive an email notifying
  them of your registration
• A regional VO administrators will then approve
  your registration if they know you/check up on
  you.
• My last re-registration was done in about 2
  minutes
• and approved within the hour (approval time may
  vary)

8
VOMS Service

• Some example uses for VOMS server
• Download lists of the certificates of all ILDG
  users and allow them all to read data files from
  storage nodes (eg mode of operation at FNAL)
• Download lists of users by groups. Give users in
  some groups write access, but only read access to
  others (UKQCD mode)
• Use VOMS information to compare against local
  access control lists.
• Ignore VOMS server for fully open data eg USQCD
  MDC and File Catalogues

9
Current VOMS Admins

• CSSM
• No users from CSSM registered yet...
• JLDG
• Mitsuhisa Sato, Tomoteru Yoshie
• LDG
• Dirk Pleiter, Hinnerk Stueben
• UKQCD
• George Beckett, Chris Maynard
• USQCD,
• Balint Joo. Jim Simone

10
File Transfer Services

• This part of the 'middleware' depends heavily on
  local infrastructure
• SRM v2 was specified in the original
  architecture, could also auto negotiate download
  transfer protocols.
• de-facto standards lean towards
• SRM v1 (supplied by dCache) moving to v2.2 soon
• GridFTP
• HTTP (on web-site portals)
• This area of ILDG is not entirely straightforward
  to unify
• Mix of services may stay for a while
• Users may need several download clients on their
  machines (wget, srmcp, gridftp) etc.

11
File Transfer Service Status

• CSSM
• Testing dCache and aim to provide SRM by end of
  year
• JLDG
• Web based downloads for now. Moving towards
  GridFTP/SRM
• LDG
• SRM (version supported by dCache)
• UKQCD
• GridFTP (through QCDGrid)
• USQCD
• SRM (version supported by dCache) and GridFTP at
  FNAL
• FNAL this is to become the main ILDG data site
  in US for now
• HTTP (BNL QCDOC archive and Gauge Connection)

12
File Transfer Interoperability Tests

• Successful tests have been carried out between
  several sites
• From FNAL and LDG to 'external' machine using
  srmcp SRM client (which is conveniently packaged
  with dCache)
• From FNAL to LDG and vice versa
• From FNAL and LDG to UKQCD
• Transfers from DESY and FNAL to JLDG in
  processs...
• Overall Globus Grid FTP client and dCache SRM
  client have been quite successful.
• Need to carry on with interoperability trials eg
• Downloading data from UKQCD
• Downloading data from JLDG
• Downloading data from CSSM

13
Demo

• Let's try a demo following this link
• LDG 'List Ensembles' Link
• Pick a USQCD ensemble
• Pick 'List Configs'
• Click on one of the listed Configs
• display info about the config
• Click 'Show Copies' to look up the SRM URL
• Currently USQCD MDC out of sync with FNAL dCache,
  so downloading URL with srmcp may not work just
  yet.

14
Outlook

• Next Middleware Convenors face to face meeting
  organised
• Jlab, Dec 11-13 to discuss remaining and new
  issues
• Running Mirror VOMS servers (reliability)
• File Transfer Issues will need to be resolved
• Access control requirements
• Replication beyond regional grid boundaries
• Other non middleware issues
• Wiki/Web site
• shut down for cybersecurity reasons
• sticky plaster (band aid) solution currently in
  place