An Overview of the Federated MCAT design

1

• An Overview of the Federated MCAT design
• (SRB 3.0)
• By Michael Wan
• http//www.npaci.edu/DICE/SRB/

SDSC/UCSD/NPACI
2
Single MCAT architecture

• Federated middleware system
• Single MCAT server
• Metadata catalog
• Use traditional DBMS
• Federated resource servers
• Single client signOn on anyone of the servers
• Access to all resources in the federation
• Robust server-server operation

3
Single MCAT architecture
Peer-to-peer Brokering
Client Read
Parallel Data Access
Logical Name Or Attribute Condition
1
6
5/6
SRB server
SRB server
3
4
5
SRB agent
SRB agent
2
Server(s) Spawning
R1
MCAT
1.Logical-to-Physical mapping 2. Identification
of Replicas 3.Access Audit Control
R2
Data Access
4
Single MCAT Concepts and Features

• A single global User Space
• Single sign-on, access all resources
• Single class of Admin User
• Multiple authentication schemes
• GSI, secure passwords, tickets
• Robust access control

5
Single MCAT Concepts and Features (cont1)

• Data and Collections
• Logical Name space -
• UNIX like directories (collections) and files
  (data)
• Mapping of logical name to physical attributes -
  host address, physical path.
• UNIX like API and utilities for making
  collections (mkdir) and data creation (creat)
• Virtualisation of Resources
• Mapping of a logical resource name to physical
  attributes Resource Location, Type
• Client use a single logical name to specify a
  resource

6
Single MCAT Concepts and Features (cont2)

• Performance enhancement
• Client and Server-driven Parallel I/O strategies
  (3-4 times)
• Interface with HPSSs mover protocol for parallel
  I/O
• Parallel third party transfer for copy and
  replicate
• One hop data transfer between client and data
  resource
• Container physical grouping of small files for
  tape I/O
• Bulk load and unload speed up uploading and
  downloading of small files (10-50 times)

7
Motivation for a Federated (multiple) MCAT system

• MCAT performance network latency for a world
  wide network.
• East/west coast latency for a simple query 1-2
  sec.
• Local control
• Security issue multiple admin users from
  different sites
• organizations want to have more control on the
  data and resources that they are sharing
  worldwide.
• Scalability of the MCAT.
• No single point of failure

8
Design goal of 3.0

• Very Basic features - incremental changes
• Multiple MCATs
• MCAT ZONE a new term
• defines a federation of SRB resources controlled
  by a single MCAT
• Each Zone has its own sys admin - local control
  of users and resources
• Peer to peer zone architecture
• Each Zone can operate entirely independently from
  other zone.
• Data and Resource sharing across ZONES
• Use storage resources in foreign zones
• Access data stored in foreign zones
• Copy data across zones

9
Peer to peer Federated MCAT
MCAT1
Server1.1
MCAT3
Server1.2
Server3.1
MCAT2
Server2.2
Server2.1
10
Peer to peer Federated MCAT implementation

• Development tasks needed
• 1) New metadata for Zones
• new Zone table
• add zone to user metadata
• 2) Authentication across zones
• 3) Resource and data access across zones
• Version 3.0 - Unix server, Scommand client only

11
1) Zone metadata implementation (1)

• System of MCAT zones
• Each MCAT manages most metadata independently of
  other zones
• No change for most metadata and metadata
  operations
• New meatdata
• Zone Info - new table defines all zones in the
  federation
• zoneName logical name
• Network address
• Local/Foreign
• Authentication info
• Each MCAT maintains a complete zone table

12
1) Zone metadata implementation (2)

• User Info - defines all users in the federation
• Single global user name space
• Each MCAT maintains a table of all users
• Local/Foreign user a user local to only one
  zone
• Sensitive info (password) only stored in users
  local zone

13
1) Zone metadata implementation (3)

• New Admin tool for handling new metadata
• Zone Authority page
• obtain a unique zoneName
• Web page available with 3.0 release
• Create and modify Zone metadata
• Java admin GUI , command line
• User Admin tool
• Modified to handle Zone related metadata
• New tools to poll foreign MCATs for new users and
  add to local MCAT.

14
2) Cross Zone Authentication (1)

• Support 2 authentication schemes
• Encrypted Password
• challenge-response scheme
• no password over the net
• GSI
• Phased out plain text password and SEA

15
2) Cross Zone Authentication (2)

• Support a robust set of server-server operations
• servers running as privileged SRB users perform
  operations on behalf of client users.
• Want to limit privilege of admin user from
  foreign zone
• an admin user can only requests a foreign zone to
  perform operations behalf of client users from
  the SAME zone
• sensitive info - password of local user stored
  only in local MCAT
• security compromise in one zone cannot spread to
  other zone

16
2) Cross Zone Authentication (3)

• Implication of security measure
• Lost a little transparency.
• Users must first connect to a server in its own
  zone first for across zone operation
• Only your own admin user can go across zone for
  you
• Very minor effect on data operation

17
3) Resource and data access across zones (1)

• Make easy by robust server-server support
• Most existing operations are supported across
  zones
• put, get, copy, bload, bunload, register,
  container operations, etc
• replication across zone not supported (use Scp
  for now)
• Typical operation open a collectionName /x/y/z
  for read.
• Data server queries MCAT for location, file type,
  etc
• But which MCAT to query ?
• The first entry in the collectionName specifies
  the zoneName where the metadata is stored
• for /z1/x/y/z z1 is the zoneName
• Similar to mount point in FS
• Most data handling codes unchanged. New codes to
  determine which MCAT to go to.

18
3) Resource and data access across zones (2)

• Most work involves adding logic to determine
  which MCAT to go to.
• Sput S s1 foo /z1/x/y/z
• (Scd /z1/x/y/z Sput S s1 foo .)
• Upload the local file foo to SRB.
• Create foo in mcat Zone z1, in resource s1 of
  z1
• SRB server query mcat z1 for metadata for
  resource s1 (network address, file type)
• Upload foo and put it in resource s1
• Register the file /z1/x/y/z/foo with mcat z1

19
3) Resource and data access across zones (3)

• Sget /z1/x/y/z/foo .
• Download the SRB file foo to local FS
• SRB server queries mcat z1 for metadata for
  file /z1/x/y/z/foo
• Discover file is in resource s1.
• Request resource s1 server to download the
  file.

20
3) Resource and data access across zones (4)

• Scp S s2 /z1/x/y/z/foo /z2/a/b/c
• Copy the SRB file foo managed by mcat z1 to the
  resource s2 of mcat z2
• Query z1 for file foo, found file in resource
  s1. Query z2 for resource s2
• Copy the file foo stored in resource s1 to
  sources2.
• Register the file /z2/a/b/c with mcat z2

21
3) Resource and data access across zones (5)

• Some Scommand does not involve collectionName -
  use z option
• -z zoneName option - explicitly specify a
  zoneName
• Use cwd if -z option not used
• SgetR -z z1
• Smkcont -z z1 -S s1 cont1
• If cwd/z2/x/y/z, Slscont

22
3) Resource and data access across zones (6)

• Registration of data files in more than one Zone
• File created/registered in zone z1
• /z1/u.d/x/y in resource s1 in zone z1
• Register also in zone z2
• /z2/u.d/x/y in resource s1 in zone z2
• s1 should be known to z2
• Note that collectionName changes.
• Some system-metadata carried over when doing
  inter-zone registration
• Copy user-defined metadata across the zones (if
  needed)
• Lazy synchronization (user-controlled)
• In 3.0
• Zone z1 and z2 are unaware of each others copies
  
• In later versions - awareness improved
• Metadata synchronization
• Deletion notification
• User-controllable delay in synchronization

23
Zone A
Zone B
/A/u3
/A/u1
/B/u1
/B/u3
/A/u4
/A/u2
/B/u2
/B/u4
/B
/A
/A/u1/f1? R1P1
/B/u3/f2? R2P2
/A/u3/f2? R2P2
/B/u1/f4? R1P4
User Synchronization
/A/u2/f1? R1P3
User Synchronization
Resource Synchronization
Zone Synchronization
DataSynchronization
24
3.0 Status

• Slightly delayed
• All core development done
• Zone metadata
• cross zone authentication
• cross zone resource and data access
• Items not yet complete
• User management tool - another week
• Documentation
• testing
• Release date - Later part of September

25
Future plans

• 3.1 - The rest of SRB software - window, inQ,
  mySRB, Jargon and Matrix
• 3.2
• use of proxy certificate for cross zone
  authentication
• Kerstin Kleese group at DL
• Cross zone replication
• Synchronization of copies
• versioning
• locking across zone ?
• Cross zone soft link