Xrootd Proxy Service

1
Xrootd Proxy Service

• Andrew Hanushevsky
• Heinz Stockinger
• Stanford Linear Accelerator Center
• SAG 2004
• 20-September-04
• http//xrootd.slac.stanford.edu

2
The BaBar Experiment

• Use big-bang energies to create B meson particles
• Look at collision decay products
• Answer the question where did all the
  anti-matter go?
• 500 physicists collaborating from 70 sites in 10
  countries
• USA, Canada, China, France, Germany, Italy,
  Norway, Russia, UK, Taiwan
• The experiment produces large quantities of data
• 300 TBytes/year for 10 years
• Most data stored as objects using Root
  persistency framework
• Some data stored in Objectivity/DB database
• Expected to double every year as detector
  luminosity increases
• Heavy computational load
• 5,000 1-2GHZ CPUs spread over 35 sites
  world-wide
• Work is distributed across the collaboration

3
BaBar is the Forerunner

• LHC at CERN
• The Large Hadron Collider
• Due to start in 2007
• Will generate several order of magnitude more
  data
• Will require even more compute cycles
• Example
• ATLAS
• Probe the Higgs boson energy range
• Explore the more exotic reaches of physics

4
The Data Access Need

• Scalable high performance access to data
• Must scale to 100s if not 1000s of data servers
• Most data is read-only
• Data is written only once
• Versioned
• Secondary access to distributed data
• As a backup strategy

5
Solution Fundamentals

• Extensible base server architecture
• Allows for high performance implementation
• Rich but efficient server protocol
• Combines file serving with P2P elements
• Allows client hints for improved performance
• Administrative security
• Implies a structured peer-to-peer framework

6
The Implementation

• High Performance File-Based Access
• Fluidly scalable
• Works well in single server environments
• Scales beyond 32,000 cooperative data servers
• Naively extensible
• Requirement for this level of scaling
• Servers can be added at any time without
  disruption
• Fully fault-tolerant
• Servers can be removed at any time without
  disruption
• Flexible Security
• Allowing use of almost any protocol

7
Entities Relationships
xrootd Data Network (redirectors steer clients
to data Data servers provide data)
olbd Control Network Managers
Servers (resource info, file location)
Redirectors
olbd
M
ctl
olbd
xrootd
S
Data Clients
xrootd
Data Servers
8
Example SLAC Configuration
data servers
kan01
kan02
kan03
kan04
kanxx
redirectors
bbr-rdr-a
bbr-rdr03
bbr-rdr04
client machines
9
Data Growth More Fault Tolerance

• BaBar Data Is Replicated
• Backup Strategy
• Processing Strategy
• Some data only available at one site
• Use grid techniques to make data accessible
• But, when thing go wrong would like access
• The proxy solution

10
The 10,000 Foot View
SLAC us
INFN it
Internet
FZK de
RAL uk
IN2P3 fr
11
The Reality

• Sites has a fear of hosting
• Distributed Denial of Service Attacks
• Massive illegal file sharing
• Only certain hosts allowed to get outside
• Rarely batch worker machines
• The ones that need remote data most
• The Firewall Issue

12
A Closer Look
SLAC
Firewall
Firewall
Firewall
IN2P3
RAL
IN2P3 proxy
RAL proxy
xrootds
Firewalls require Proxy servers
13
Proxy Service

• Attempts to address competing goals
• Security
• Deal with firewalls
• Scalability
• Administrative
• Configuration
• Performance
• Ad hoc forwarding for near-zero wait time
• Intelligent caching in local domain

14
Proxy Implementation

• Uses capabilities of olbd and xrootd
• Simply an extension of local load balancing
• Implemented as a special file system type
• Interfaces in the Logical File System layer (ofs)
• Functions in the Physical File System layer (oss)
• Primary developer is Heinz Stockinger

15
Proxy Interactions
data01
data02
data03
data04
RAL
proxy olb
local olb
4
5
local olb
proxy olb
SLAC
3
1
red01
data02
data03
proxy01
2
client machines
16
Why This Arrangement?

• Minimizes cross-domain knowledge
• Necessary for scalability in all areas
• Security
• Configuration
• Fault tolerance recovery

17
Scalable Proxy Security
SLAC PROXY OLBD
RAL PROXY OLBD
Data Servers
Data Servers
Firewall
1 Authenticate develop session key 2 Distribute
session key to authenticated subscribers 3 Data
servers can log into each other using session key
18
Proxy Performance

• Introduces minimal latency overhead
• Virtually undetectably from US/Europe
• Negligible on faster links
• 2 slower on fast US/US links
• 10 slower on LAN
• Can be further improved
• Parallel streams
• Better window size calculation
• Asynchronous I/O

19
Proxy Study Conclusion

• Proxy Service easily integrates into xrootd
• Largely due to peer-to-peer architecture
• Provides enhanced service at minimal cost
• Allows access to addition data sources
• Increases fault tolerance
• Covers up for grid transfer mistakes
• Scalable in all aspects
• Security, number of servers, administration

20
Overall Conclusion

• xrootd provides high performance file access
• Improves over afs, ams, nfs, etc.
• Unique performance, usability, scalability,
  security, compatibility, and recoverability
  characteristics
• Should scale to tens of thousand clients
• Will be distributed as part of CERNs root
  package
• Open software, supported by
• SLAC (server),
• INFN-Padova (client)
• CERN (security, packaging)