Online-Offsite Connectivity Experiments

1
Online-Offsite Connectivity Experiments

• Catalin Meirosu, Richard Hughes-Jones
• CERN and Politehnica University of Bucuresti
• University of Manchester
• The ATLAS Computing Model Workshop,
• Freiburg-im-Breisgau, Oct. 3rd, 2004

2
Outline

• What are we trying to prove and why
• Network setup
• Network measurements tools and results
• Conclusion

3
Why Connectivity Experiments for ATLAS

• Bandwidth from the online not going via Tier-0
  yet to be evaluated
• Clear requirement for direct outward flow for
• Some calibration tasks
• Many monitoring tasks
• Potential problem ATLAS might be low on
  computing resources, especially at the beginning
• Could require more use of offsite resources
• Even occasional remote event filtering might be
  useful
• We also need to prove that the network can handle
  the traffic originating in the Tier-0
• Tier-0/Tier-1 phase of DC2 will test part of
  this, but not high bandwidth
• At CERN, negotiations with bandwidth providers
  are happening now ! (carried on by the CERN IT,
  for all the LHC experiments)
• Any substantial non-Tier-0 requirements need to
  be established quickly
• These experiments are a proof of concept
• How much bandwidth can be used on a connection
  with a given capacity
• What can be achieved using ATLAS-related
  applications
• Proof of capability of the existing TDAQ software
  interworking with remote sites

4
(No Transcript)
5
Bandwidth measurements

• The networking is layered, from the physical
  transmission media (layer 1) to the application
  (layer 7)
• Tests at layer 2,3
• relevant when the remote and the local sites are
  logically in the same LAN
• programmable Gigabit Ethernet network interfaces
  used for traffic generation
• example throughput between CERN INP Krakow,
  August 20041000 Mbit/s
• Layer 4 tests TCP, UDP
• Relevant for general-purpose, Internet-style
  connectivity
• Performed tests between Geneva and Copenhagen,
  Edmonton, Krakow, Manchester
• Test equipment server PCs, running patched Linux
  kernels and open source software for network
  measurements

• Example Geneva Manchester
• The network can sustain 1Gbps of UDP traffic, but
  the average server has problems with smaller
  packets
• Degradation for packets smaller than 1000bytes,
  caused by the PC receiving the traffic

6
Real application in an ATLAS context
ATLAS Event Filter scenario
EF
request
network
event
SFI

• Simple request-response program
• Emulation of the request-response communication
  between the SFI and EFD in the Event Filter
• Runs over TCP/IP
• The client sends a small request message
• The server answers with an up to 2 MB message
• Results to be understood

7
Request Response results, CERN Uni.
Manchester connection

• Good response of the standard Linux TCP stack if
  properly tuned, poor if not
• Need to understand TCP implementation issues, not
  only the generic protocol
• 800Mbit/s achievable with tuned stack, 120 Mbit/s
  without the same end nodes were used in both
  cases !

• 64 byte Request green1 MByte Response blue

Out-of-the-box TCP settings
Tuned TCP stack
8
Network Monitoring

• Essential for End-to-End understanding
• Reduced performance on data transfer applications
  is often due to packet loss so where did the
  loss occur ?
• Need access to the statistics on the network
  providers routers along our paths
• Also need campus-level information, requires
  collaboration with people responsible for
  networking at each site

9
Conclusion

• We are investigating the technical feasibility of
  remote real-time computing for ATLAS
• Have exercised multiple 1 Gbit/s connections
  between CERN and Universities located in Canada,
  Denmark, Poland and the UK
• Network providers very helpful and interested in
  our experiments
• Developed a set of tests for a throughout
  characterization of the network connections
• Very good results obtained, due to the excess of
  bandwidth available in the backbone
• Properly configured end nodes essential for
  getting good results with real applications
• Next steps
• Complete the characterization of all the
  connections
• Interested in replicating calibration /
  monitoring style traffic, when model available
• Evaluate new data transfer protocols in
  ATLAS-specific scenarios
• Explore some of the above topics over a 10 Gbit/s
  connection
• Investigate the scalability of the TDAQ DataFlow
  in this scenario