ATLAS TDAQ Networking, Remote Compute Farms

1
ATLAS TDAQNetworking,Remote Compute Farms
Evaluating SFOs
Richard Hughes-Jones The University of
Manchester www.hep.man.ac.uk/rich/ then
Talks
2
Network Purchase Review

• Considered the 4 networks in the Online
  systemDataCollection, Backend, Control,
  Management
• The review process was to
• Derive the quantities and connectivity from each
  of the document sources
• Cross-check where possible that the information
  was consistant
• Examine what was ordered
• Documents used
• network viso diagrams ROS
• Control and Data Networks Architecture Proposal
  28 April, 2006 version 0.4
• The order to IT for the network components
• Rack layouts
• Components devices Ordered
• Installation well in hand

3
Remote Computing Concepts
Remote Event Processing Farms
Copenhagen Edmonton Krakow ManchesterAmsterdam
ATLAS Detectors Level 1 Trigger
Event Builders
lightpaths
GÉANT
Level 2 Trigger
Local Event Processing Farms
CERN B513
Mass storage
Experimental Area
4
Tests Traffic on Pionier

• Summer 2006
• Set up Advanced Network TestGPS systems CERN,
  Krakow, Manc
• Examined Network Performance
• PC-PC
• ANT-ANT

5
CERN ?? Krakow Throughput
CERN ? Krakow
Krakow ? CERN

• 800Mbit/s
• 20 Loss in the network

• 600-800Mbit/s
• 40 Loss in the network
• 3 more re-ordering

6
Remote Farms TDAQ Initiative

• Aim of the Initiative
• Develop the methodology
• Provide or identify the required tool kits
• Demonstrate Proof of Concepts for using remote
  Grid or compute farms in real time for ATLAS TDAQ
  applications.
• Not guaranteed real-time services for ATLAS yet
• Use Cases
• Calibration specific data from a small section
  of the experiment
• Monitoringdata from several sub-detectors,
  allowing more detailed examination of a
  sub-detector performance
• Remote event storage
• Remote Event Filter Processing extend the
  computing capabilities of the Event Filter to
  remote Grid Farms
• Security Protect Point1
• Application Gateways
• Links to Grid operations
• Document in Draft

7
SFO Evaluation

• SFO
• Buffers events from the Event Filter
• Transfers them to Mass Storage (CASTOR)
• Data movements
• Receive events over the network from Event
  Filterspecific data from a small section of the
  experiment
• Store them on one RAID5 disk sub-system in
  multiple streams
• When this is full swap writing to second RAID
  system
• Read data back from first RAID5 disk sub-system
• Send events over second network to mass storage
  at CERN
• As part of the RFI (Tender)
• Test performance of RAID controllers

8
RAID Controller Performance Single Flow

• Elonex system had the Intel S5000PSL motherboard
• Two dual-core Intel Xeon CPU 5130 2.00GHz CPUs
• 3Ware 9550SX-12MI 12 port SATA-RAID on PCI-e bus
• Seven465.76 GB Hitachi Deskstar E7K500
  Configured as RAID5 64k byte stripe size

• RAID5 ext2 file system
• One flow
• 2 GByte file Read 2750 Mbit/s,
  Write 1215 Mbit/s

9
RAID Controller Performance Multiple Flows

• RAID5 ext2 file system
• 8 GByte file
• 2 Flows 180 Mbit/s /flow
  5 Flows 10-25 Mbit/s /flow
• Too slow !

10

• Any Questions?

11

• Backup Slides