Network Engineering @ SLAC

1
Network Engineering _at_ SLAC

• S. Luitz, D. Millsom, D. Salomoni, J.Y. Kim, A.
  Zele

2
Summary

• Introduction
• Major Issues
• Network Architecture
• Performance
• Retrospective

3
Introduction

• SLAC - Stanford Linear Accelerator Center
• High Energy Physics
• Stanford Synchrotron Radiation Laboratory
• 1300 full-time employees, 700 on-site
  collaborators
• Collaborate with 200 institutions internationally
• Mixture of real-time data acquisition, numerical
  analysis, business services/administration
• Very large quantities of data, e.g. 1-2
  petabytes/year for the BaBar experiment

4
Major Issues

• Bandwidth demand doubles every 1.5 years
• High reliability and availability requirement,
  both for experimental work and business services
• Network performance requirements dictate use of
  leading-edge technology
• Leading-edge technology challenges reliability
• International collaboration dictates need for
  open network
• Security - DOE requirements, open network is more
  prone to hacking

5
Major Issues...

• Network services have become so essential that
  failures can bring organizations to a standstill
• They are often more critical than the phone
• One reason critical information (data and
  applications) has moved into the (often-times
  central) computer/storage facilities
• SLAC particular issue - very high
  bandwidth/availability for data acquisition

6
Bandwidth Growth
7
Network Architecture

• Layer one
• Star topology
• Fiber/Copper
• Redundant
• Ethernet Full/Half Duplex, 10/100/1000 Mbps
• Backbone almost entirely Gigabit Ethernet (with
  Gigabit EtherChannel)
• Islands
• BaBar detector/data acquisition
• Main Control Center
• Business Services
• Layer two
• Switched VLAN (45 switches, 40 VLANs)
• Layer three
• Centralized routing

8
SLAC Switched LAN Spring 2000
10BaseT
ESA
FDDI/CDDI
100BaseFL
Old Servers
100BaseT
1Gbit FL
DMZ Switch
SSRL
4Gbit FL
Legacy
Concentrator
Monitoring
Gigaswitch
Routers
Core
Router
Switches
Switch
Hub
BSD
Netscout
6 Farm edge switches
IR2
4 Server switches
20 Building edge switches
MCC1
MCC2
MCC3
BaBar
SSRL
9
(No Transcript)
10
(No Transcript)
11
External Connections
45 Mb/s
2 Mb/s
155 Mb/s(? 622 Mb/s)
622 Mb/s
155 Mb/s
12
Performance

• A number of tools developed in-house to monitor
  the status of the network
• E.g., port, backplane, CPU utilization, device
  reachability, L2/L3 traceroute, DMZ traffic
  statistics
• No components are currently approaching saturation

13
(No Transcript)
14
(No Transcript)
15
Retrospective

• The network design has proved to be scalable,
  highly available and provides high bandwidth in a
  secure environment
• Some lessons learned
• be careful with management and configuration of
  layer-2 switching and spanning trees (e.g., watch
  for VTP configuration issues) and with CGMP/IGMP
  multicast support
• do not attempt to mix standards (e.g., Cisco ISL
  and IEEE 803.1Q) even when theoretically possible
• adequate security and openness are often
  conflicting requirements
• Open issues
• effective monitoring (e.g., SPAN) and data
  gathering in a high-speed switched environment
• secure SNMP monitoring (SNMPv3 support) and
  secure access to the network devices (SSH)