Exchange Point Technology Nanog 20, Oct 2224, 2000

1
Exchange Point TechnologyNanog 20, Oct 22-24,
2000

• Lane Patterson
• Member Research Staff
• ltlane_at_equinix.comgt

2
Intro Framework
3
IX Technology Universe

• 10/100 Ethernet, FDDI
• ATM OC3/12 (SAR limitations)
• GigE
• Private Copper/Fiber Cross Connect
• SONET ADM/DCS
• Frame Relay on POS OC48/192
• LSR (over POS, Ethernet, )
• 10GigE
• Optical Switching (w/ signaling of some sort)

4
IX Connection Models

• Layer 2 Non-Broadcast Multi-Access (NBMA)
• TE and connection-oriented more control at the
  expense of management and operational overhead
• Frame Relay or ATM PVCs
• MPLS LSPs
• Even 802.1p/q in a VLAN per /30 model
• Layer 2 Shared Media
• FDDI, Ethernet, DPT/SRP
• Layer 1
• SONET DCS style STS-1, OC-N
• Wavelength, Dark Fiber
• Nailed Up TDM Today GMPLS tomorrow?

5
IX Applications Current and Potential

• Unicast Peering Aggregation
• Private or Public
• Multicast Peering Aggregation
• Interdomain Interconnection Policy
• Human Enforced Clue
• Route Servers
• Interdomain MPLS/GMPLS?
• QoS Brokering?
• Not focusing outside of interconnection in this
  talk (e.g. Stratum 1 servers, other services)

6
What Were Doing
7
Gigabit Ethernet

• The one big Unicast VLAN model
• But private VLANs and tagged interfaces OK
• Must Mitigate Shared Risk
• IX Policy and Enforcement
• Switch Filters and Knobs
• Cost, simplicity, and operational advantages over
  ATM
• More traffic control features now on GigE lowers
  the contrast with ATM
• MTU a problem?
• Lots of core Internet MTU debate
• But more router data helpfulflow cache packet
  distributions, frag stats, better analysis of
  packets that are gt1500, etc.
• Waiting on vendor support MTU per VLAN tag
• Will be announced as a product
• Im the RD guy not the product guyusual
  non-Nanog channels apply

8
GigE Shared Risks

• No Brainers Participants
• ARP no ip proxy-arp
• Broadcast no ip directed-broadcast
• ICMP Redirects no ip redirects
• CDP/other noise no cdp enable
• No IGP passive interface
• No 3rd party switches, BPDUs
• Multicast keep off unicast exchange VLAN

9
GigE Shared Risks (contd)

• IX-side
• STP tuning a must
• Upcoming 802.1w and STP improvements from todays
  2-4 second reconvergence to milliseconds
• Block BPDUs on non-IX-trunk ports
• Layer 2/3 filters per port
• Trend and alarm bridge tables ( and identity of
  MACs on each port)
• Some controls still needed that are easy
• Some controls may not be worth it
• Too much maintenance overhead for ISPs
• Static ARP, MAC per PORT/VLAN
• Shaping/Policing policies
• Filter Maintenance
• If we really want this, go back to NBMA models

10
Future GigE Developments

• Vendors are active
• 10GigE
• Other proprietary features for control, scaling
• Active 802.1 projects
• Still Need Multicast IX Features
• PIM snooping for Port/Group state

11
What Were Tracking/Testing
12
SONET Cross-Connects

• Aggregate private circuits through SONET DCS
• Assumes OC48/192 Ports Channelizable to OC-N
  (both on Router and DCS)
• Recent DCS Products up to 512 OC48 down to
  STS-1
• Worthwhile for co-located routers?
• Already precedent FloridaMIX
• Good combo with DWDM into building
• Extends current SONET aggregation at DS3 OC12
• Cost of SONET DCS ports vs. Other Alternatives
• Policed Ethernet dot1q alternatives
• Cost/density/flexibility trade-offs versus
  continuing to burn DS3/OC3/OC12 router ports
• Will report back on future testing progress
• Input is welcome

13
NBMA Migration Paths

• Frame Relay at OC48/OC192 most promising ATM
  Upgrade today
• Mature standards
• Switches becoming available
• MPLS LSRs need more momentum
• Still intra-domain focused and limited set of
  ISPs
• Inter-domain signaling issues
• Promising longer term for new inter-provider
  features
• MPLS/BGP VPN
• Inter-provider QoS
• Dynamic B/W provisioning circuit-em, signaling
• None of these proven yet

14
NBMA Migration Paths (contd)

• Equinix committed to more MPLS LSR Testing
• Current Inter-Domain Models
• Non-terminating IX LSR
• Static Labels
• RSVPERO but no IGPstatics across
  directly-connected IX /30s
• Terminating IX LSR
• Add CCC-style to above combinations
• Need more robust Inter-Domain Signaling models
• MBGP NLRI for label exchange a first step
• Bilateral policy directly on LSRs or real 3rd
  party gateway infrastructure?
• Same issues carry over to GMPLS and optical world

15
Optical Switching

• Dense numbers of signaled wavelength and fiber
  cross connects
• Testing one OXC vendor near term
• Testing open to interested parties
• Still early in development
• For now, wavelength exchange is just a private
  cross connect between customer-owned DWDM
  equipment
• Fiber/wavelength density and signaling-driven
  applications will drive this onto OXCs
• Then Bill Norton can do some whiz-bang cost
  benefit graphs
• Multiservice Exchange

16
Virtual Device Trends

• or lack of them
• Useful for customer-driven, on-demand
  provisioning
• Today not many real virtual devices, just NMS/OSS
  integration through secure web front-end.
• Future boxes that can push this
• 3rd party policy brokers
• Register policy
• Integrate statistics for operation, billing
• Admission control in secure signaling domain
• Real virtual devices
• Log in to your virtual instance on a shared box
• Bandwidth trading systems

17
Going Forward

• Large IX Facilities
• Focus on removing space, power, and
  interconnection constraints from scaling equation
• Continued ability to accept new participants
• Only limit should be strandswavelengths into the
  building/campus
• Today,
• Public Fabric 10/100/1000 Ethernet moving to
  10Gig
• Private Fabric dark fiber cross connects
• Under Consideration SONET cross-connects and
  aggregation onto OC12/48/192

18
Last Thoughts

• When future cores are 80 wavelengths of OC192,
  and migrating to 320 wavelengths of OC768, what
  do exchange points need to look like?
• Other than more bandwidth, what inter-domain
  services will take hold?
• Will we have any hair left by then?