Networks with Very Small Buffers

1
Networks with Very Small Buffers
Yashar Ganjali, Guido Appenzeller, High
Performance Networking Group
Prof. Ashish Goel, Prof. Tim Roughgarden, Prof.
Nick McKeown
The current Rule of Thumb
Sum of congestion windows is Gaussian
How small buffers can be?

• Date-rate of most flows small fraction of line
  rate
• Access links throttle flows to low rate (1-2Mb/s)
• CoreAccess gt 10001
• Todays TCPs window size is limited
• If packet arrivals were Poisson
• We get 80-90 utilization with 5-10 packet
  buffers
• In an all-optical network capacity is plentiful
  (presumably)

• For many flows, TCP windows are multiplexed
• Buffer size is reduced
• Sum of congestion windows is Gaussian
• Reason the central limit theorem
• Width of Gaussian is dependent on square root of
  number of flows
• Buffer has to be wide enough to fit Gaussian

• A router needs a buffer size
• 2T is the two-way propagation delay
• C is the capacity of bottleneck link
• Context
• Mandated in backbone and edge routers
• Appears in RFPs and IETF architectural guidelines
• Usually referenced to Villamizar and Song High
  Performance TCP in ANSNET, CCR, 1994
• Already known by inventors of TCP Van Jacobson,
  1988

What we know so far about very small buffers?
Complete Centralized Control
Arbitrary Injection Process
If Poisson Process with load lt 1
Theory
Experiment
Constant fraction throughput with constant
buffers Leighton
Any rate gt 0 need unboundedbuffers
Need buffer size of approx O(logD
logW) Dof hops Wwindow size Goel 2004
TCP Pacing Results as good or better than
forPoisson
Our new rule for sizing buffers
Validation

• A router needs a buffer size
• 2T is the two-way propagation delay
• C is the capacity of bottleneck link
• n is the number of long-lived flows through the
  router
• Why? What has changed?
• High-end routers serve many unsynchronized flows
• Statistical multiplexing reduces the amount of
  buffering needed

Fast access links, TCP Reno vs. Paced TCP

• Theoretical results validated by
• Thousands of ns-2 simulations
• Network lab (Cisco routers) at University of
  Wisconsin
• Stanford University dorm traffic
• Internet2 experiments
• Buffer size is reduced
• Ongoing work with network operators and router
  vendors .

Congested core router RTT 100ms Each flow
limited to 2.5Mb/s
What determines router buffer size?

• TCP needs buffers to fully utilize a link
• Only with maximum congestion window the links is
  full
• Window scales down just after it reaches maximum
• The main role of a router buffer is to compensate
  for the difference in outstanding packets
• TCP window size changes over time (the TCP saw
  tooth)
• Number of outstanding packets changes over time
• Buffer has to buffer these packets

Slow access links, lots of flows
Congested core router RTT 100ms Each flow
limited to 2.5Mb/s

• Example 40 Gb/s linecard with 40,000 x 1Mb/s
  flows
• Rule-of-thumb Buffer 10Gbits
• New rule Buffer 50Mbits