FAST TCP

1
FAST TCP

• Speaker Ray
• Veune Room 1026
• Date 25th October, 2003
• Time 1000am

2
Motivation

• Demand for ultrascale networking
• HENP (High Energy and Nuclear Physics)
• Data volumes of tens of Petabytes (1015) to
  Exabytes (1018)
• Require Terabit/sec (1015 bit/sec or
  1000Gbit/sec)
• Scalability problem of TCP
• Losses must be extremely rare
• TCP must induce loss
• Underutilization and oscillation

3
Scalability problem of TCP extremely loss packet
loss possibility

• Rate 1.3 MTU / (RTT sqrt(Loss))
• MTU 1500bytes, RTT 10ms

4
Scalability problem of TCP inevitable packet loss

• TCP needs to create losses
• Single bit network feedback signal

5
Scalability problem of TCP Underutilization and
oscillation

• AIMD (1, 0.5)
• At large window size (in excess of 10,000 pkts)
• Halving window on loss event is too drastic
• Increasing window by one packet per RTT is too
  conservative

6
FAST TCPAchievement

• CERN (European Organization for Nuclear Research)
  sent 1.1 Terabytes of data at 5.44 Gbps
• Full-length DVD film in 7 seconds !!

7
FAST TCP

• Flow based vs Packet based
• Network delay vs Packet loss
• TCP-Vegas vs TCP-Reno
• Stabilized Vegas

8
TCP VegasTechniques

• New Retransmission Mechanism
• Congestion Avoidance Mechanism
• Modified Slow-Start Mechanism

9
TCP VegasNew Retransmission Mechanism

• Timeout
• n duplicate ACKs

10
TCP VegasNew Retransmission Mechanism

• Check time record for the first duplicate packet
• non-duplicate ACKs first or second after
  retransmission
• Catch other segment lost previous to
  retransmission

11
TCP VegasCongestion Avoidance Mechanism

• Detect network delay by monitoring RTT
• BaseRTT and ActualRTT

12
TCP VegasCongestion Avoidance Mechanism

• Expected WindowSize / BaseRTT
• Diff Expected - Actual
• Diff 0, decrease sending rate
• Diff 0, increase sending rate
• a

13
TCP VegasCongestion Avoidance Mechanism

• Extra buffers occupied
• BaseRTT 100ms, segment size 1KB
• Expected WindowSize / BaseRTT
• a 30KB/s, ß60KB/s
• a 30KBps 100ms / 1KB 3
• ß 60KBps 100ms / 1KB 6

14
TCP VegasCongestion Avoidance Mechanism

• aß
• Diff
• increase one segment per RTT
• Diff a
• no change in windows size
• Diff a
• decrease one segment per RTT

15
TCP VegasSlow-Start Mechanism

• TCP-reno
• Send two segment for each ACK received
• Exponential growth every RTT
• TCP-Vegas
• Exponential growth every alternative RTT
• ?threshold
• Diff ?
• Changes from slow-start mode to linear I/D mode

16
Stability of TCP Vegas Network model

• Set of L links with finite capacities c
• c (cl , l ? L)
• N sources indexed by r
• Each source r uses a set of link defined by the L
  ? N routing matrix
• Rlr

• if source r uses link l
• 0 otherwise

17
Stability of TCP VegasNetwork model

• For each link l, the congestion measure pl(t) is
  call price
• For each source r, it maintains a rate xr(t) in
  packets/sec
• Equilibrium forward delay from source r to link l
  ?
• Equilibrium backward delay from link l to source
  r ?

18
Stability of TCP VegasNetwork model

• Aggregate price source r observes in its path
• qr(t) ? Rlr pl (t - ? )
• Aggregate source rate link l observes
• yl (t) ? Rlr xr (t - ? )

x1(t)
p1(t)
p3(t)
p4(t)
x2(t)
p2(t)
l
r
19
Stability of TCP VegasNetwork model

• Tr denote equilibrium RTT
• ? ? Tr, ? l ? L
• Dynamical system of TCP Vegas
• ?pl (t) ( yl ( t ) cl ) / cl if pl (t) 0
• ( yl ( t ) cl ) / cl ) if pl (t) 0
• ?xr (t) 1/Tr 2(t) sgn( 1 xr(t)qr(t) / ?rdr )
• Tr (t) dr qr( t )
• Where sgn(z) 1 if z 0, -1 if z z 0
• (z) max 0 , z

20
Stability of TCP Vegas Approximate model

• ?xr (t) 1/Tr 2(t) sgn( 1 xr(t)qr(t) / ?rdr )
• sgn(z) ? 2/? tan-1 (?z)
• ? ?? ?
• ?xr (t) (2/?Tr 2(t))tan-1 ?(1 xr(t)qr(t) /
  ?rdr )

21
Stability of TCP Vegas Approximate model

• In equilibrium, the source rate xr and aggregate
  price qr satisfy
• xr qr ?r dr

22
Stability of TCP Vegas Theorem 1

• Suppose for all r, k0Tr ?? maxr Tr for some k0.
• Let M be an upper bound on the number of links in
  the path of any source, M ? maxr ?l Rlr.
• The Vegas model is locally asymptotically stable
  around the equilibrium point (xr , yl , pl ,
  qr ) if
• maxr xr Tr sinc ? (n / xr Tr )
• n 2?/?
• Let ?(a) be the unique solution in ( 0, ?/2) of ?
  tan? a as a strictly increasing function of a
• sinc ? sin? /?

23
Stability of TCP Vegas Theorem 1

• maxr xr Tr sinc ? (n / xr Tr )
• ?() is strictly increasing
• sinc() is strictly decreasing
• LHS is strictly increasing in windows size xr Tr
• Theorem 1 Stability condition impose a limit on
  max windows size

24
Stability of TCP Vegas Corollary 2

• maxr xr Tr sinc ? (n / xr Tr )
• All source has the same target queue length, ?r
  dr ? for all r
• Corollary 2 LHS is strictly increasing in qr /
  Tr , implying a lower bound on queueing delay

25
Stability of TCP Vegas Corollary 3

• Since ?() 2 / ?, k0 ? 1
• Corollary 3 minr qr / Tr 2M / ?
• If M?? 2, then RHS bigger than 1, since Tr dr
  qr
• M 1
• The stability condition cannot be satisfied if a
  source has more than one link
• Sufficient in multilink case
• Necessary and sufficient in single-link-homogeneou
  s-source case

26
Stability of TCP Vegas Single link with
homogeneous source

• A single link of capacity c,
• Shared by N homogeneous source,
• with round trip propagation delay d

27
Stability of TCP Vegas Single link with
homogeneous source

• From corollary 3 qr / Tr 2 / ? for all r
• Tr / qr
• d / qr d
• Since qr ? / xr (? N)/c
• cd
• Conclusion bandwidth delay product should be
  small

28
Stabilized Vegas

• ?xr (t) (2/?Tr 2(t))tan-1 ?(1 xr(t)qr(t) /
  ?rdr )
• ?xr (t) (w/Tr 2(t))tan-1 ?r(t)(1
  xr(t)qr(t)/?rdr -??r(t) ?qr(t))
• 1 xr( t ) qr( t ) /?r dr
• 1 xr( t ) qr( t ) /?r dr -??r( t ) ?qr( t )
• ?r( t ) (1 / ?) ( Tr( t ) / qr( t ) )
• ?r( t ) ( ??? / w ) ( xr( t ) Tr( t ) )

29
Stabilized Vegas

• The gain ?r( t ) serves as a normalization to
  ?qr( t )
• Additional differential term ??r( t ) ?qr( t )
  anticipates the future of qr( t )
• Without xr( t ) will increase if xr(t)qr(t)?rdr
• Even xr(t)qr(t)/?rdr is small, xr( t ) may
  decrease if prices are rapidly growing

30
Stabilized Vegas

• Stability condition for stabilized Vegas
• where ? tan-1 ( (2??)/(1-?) )
• Stabilized Vegas can choose a small
• ( a0, ??(0,1) ) such that RHS can be larger
  for better stability of the original Vegas cd (?/2 1) ? N

31
Simulation Results
One-on-One (300KB and 1MB) Transfers c
200KB/s 50ms delay
32
Simulation Results

• ? 20
• N 100 flows
• Fixed packet size of 1KB
• FIFO /w Droptail, queue capacity 20000
• ( a , ? ) ( 0.5 , 0.015 )

33
Simulation Resultsc 100 pkts/s and d 10ms
34
Simulation Resultsc 1000 pkts/s and d 10ms
35
Simulation Resultsc 100 pkts/s and d 100ms
36
Simulation Results
37
Experimental Results

• FAST TCP was first demonstrated publicly in
  during the SuperComputing Conference (SC2002) in
  Baltimore, MD, in November 1622 2002
• Caltech-SLAC research team
• CERN
• DataTAG
• StarLight
• TeraGrid
• Cisco
• Level(3).

38
Experimental Results
39
Experimental ResultsThroughput and utilization

• SC2002 FAST experimental result
• Current TCP implementation in Linux v2.4.18 on
  Jan 27-28, 2003

40
FAST TCPConclusion

• Problem of current TCP
• Equilibrium
• Dynamic problem
• FAST TCP
• Equation-based control with queuing delay
• TCP Vegas
• Stabilized Vegas

41
QA
42
Thank you