Optimizing TCP Forwarder Performance

1
Optimizing TCP Forwarder Performance

• IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 8, NO.
  2, APRIL 2000
• ???? M9129018
• ???

2
Outline

• Introduction
• TCP Forwarding
• Connection Splicing
• Connection Splicing In SCOUT
• Conclusion

3
Introduction
4
Introduction

• TCP forwarder
• A network node that establishes and forwards data
  between a pair of TCP connection
• TCP forwarding
• Indirect TCP communication via a proxy
• Connection splicing
• Improve TCP forwarding performance

TCP forwarder
TCP connection
TCP connection
5
TCP Forwarding
6
TCP Forwarding

• Proxy
• Mediate the communication
• Interpose between two connection
• Control the flow of data between the
  communicating parties
• Proxy has two mode
• Control mode
• Forwarding mode

Control Mode
Forwarding Mode
Control Mode
Back to control mode
Processing control function
Move data between connection
7
TCP Forwarding (cont.)

• Proxy can be classified into four categories
• First
• In control mode only during connection setup
• After connection setup, switch to forwarding
  mode for the duration of connection
• Second
• Authenticate the user or request
• Check user ID, password , and destination of the
  Telnet request

FTP Proxy
Telnet Proxy
8
TCP Forwarding (cont.)

• Third
• Remains in control mode for all data transferred
  in one direction (HTTP proxy)
• Switch to forwarding mode for data transferred in
  the other (HTTP server)
• Fourth
• Remains in control mode and continuously monitors
  data passed in both directions

HTTP Proxy
Proxy
9
Firewall

• Data from one network pass through the proxy
  which forwards them to the other network
• If the desired security guarantees are not
  violated

10
Mobile Computing

• Filtering data
• Reduce or remove too big data
• When mobile host is connected to wired network
• Only relay data in forward mode
• Allow a mobile host to change its point of
  attachment to network
• Mobile host can terminate TCP connections
• Move to new location with a new IP address
• Establish a new set of TCP connections to proxy

11
Connection Splicing
12
Connection Splicing

• The basic idea of connection splicing
• To detect when a proxy makes a transition from
  control mode to forwarding mode
• Splice two TCP connections together into a single
  forwarding path through the system

Unoptimized TCP forwarder
Optimized TCP forwarder With spliced connection
13
Forwarding

• Primary task on FWD processing step
• Change the header of incoming TCP segment to
  account for the differences in the two original
  TCP connections

14
Forwarding (cont.)

• From connection A to connection B
• Output.DstPort RemotePortB
• Output.SrcPort LocalPortB
• Output.SeqNum Input.SeqNum SeqNumOffsetA-gtB
• Output.Ack Input.Ack SeqNumOffsetB-gtA
• Output.Cksum Input.Cksum CksumPatchA-gtB

Connection A
Connection B
TCP forwarder
15
Splicing

• TCP buffers contain acknowledged data
• Forwarder cant let TCP acknowledge new data
• Give it more data to deliver reliably
• Impractical to wait until two connections go idle
  before completing the splice

16
Splicing (cont.)

• Two way to handle newly arriving segment during
  transition period
• Delay the activation of spliced connection until
  after buffers have drained
• TCP acknowledge segments
• After transition is complete, buffered segments
  are processed by FWD
• Allow FWD to begin forwarding data concurrently
  with draining the buffers
• All newly arriving segments are delivered to both
  the original TCP protocol and to FWD

17
Unsplicing

• When the forwarding proxy switches from
  forwarding mode to control mode, connections must
  be unspliced
• Difficult to decide when proxy should switch back
  to control mode
• Proxy has to find control information by looking
  at out-of-order segments

18
Unsplicing (cont.)

• Dealing with acknowledgements makes it difficult
  to unsplice a connection
• No acknowledged segment
• Reconstruct TCP connections
• Acknowledged segment
• Wait for all of segments be acknowledged
• Continuously monitor segment stream until copy
  all unacknowledged segments

19
Flow Control

• During unoptimized operation
• Flow control is handled by two independent TCP
  protocols on forwarder, and TCP protocol on the
  end hosts
• During optimized operation
• Flow control is handled by the end host only
• TCP forwarder can restrict window size to avoid
  unnecessary retransmissions

20
Additional Optimizations

• Connection splicing optimization can be applied
  not only at TCP level, but also to unfragmented
  IP datagram
• Forwarder can process IP datagrams similarly to
  an IP router, with additional TCP segment header
  manipulation

21
Connection Splicing in SCOUT
22
Connection Splicing In SCOUT

• SCOUT is a configurable OS explicitly designed to
  support data flow
• Video streams through an MPGE player
• A pair of TCP connections through a firewall

23
2-Path

• As going from one path to another often will
  require a context switch
• Like firewall structure

24
1-Path

• Similar to 2-path configuration, except two
  network devices are connected by a single path

25
FWD

• Optimized version of 1-path
• Splice into a single connection forwarder is
  reduced to updating TCP header
• Support reassembly of IP packets

26
IP/FWD

• Further Optimized version of FWD
• Network level packets are modified directly and
  forwarded
• Dont support reassembly of IP packets

27
IP Router

• Modify network packets directly in the same way
  as IP/FWD
• Not update TCP header

28
In Linux Configuration

• TIS firewall
• Offer full filter functionality, but use a null
  filter
• Filtering IP router
• Filtering on IP addresses, protocol port number
• Like IP/FWD case in SCOUT
• IP router
• Basic in-kernel Linux IP forwarding with no
  filtering

29
Test Setup

• 200MHz PentiumPro workstation
• 256KB cache, 128MB RAM
• Digital Fast EtherWORKS PCI 10/100 32-bit PCI
  10/100 MB/s adapters
• Linux version 2.0.30

30
Processing Overhead

• Back-to-back latency network interface latency

31
Processing Overhead (cont.)

• Summarizes the processing of a single packet in
  firewalls and routers for both SCOUT Linux

32
Aggregate Throughput

• Measure aggregate throughput of one, two, and
  three concurrent TCP connections over 2-path
  IP/FWD
• Packet is 1460 bytes in 100Mbit Ethernet

Mbyte/S
33
Cost of Unsplicing

• First
• Fix up TCP header during spliced operation
• FWD keeps track of SN, ACK number, window of
  spliced TCP connection
• Second
• Determine when to unsplice
• Third
• Require to initiate two TCP state machine
• Last
• Impact on end-to-end throughput

34
Conclusion
35
Conclusion

• Connection splicing is a good idea, but it
  doesnt tell us how to implement

36
THE END
37
(No Transcript)
38
Cost Of Splicing

• TCP sequence number trace showing the effects of
  the SCOUT implementation of splicing

39
Connection Splicing

• An optimization technique that improves TCP
  forwarding performance
• Basic idea of connection splicing
• To detect when a proxy makes a transition from
  control mode to forwarding mode
• And then splice the two TCP connections together
  into a single forwarding path through the system

40
Optimizing two TCP connectionsinto a single
spliced connection (1)

• Unoptimized TCP forwarder
• Require TCP segments to traverse TCP twice, with
  each instance of TCP maintaining the full state
  of the connection

41
Optimizing two TCP connectionsinto a single
spliced connection (2)

• Optimized TCP forwarder (with spliced connection)
• Replace the proxy and two TCP processing steps
  with a single FWD processing step
• FWD maintains just enough state to forward TCP
  segment successfully from one network to another

42
Flow Path of TCP Forwarding

• TCP forwarding starts in the unoptimized
  configuration
• When proxy shifts from control to forwarding mode
• Makes a transition to optimized configuration
• When TCP forwarding back to control mode
• Revert back to the unoptimized configuration

43
Three Cases To Consider

• Optimized TCP forwarder in the steady state
• Unoptimized TCP forwarder becomes optimized TCP
  forwarder
• Optimized TCP forwarder back to unoptimized TCP
  forwarder

44
Forwarding (1)

• The primary task of FWD processing step
• Change the header of incoming TCP segment to
  account for the difference in the two original
  TCP connections
• If TCP connection establishment was interleaved
• One connection knew what port and sequence
  numbers were used by other connection
• Additional optimization are possible

45
Forwarding (2)
46
Forwarding (3)

• When forward connection A to connection B
• Port Number
• TCP forwarder operate as a classical proxy
• Source and destination port numbers of segments
  arriving on A have to be changed to the port
  numbers of connection B
• TCP forwarder is a transparent proxy
• Proxy uses the same port numbers
• Output.DstPort RemotePortB
• Output.SrcPort LocalPortB

47
Forwarding (4)

• Sequence Number
• TCP initializes SN randomly for each independent
  connection
• The SN for an outgoing segment is computed by
  adding a fixed offset to the SN in the incoming
  segment
• Output.SeqNum Input.SeqNum SeqNumOffsetA-gtB

48
Forwarding (5) ?????????

• Acknowledge Number
• ACK number acknowledges SN forwarded in the other
  direction
• ACK number in a outgoing segment is computed by
  subtracting from the SN in the incoming segment,
  the SN offset for segments flowing in the other
  direction
• Output.Ack Input.Ack SeqNumOffsetB-gtA
• In my opinion
• Output.Ack Input.Ack SeqNumOffsetA-gtB

49
Forwarding (6)

• Checksum
• Modifying the other fields require adjusting the
  TCP checksum
• Output.Cksum Input.Cksum CksumPatchA-gtB

50
Forwarding (7)

• In the unspliced case
• Segments sent to proxy are put to Incoming TCP
  stack
• Check if they can reach their destination
• Data are buffered in outgoing TCP stack until
  they are acknowledged by the destination
• In the spliced case
• No longer traverse the two TCP protocol stack
• Not acknowledge proxy, nor resend data to
  destination

51
Splicing (1)

• The real problem is transitioning from the
  unspliced state to the spliced state

52
Splicing (2)

• Acknowledged data must be reliably delivered to
  their destination
• During the time the data are being drained,
  however, new segments may arrive
• Forwarder cant let TCP acknowledge new data
• Impractical to wait until two connection go idle
  before completing the splice

53
Splicing (3)

• Two ways to handle newly arriving segments during
  this transition period
• Delay activation of spliced connection until
  after the buffers have drained
• This solution may drop data if FWD buffers
  overflow while TCP buffer are being drained
• Allow FWD to begin forwarding data concurrently
  with draining the buffers
• All newly arriving segments are delivered to both
  the original TCP protocol and to FWD
• Cause data to be delivered out-of-order

54
Splicing (4)

• Before packet processing can be altered
• Computering SN offset checksum patches for FWD
• SN offset can be calculated as soon as all
  acknowledged data have been drained from
  forwarder buffer
• Checksum patch can be calculated as soon as the
  other offset known

55
Unspliced (1)
56
Flow Control
57
Additional Optimizations
58
Other Issues
59
Connection Splicing In SCOUT
60
(No Transcript)
61
2-Path
62
1-Path
63
Proxy For Unoptimized Forwarding

• Detect a transition to forwarding mode
• Stops processing incoming segments allows
  segments to accumulate in the paths input queue
• Unlinks two TCP stages proxy stage from the
  path replaces them with a

64
FWD
65
IP / FWD
66
IP Router

• Modifies network packets directly in the same way
  as IP/FWD
• Not support reassembly of IP packets

67
Compare With Linux

• TIS Firewall
• Trusted Information System
• TIS firewall toolkit offers full filter
  functionality
• Use a null filter
• Filtering IP Router
• The in-kernel Linux IP forwarding has support for
  filtering on IP address, protocol number port
  number
• Closest thing in Linux to IP/FWD case in SCOUT

68
Compare With Linux

• IP router
• Basic in-kernel Linux IP forwarding with no
  filtering

69
Test Environment

• 200 MHz PentiumPro workstation
• 256 KB cache
• 128 MB ram
• Digital Fast EtherWORKS PCI 10/100 32-bit PCI
  10/100 Mb/s adapter
• Linux 2.0.30

70
Test Environment