User Level Communications and Fast Messaging Layers

1
User Level Communications and Fast Messaging
Layers

• G?????? Sta?????

2
Presentation Layout
Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
3
Introduction and Problem Specification
Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
4
Message Passing
Sender Memory
Receiver Memory
User Address Space
User Data
User Data
Message Passing Library
Message Passing Library
Kernel Protocol Stack
Kernel Protocol Stack
Kernel Address Space
NICs address space
NICs address space
5
Communication Layers in a Typical OS

• TCP protocol offers in order, reliable data
  transmission
• UDP protocol offers unreliable data transmission

Socket API
User Level
Transport Layer ( TCP / UDP )
Network Layer ( IP )
Kernel Level
Network Programming Interface (D.L packets)
Device Driver
6
Communication Layers in a Typical OS (cont.)

• TCP/IP is by far the most popular protocol suite
• Local Area Networks are getting Faster...
• ... but Applications throughput was not raised
  at same rate
• Was it TCPs fault?

7
Measurements and Comparison 5
8
Overhead Analysis

• An Analysis of TCP Processing Overhead in 1989 by
  Clark et al. 2 shows that TCP itself is a
  flexible protocol (proved also highly scalable)
• Overhead is mainly generated at
• Bad Implementation of TCP/IP suite
• Several Copies of data in Memory
• Timer usage and CRC computation

9
Overhead Analysis (cont.)

• ...But the main overhead of protocol is though
  the actions need to be taken by OS not the code
  of protocol itself

10
Approaches for Faster Communications
Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
11
Approaches for Faster Communications

• In order an application to exploit the raw
  network bandwidth different ways has been
  followed
• Specialized Hardware Solutions
• Software Optimization Solutions

12
Specialized Hardware Solutions

• Networks of Parallel Machines
• Virtual Architecture Interface (VIA)
• Myricoms Myrinet NIC and Switches

Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
13
Software Optimization Solutions

• Efforts to optimize the overhead factors of
  TCP/IP
• Implementation and Usage of lightweight
  proprietary protocols

Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
14
1. Optimization of TCP/IP overhead factors
Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
15
TCP Overhead Factors

• Overhead is mainly generated at
• Bad Implementation of TCP/IP suite
• Several Copies of data in Memory
• Timer usage and CRC computation

16
Memory Access Overhead
Memory

• Two memory copies for user data is much overhead
• For small packets (like NFS traffic) found that
  25 of packet processing time spent in memory
  copies

User Process Pages
1.Copy User Data to mbufs structure in Kernel
Kernel Pages
2.Copy Packet to Devices address space
17
Memory Access Techniques

• User accessible interface memory
• Kernel-Network shared memory
• User-Kernel shared memory
• User-Kernel page remapping with Copy on Write
  (COW)

NIC
CPU
Memory
Memory
Cache
18
Timer and Checksum

• Faster Timer implementations must developed
• Checksum may be avoided whenever possible (p.e
  UDP )

19
Checksum Overhead
20
2. Lightweight Protocols
Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
21
Common Message Path
User Process

• Traditionally protocols are implemented inside
  kernel

Kernel Communication Protocols
NIC
22
Implementation of Lightweight Protocols

• If we develop a proprietary protocol it isnt
  always easy to put it in kernel because
• Bugs may pop at anytime
• Debugging of kernel is difficult and prevents
  other users from using the workstation
• Kernel code isnt always available

23
Implementation of Lightweight Protocols (cont.)

• Also it isnt always efficient to put them in
  Kernel because
• Context Switch between User Level and Kernel is
  expensive
• Memory Copies from User Space to Kernel Space is
  expensive

User Process
User Buffers
Kernel Communication Protocols
Kernel Buffers
24
User Level Communications
User ProcessProtocol Library

• A Solution to those problems is implementation of
  protocols as Communications Libraries, utilized
  directly by User processes

Kernel
NIC
25
User Level Communications Problems

• Address Translation, since User processes knows
  Virtual Addresses
• Control Transfer without kernel intervention
• Protection, since each User accesses NIC directly
• Efficient utilization of NIC between processes

26
Implementation Survey
Introduction and Problem Specification
Approaches for Faster Communications
Specialized Hardware Solutions
Software Optimization Solutions
2. User Level Communications
1.TCP optimization
Implementation Survey
27
User Level Implementations 1, 4

• Active Messages
• AM-II
• Fast Messages
• VMMC
• BIP
• Fast Sockets
• SHRIMP

• U-Net
• Hamlyn
• Trapeze

28
Active Messages 3

• Very popular lightweight protocol
• Uses Handlers to place packet data in User Space

29
Fast Sockets 6
User ProcessFast Sockets API

• Fast Sockets presents traditional sockets API to
  User Application

Active Messages
NIC
30
References

• R. A.F. Bhoedjang, T.Ruhl and H. E. Bal,
  User-Level Network Interface Protocols, IEEE
  Communications Magazine, pages 53-60, Nov. 1998.
• Clark, D.D., Jacobsen, V., Romkey, J., Salwen,
  H., "An Analysis of TCP Processing Overhead, "
  IEEE Communications Magazine, Vol. 27, No. 6,
  1989.
• T. von Eicken, D. E. Culler, S. C. Goldstein and
  K. E. Schauser, Active Messages A mecchanism
  for integrated communication and computation, In
  The 19th Annual International Symposium on
  Computer Architecture, pages 256--266, Gold
  Coast, Australia, May 1992
• T. von Eicken and W. Vogels, Evolution of the
  Virtual Interface Architecture, IEEE
  Communications Magazine, pages 61-68, Nov. 1998.
  
• J. Kay and J. Pasquale, Mesurement, Analysis and
  Improvement of UDP/IP Throughput for the
  DECstation 5000, In Proceedings of the Winter
  1993 USENIX Conference, pages 249--258, San
  Diego, CA, 1993.
• S. H. Rodrigues, T. E. Anderson and D. E. Culler,
  High-Performance Local Area Communication with
  Fast Sockets, In Proceedings of Usenix Annual
  Technical Conference, 1997.