Network softwarehardware interaction a case study

1
Network software/hardware interaction -- a case
study

• V. Karamcheti and A. A. Chien, Software Overhead
  in Messaging layers Where does the time go? ACM
  ASPLOS-VI, 1994.
• A case study on the communication performance
  issues on the CM5 machine.

2
Background

• The network requirement for a typical high
  performance computing user
• In-order message delivery
• Reliable delivery
• Error control
• Flow control
• Deadlock free
• Where should these functions be realized?
• Network hardware? Network systems? Or a
  hardware/systems/software approach?

3
Background

• Where should these functions be realized?
• How does the Internet realize these functions?
• No deadlock issue
• Reliability/flow control/in-order delivery are
  done at the TCP layer?
• The network layer (IP) provides best effort
  service.
• IP is done in the software as well.
• Drawbacks
• Too many layers of software
• Users need to go through the OS to access the
  communication hardware (system calls can cause
  context switching).

4
Background

• Where should these functions be realized?
• High performance networking
• Most functionality below the network layer are
  done by the hardware (or almost hardware)
• This provide the APIs for network transactions
• If there is miss-match between what the network
  provides and what users want, a software
  messaging layer is created to bridge the gaps.

5
Messaging Layer

• Bridge between the hardware functionality and the
  user communication requirement
• Typical network hardware features
• Arbitrary delivery order (adaptive/multipath
  routing)
• Finite buffering
• Limited fault handling
• Typical user communication requirement
• In-order delivery
• End-to-end flow control
• Reliable transmission

6
Messaging Layer
7
Communication cost

• Communication cost hardware cost software
  cost
• Hardware message time msize/bandwidth
• Software time
• Buffer management
• End-to-end flow control
• Running protocols
• Which one is dominating?
• Depends on how much the software has to do.

8
What this study did

• Analyzing the software cost in the CM5 machine.
• Investigating what overhead can be reduced if the
  underlining network provide higher level of
  service.

9
CM-5 Network hardware

• Send store the dest node number and data in the
  NI send buffer
• Receive load from the NI receive buffer
• NI status is queries by loading the control
  registers

10
CM-5 Network hardware

• Out-of-order delivery (adaptive routing)
• Nodes, NI, and the network have finite buffering
• Error detection, but not error correction.
• Fixed packet size of five 32-bit words

11
CM-5 active message layer (CMAM)

• Active message
• A message with a small amount of computation at
  the receiving end.
• Each message contain an address of a user-level
  handler which is executed on message arrival with
  the message body as an argument.
• The handler extract data from the network and
  integrate it into the ongoing computation.
• CMAM vs Send/Recv
• User direct access to network, no OS involve, the
  data go directly to the user space
• CMAM can be considered as some kind of software
  RDMA.

12
Software overhead cost analysis

• Consider implementation of 3 protocols
• Single-packet delivery
• Finite sequence, multi-packet delivery
• Indefinite sequence, multi-packet delivery
• Use instruction counts for measurement

13
Single-packet delivery
14
Finite sequence, multi-packet delivery
Packet transfer (4), buffer management (1, 2, 3,
5) Fault-tolerance (6), in-order delivery (extra
inst. In 4).
15
Indefinite sequence, multi-packet delivery
In order delivery sequence (store/send/read) Fa
ult-tolerance (1) and acks.
16
Message size 16 words
17
Message size 1024 words
18
What do we see in the study?

• The mis-match between the user requirement and
  network functionality can introduce significant
  software overheads (50-70).

19
Messaging layer with high-level network feature

• Given that CMAM is considered to be very
  efficient, there are 2 choices to reduce software
  overhead
• Lower user requirement
• Raise level of service provided by the network
• Compressionless Routing (CR)
• Order-preserving transmission
• Deadlock freedom independent of packet acceptance
  guarantees
• Fault-tolerant transmission at packet level

20
Single-packet delivery

• Has the same cost as the previous CMAM case
• However, it is guaranteed to be fault-free, no
  deadlock or buffer overflow

21
Finite sequence, multi-packet delivery

• No buffer allocation messages
• No overhead for in-order delivery
• No end-to-end acks.

22
Indefinite sequence, multi-packet delivery
23
Message size 16 words
24
Message size 1024 words
25
Discussion

• Larger packet sizes
• Reduce overhead, especially for
  indefinite-sequence protocol
• Improved network interfaces and DMA hardware
• Network interface
• only make basic cost faster, but not protocol
  cost in messaging layer
• Make it more important for messaging layer to be
  effective
• DMA
• only reduce cost in moving large amounts of data

26
Discussion (Cont.)

• Implication of network design
• Improving routing performance may increase
  software cost, e.g. out-of-order delivery
• Providing low level features to applications
• Put burden on parallel software programmers
• problematic

27
Conclusion

• In the design of the communication system,
  holistic understanding must be achieved
• Focusing on network hardware may not be
  sufficient. Software overhead is much larger than
  routing time.
• It would be ideal for the network to directly
  provide high level services.