Checksums and Packet Reordering

1
Checksums and Packet Reordering

• Sridhar Machiraju
• March 21st, 2001
• Network Reading Group, UCB

2
Checksums

• Measurements and Analysis of End-to-End Internet
  Dynamics, V. Paxson, Ph.D. thesis, 1997, UCB.
• When the CRC and TCP Checksum Disagree, J. Stone
  and C. Partridge, Sigcomm 2000.

3
Paxsons Thesis

• Since packet contents were not captured, bit
  errors were inferred.
• LBL(?) had a large share of (bursty!) checksum
  errors. It used SLIP compression (CSLIP). It is
  better to have a strong datalink checksum. (More
  on this later)
• About 1 in 5000 TCP packets had a corrupt
  checksum.
• 16-bit TCP checksum implies that 1 in 300 million
  packets have incorrect checksum.

4
Paxson (contd.)

• Data packets had more errors than ACKs. Why?
• ACKs are smaller in length than data packets
  (assuming uniform bit-errors).
• Also, IP checksum would take care of integrity of
  the IP header (which accounts for a larger
  portion of the packet with ACKs).
• Likelihood of desynchronization increases with
  packet length and if link is point-to-point.

5
Stone and Partridge

• 1 in 7500 packets passed the receiver datalink
  CRC but failed the TCP checksum.
• Contradict Paxsons observation non-data
  packets had as many errors as data packets!
• Errors occurred in end-host software, router
  memory, network-interface hardware etc.
• Consecutive datagrams sent on links implementing
  Van Jacobson compression had wrong checksums due
  to framing errors and dropped packets.

6
Stone et al. (contd.)

• ACK-of-FIN bug in Windows NT was also noticed.
• About 1 in O(1 billion) packets has an error
  undetected by checksum!

7
Conclusions

• Be wary of header compression.
• (E2E) Application-level checksums are very
  necessary remember the NFS story see the Feb,
  2002 e2e mailing list archives?
• Hardware cannot be trusted.
• Note that application-level checksums are
  probably needed and IPSec would not be sufficient.

8
Packet Reordering

• Packet Reordering is not Pathological Behavior,
  Bennet, C. Partridge, N. Shectman, IEEE/ACM TON,
  Dec 1999.
• On Making TCP More Robust to Packet Reordering,
  E. Blanton and M. Allman, ACM Computer
  Communication Review, Jan 2002.

9
In 1995,

• Paxson observed reordering in 1994/1995 due to
  different paths.
• Another explanation a router gets a route
  update and forwards the existing queue of packets
  whenever possible!
• Bennet et. al. performed a study in 1998.
• They made extensive observations at the Mae-East
  peering point.

10
In 1998,

• At the Mae-East exchange,
• Wide variety of equipment, hosts.
• Well-known topology and traffic statistics.
• Heavily loaded site.
• Sent bursts of ICMP messages to hosts
  topologically close to Mae-East.
• The result was that 90 of the hosts experienced
  reordering!
• Reason Use of Parallel Links between ISPs at the
  heavily loaded Mae-East exchange.

11
A tale of 2 ISPs
12
Hunt Groups in Switches

• The manual for the DEC GigaSwitch says that
  tcprexmtthresh should be 100!
• Reordering occurs because of a parallel set of
  links (called hunt groups) between 2 switches and
  Head-Of-Line Blocking.
• This could be eliminated in output-buffered
  router and input-buffered routers too but the
  problem is cost and complexity.
• Question Can such routers be built today?

13
IP/TCP layer Solutions?

• Hash all packets of a flow and send them on only
  one physical link.
• Same solution applies for multiple ASICs too.
• Question How bad will the utilization be if
  hashing is not symmetric.
• TCP SACK option could be used to detect
  reordering and increase tcprxmtthresh (more on
  this later).

14
Router Architectures

• Why not number packets and reorder them? The
  algorithms are non-work conserving.
• A Reliable and Scalable Striping Protocol, H.
  Adiseshu, G. Parulkar, and G. Varghese, ACM
  Sigcomm 96.
• Unsolved problem bit-by-bit striping.
• The appendix proves that under a set of
  assumptions, absence of reordering and
  work-conserving are not both achievable.

15
Impact of reordering on TCP

• Blanton et. al. completely simulation-driven
  study.
• Spurious (fast) retransmits.
• Spurious retransmission also causes suspension of
  RTT estimation.
• With fast retransmit, reordering is interpreted
  as packet loss and sending rate is decreased.
• Make TCP bursty because an ACK will acknowledge
  receipt of gt1 segments.
• Reordering inflates RTT estimates. Is this a
  problem, considering the 100s of ms granularity
  of TCP clocks.

16
Detection of Unnecessary Retransmits

• Use of the D(uplicate)SACK option.
• The receiver sends information about previous
  reordering too.
• If the RTO expires and the ACK for the lost
  packet arrives soon, it is an ACK for the
  original packet.
• Use of Eifel retransmission timer.

17
Possible Solution

• The impact of reordering reduces as the amount of
  reordering increases.
• The authors consider the use of DSACK option a
  retransmission is noted along with the SACK
  option.
• One solution The cwnd is not decreased if no
  actual loss occurred. But this does not prevent
  spurious retransmissions.
• The rxmtthresh needs to be modified based on the
  amount of reordering.
• Tradeoffs Experiencing timeout versus converging
  to optimal rxmtthresh quickly.