Netzwerktechnik FHS Kufstein Tirol

1
NetzwerktechnikFHS Kufstein TirolGetting the
most out of the Internet
Michael Welzl http//www.welzl.atDistributed
and Parallel Systems Group Institute of Computer
Science University of Innsbruck
2
Both sides of the story...

• End users want
• Efficient Internet data transfer
• e.g., 100 Mbit/s should really be 100
  Mbit/s!(not always true! e.g. theoretical vs.
  real wireless bandwidth)
• application (video, audio, ..) quality should be
  good
• Cheap service
• Service providers want
• Money!
• Save money efficient use of existing capacity
• Earn extra money provide special services with
  guarantees(e.g., video conferencing)

Thus, two major parts 1. Efficient End-to-End
Internet Data Transfer2. Quality of Service
3
Outline

• A brief networking (Internet) overview
• Efficient End-to-End Internet Data Transfer
• Problem statement Internet properties and our
  goal
• IP, router architecture, the congestion problem,
  heterogeneous infrastructure, traffic properties
• Instruments what can be done?
• end system
• measure and change! rate, packet size,
  application data, flow / congestion control
• router
• communicate with end system (e.g. ICMP), control
  packets in queues
• Solutions how are the instruments applied?
• TCP congestion control, Path MTU Discovery, ALF,
  special mechanisms for special flows (SCTP, DCCP,
  UDP Lite), RED, ECN, ..
• Quality of Service
• IntServ / RSVP, DiffServ, QoS Routing, Traffic
  Engineering, MPLS, MP?S, Internet2 QBone
• Lessons learned and the future of Internet QoS

4
1. Networking a brief overview

• Network layers, ISO/OSI vs. DoD model,
  terminology, the Internet

5
Logical communication flow
draw a green rectangle
6
Physical communication flow
language (protocol)
draw a green rectangle
C H A O S
translation
translation
Network(Internet Cloud)
Choose a path
check for logical errors
check for bit errors
01001010111101010010101110100010101100001111010001
0111001011
7
Abstraction

• Programming Languages
• Machine code
• Assembler
• Low level languages
• High level languages (special purposes),OO
  Design, ..
• Simplification by abstraction
• Same for networks network layer models

8
ISO/OSI Reference Model

• THE famous layer model
• 7 layers
• precise terminology, huge amount of theoretical
  work
• layer provides service to upper layers
• strict rules (layers must not be skipped, but
  they are interchangeable)

9
OSI Architecture

• ISO Basic Reference Model for Open Systems
  Interconnection (OSI)
• OSI model is concept, architecture, common
  terminology.

10
OSI Layers

• Application oriented layers7. Application
  Layer actual communicating apps6. Presentation
  Layer semantics5. Session Layer structured
  dialogue - synchronisation, ..
• Transport oriented layers4. Transport Layer
  end2end msg. stream, no knowledge of routing3.
  Network Layer routing, packets between adjacent
  systems(LANs 2b Logical Link Control L.2a
  Medium Access Control)2. Data Link Layer
  error-recovery, frames (not packets) stream
  between adjacent systems1. Physical Layer
  unsecure bitstream between adjacent systems

11
The most successful (packet) network is...(and
the winner is...)
The Internet
12
How the Internet has changed

• Original goal robust communication on a
  long-term basis
• Original size ARPANET...

13
How the Internet has changed

• Currently 69 new hosts added each minute!
• IEEE Spectrum, Feb. 2001
• Commercial demand for new, accordingly priced
  services(VoIP, streaming audio/video,
  videoconferencing, ..)
• Overprovisioning does not suffice - demands
  increase
• Goal is not just speed / reliability
  anymoreInternet "best effort" service is not
  good enough

14
Common Internet myths misconceptions

• The network automatically finds the best path for
  my packets.
• IPv6 is brand new, provides QoS and will be
  widely deployed soon.
• The Internet changes with incredible speed.
• Overprovisioning can solve all QoS problems.
• Somewhat less commonInternet2 will soon replace
  the Internet as we know it.
• World Wide Web Internet.
• The Internet exists since ...
• TCP/IP is (are) the Internet protocol(s).

15
Primary reason for tremendous success
end2end argumentmove complexity to higher
layers in the stack...important Internet design
rule!
Example security Security provided by the
network may be too bad or unnecessary for some
applications. The network cannot provide
authentication only the application has access
to the necessary information
Today, scalability remains the no. 1 goal
16
DoD reference model

• DoD-model older (approx. 1969) than OSI (approx.
  1977)
• Only 4 layers
• Application Layer
• Transport Layer
• Internet Layer ( layer 3, Network layer)
• Network Access Layer ( everything underneath the
  Internet)
• Layers 5 and 6 missing
• Less restrictive - layers may be skipped

17
OSI, DoD and reality

• OSIMany service functions carried out in
  several layers / services? Overhead, even
  reversal!
• Why should I implement layers 5, 6 in my app /
  OS ?
• Commercial failure - but still useful to explain
  networks
• DODactually obsolete Internet defined by
  TCP/IP stack(defined by RFCs)

18
TCP/IP Protocol Stack

• IP addressing, routing, fragmentation/reassembly,
  TTL
• UDP ports, checksum
• TCP UDP connection-oriented service
  (retransmission/ACK),combined flow control /
  congestion control

19
A shaky invariant the Internet Hourglass
Everything Over IP
IP Over Everything
20
IP Routing Domain level (IGP's)

• Distance Vector Routing RIP, routers only
  keepinformation about adjacent routers
• Link State Routing OSPF, IS-IS - each router
  knows about the whole domain (less scalable!)
• based on routing metric, usually no. of hops -
  not always optimal

21
Interdomain routing (EGP's)

• BGP Distance Vector Routingbetween "Autonomos
  Systems"(registered unique AS number)
• Metrics often configuredmanually according to
  costs
• Peering relationshipsusually cheaper than
  thebackbone
• Implementation filteronly accept routes
  fromspecific sources

22
Who defines the Internet?

• Preliminary research in IRTF ( http//www.irtf.org
  )
• Standards (RFCs) defined by IETF (
  http//www.ietf.org ) -mostly Working Groups
• Decisions by IESG (as of Feb. 2001, 14 elected
  members)
• IAB stimulates IETF / IESG actions
• RFCs have different status standard, proposed
  standard, draft standard, experimental,
  informational, ..
• Internet-draft preliminary - may turn into RFC

23
Who really defines the Internet?

• RFCs define the Internet. Only standard RFCs
  mandatory
• Extremer views, but maybe closer to reality
• Jon Postel defined the Internet.
• Cisco defines the (core) Internet.
• Microsoft defines the (end2end) Internet.
• Some standards never made it.Some other things
  did (MBone, NAT, P2P, ..)

24
Standards that never really made it(and probably
never will)

• TTL as an actual time value
• The (originally planned usage of the) IP TOS
  field
• Several IP options Strict / Loose Source
  Routing Record Route Timestamp
• Source Quench

25
Who runs the Internet?

• (See http//www.caida.orgfor more pictures)
• Hierarchical structure
• Distributed ISP admins run it
• No Internet "police"
• Each ISP interested in his/her own services, but
  end2end paths may include the backbone
• CPU cycles scarce in core routers

26
If you really want to know how it works

• ... check out
• http//www.warriorsofthe.net

important links http//www.ietf.org/ http//www.i
rtf.org/
27
2. Efficient End2End Internet Data Transfer

• Problem statement, terms, instruments (what can
  be done?), solutions (how are the instruments
  applied?)

28
Problem statement

• Efficient transmission of data streams across the
  Internet
• various sources, various destinations, various
  types of streams
• What is efficient?
• terms latency, end2end delay, jitter,
  bandwidth(nominal/available/bottleneck -),
  thoughput, goodput, loss ratio, ..
• goals high throughput (bits / second), low
  delay, jitter, loss ratio
• Note Internet TCP/IP based world-wide network
• no assumptions about lower layers!
• ignore CSMA/CD, CSMA/CA, token ring, baseband
  encoding, frame overhead, switches, etc. etc. !

typically bit/s at this level!
29
Bandwidth

• Mirriam-Webster online ( http//www.m-w.com )
• Main Entry bandwidth, Pronunciation
  'band-"width
• Function noun, Date circa 1937
• 1 a range within a band of wavelengths,
  frequencies, or energies especially a range of
  radio frequencies which is occupied by a
  modulated carrier wave, which is assigned to a
  service, or over which a device can operate
• 2 the capacity for data transfer of an
  electronic communications system ltgraphics
  consume more bandwidth than text doesgt
  especially the maximum data transfer rate of
  such a system

Traditional, real definition!
Information rate
30
Bandwidth (2)

• Common interpretation in CN contextHow many
  bits/sec can be transferred ("how thick is the
  pipe)
• Consider modem connection vs. van of DVDs
  traveling an interstate highway
• Unit definition 1 - Hz, definition 2 - bit/s
  (bps)
• And baud?
• CNET.com glossaryhttp//www.cnet.com/Resources/In
  fo/Glossary/Terms/baud.html"Most people use baud
  to describe modem speeds in bits per second--but
  they're wrong. They may say a 9,600-bps modem
  transmits at 9,600 baud, but really baud is a
  measure of how frequently sound changes on a
  phone line. Modern modems transmit more bits with
  fewer changes in sound, so baud and bps numbers
  aren't equal. However, only editors, pedants, and
  communications engineers now care about the
  distinction. But if you run into members of these
  groups, use bps instead of baud.

31
Wooly but common bandwidth related terms (and
their usual interpretations)

• Nominal bandwidth
• Bandwidth of a link when there is no traffic
• Available bandwidth
• (Nominal bandwidth - traffic) ... during a
  specific interval!
• Very important, but perhaps most ambigous term of
  all
• Bottleneck bandwidth
• smallest nominal bandwidth along a path
• often also smallest available bandwidth along a
  path
• Throughput
• bandwidth seen by the receiver
• Goodput
• bandwidth seen by the receiving application (e.g.
  TCP goodput ! throughput)

32
A simple router model
packet too large?? fragment!
Switching Fabric
In 1
Out 1
Queue 1
In 2
Out 2
In 3
Queue 2

• Switching fabric forwards a packet (dest.
  addr.) if no special treatment necessary
  fast path (hardware)
• Queues grow when traffic bursts arrive
• low delay small queues, low jitter no
  queue fluctuations
• Packets are dropped when queues overflow
  (DropTail queueing)
• low loss ratio small queues

33
Delay / Bandwidth

• Delay / Bandwidth related, but not
  interchangeable
• On the Internet, delay changes indicate (path
  changes or) congestion? TCP Vegas
• Congestion related to bandwidth ? delay related
  to bandwidth!
• but

similar delay, different available bandwidth!
34
Delay

• Latency - time to transfer an "empty" message
• also propagation delay
• limit speed of light!
• End2end delay latency msg_length /
  bottleneck bandwidth
• queuing delay
• processing delay can also play a role,esp. in
  core routers (CPU cycles scarce resource!)
• Jitter - delay fluctuations, very critical for
  most real-time applications
• Round-trip time (RTT) - time a messages needs to
  go from sender to receiver and back

maximum theoretical earth distance 66.8ms (lag
gt 50ms audible!)
The time spent in queues
35
Ambiguities

• mbit / mb / Mb / Mbit ... ?
• Latency sometimes end2end-delay
• In general make sure you know what layer you
  are talking about!
• link physical connection between one or more
  hosts or routers, or link between IP routers (may
  consist of multiple physical links!)
• capacity often physical capacity, but different
  if you talk about TCP
• Typical commercial WLAN term theoretical
  bandwidth(nominal bandwidth at physical as
  opposed to application layer)

36
The congestion problem

• Congestion control necessary
• adding fast links does not help!

total throughput w/o cc. 20kb/s total throughput
w/ cc. 110kb/s
37
Congestion collapse
cliff
knee

• Goal operation at the knee !

38
Internet traffic characteristics

• MRTG trace (based on SNMP, accessing traffic
  counters in MIB)

39
Internet traffic characteristics /2

• Traditional traffic modelling queuing
  theorynotion traffic follows poisson
  distribution
• Internet traffic is bursty - intuitive reasons
• TCP is bursty by nature (congestion avoidance,
  payload vs. acks - later!)
• ACK compression can cause payload bursts due to
  ACK-clocking (later!)
• various packet sizes
• Bursts from queues aggregate as traffic traverses
  the net
• Burstiness of one flow affects other adaptive
  flows

Still true for user arrival !
40
Internet traffic characteristics /3

• Overlapping of independent on-off sources leads
  to distribution with heavy-tailed autocorrelation
  function
• Long-range dependance "peaks sit on ripples
  which sit on waves"
• No "flattening" towards a mean as you zoom out -
  same structures may be found at different time
  scales, hence self similar
• Traffic characteristics sometimes modeled with
  time series (fARIMA models) or wavelets
• Measurement of the "degree of self similarity"
  Hurst parameter
• -gt model approximation involves Hurst parameter
  estimation
• Calculations extremely difficult -Internet
  analysis mostly based on simulation!

41
Instruments

• What can be done?

42
IPv4 Header
Examples Loose / Strict Source Routing, Record
Route, Timestamp, Router Alert
43
End systems Measure...
well studied- leads to misinterpretation
of transmission errors

• throughput ("goodput")
• ( mean, fluctuations,
• packet loss ratio..)

easy to measure independent of transmission
errors- not practical without throughput
delay( rtt, one way delay, jitter..)
similar delay, different available bandwidth!
44
... and change!

• lower layers
• throughput (gap between packets)
• flow control / congestion control
• well studied, many options - our main interest!
• packet size
• large recommendedless overhead
• small less impact of transmission
  errors,smaller latency!
• protocol

Note packet size granularity ofthroughput
measurements

• content
• compression
• hierarchical encoding
• FEC

Common difficultiesbandwidth known (depending
on content)?granularity of rate changes
45
End2end bandwidth adaptation difficulties
Bandwidth
Time
T0
T1
46
Flow Control Methods

• Remember refers to next msg-no to be
  treated!!
• Version described at left
• credit C-A is fixed-size window
• C-A pushed by ACKs
• S, R move within window
• sender keeps copies of A thru S-1
• Common design flaw
• Rcv. wants to slow down sender? holds back ACK
• ? timeouts, retransmits
• Remedy, cf. TCP
• TCP separates ACK / window size (credit) per
  packet
• _______________________________
• Rate control
• S, R negociate data rate
• may be slowed down by either party
• No explicit feedback necessary

• Sliding window flow control
• most widely used, many variants

47
Possibilities in routers

• Communicate with end systems
• alter header flags (IP only! should not look at
  other layers)
• generate signaling packets Internet Control
  Message Protocol (ICMP)(mainly error messages)
• Control packets in queuesuse queue length or
  position in queue to
• communicate (mark)
• drop, move to other queue etc.
• Problems
• CPU cycles scarce in (core) Internet routers
• Scalability! (e.g., no per-flow state)example
  ICMP Source Quench failed(congestion
  notification in times of congestion)

48
Solutions

• How are the instruments applied?

49
Path MTU Discovery

• MTU Maximum Transfer Unit (largest packet that
  will not be fragmented)
• MTU refers to link Path MTU (PMTU) minimum MTU
  of a path
• Algorithm
• set IP dont fragment flag
• start with big packets
• gradually decrease size upon ICMP Destination
  Unreachable - Fragmentation Needed reply
• layer 3 functionality - may be initiated from
  layer 4
• TCP problem with arbitrary packet drops
• Path MTU Discovery Black Hole Detection
  problemNo ICMP messages from unresponsive
  routers or filtered by firewalls...hard to
  detect and solve!
• proposed solution start with small packets and
  increase
• do not rely on ICMP messages!

50
Internet congestion control History

• 1968/69 dawn of the Internet
• 1986 first congestion collapse
• 1988 "Congestion Avoidance and Control"
  (Jacobson)Combined congestion/flow control for
  TCP
• Goal stability - in equilibrum, no packet is
  sent into the network until an old packet leaves
• ack clocking, conservation of packets principle
• made possible through window based stopgo -
  behaviour
• Superposition of stable systems stable ?
  network based on TCP with congestion control
  stable

51
TCP Congestion Control /1 Tahoe, 1988

• Distinguish
• flow control protect receiver against overload
• (receiver "grants" a certain amount of data
  ("receiver window") )
• congestion control protect network against
  overload
• ("congestion window" (cwnd) limits the rate
  min(cwnd,rwnd) used! )
• Flow/Congestion Control combined in TCP. Several
  algorithms
• (window unit SMSS Sender Maximum Segment Size,
  usually adjusted to Path MTU init cwndlt2
  (SMSS), ssthresh usually 64k)
• Slow Start for each ack received, increase cwnd
  by 1(exponential growth) until cwnd gt ssthresh
• Congestion Avoidance each RTT, increase cwnd by
  SMSSSMSS/cwnd(linear growth - "additive
  increase")

52
TCP Congestion Control /2

• If a packet or ack is lost (timeout, roughly
  4rtt), set cwnd 1, ssthresh current
  bandwidth / 2(multiplicative decrease") -
  exponential backoff
• Several timers, based on RTT good estimation is
  crucial!
• Later additions(TCP Reno, 1990)Fast retransmit
  / fast recovery (notify sender of loss via
  duplicate acks)

Congestion Avoidance
Slow Start
53
Background AIMD
54
Active Queue Management

• Today, TCP behaviour dominates the Internet (WWW,
  ..)
• example backbone measurement 98 TCP traffic
• 1993 Random Early Detection ("Discard", "Drop")
  (RED)(now that end nodes back off as packets are
  dropped, drop packets earlier to avoid queue
  overflows)
• Another goal add randomization to avoid traffic
  phase effects!
• Qavg (1 - Wq) x Qavg Qinst x Wq(Qavg
  average occupancy, Qinst instantaneous
  occupancy, Wq weight - hard to tune,
  determines how aggressive RED behaves)

55
Active Queue Management /2

• Based on exponentially weighted moving average
  (EWMA) of instantaneous queue occupancy low
  pass filter
• recalculated every time a packet arrives
• Qavg below threshold min_th Nothing happens
• Qavg above threshold min_th Drop probability
  rises linearly
• Qavg above threshold max_th Drop packets
• RED expects all flows to behave like TCP - but is
  it fair?
• Variants drop from front, drop based on
  instantaneous queue occupancy, drop arbitrary
  packets, drop based on priorities...

56
Explicit Congestion Notification (ECN)

• 1999 Explicit Congestion Notification
  (ECN)Instead of dropping, set a bit
• End systems are expected to act as if packet was
  dropped-gt actual communication between end nodes
  and the network!
• ATM and Frame Relay not only ECN but also BECN
• Internet BECN often proposed and regularly
  discussed (ICMP SQ), but very unlikely - several
  reasons
• Idea charge more when ECN flag is set
• ECN cannot totally replace loss measurements!

57
Congestion pricing

• Basic idea higher sending rate more congestion
  higher price
• Idea charge more when ECN flag is set
• Smart Market idea
• each packet bids for capacity
• out of n bids, m highest bids can be accepted
• price marginal cost (highest bid of unaccepted
  packets) ? leads to market equilibrium
• Smart Market not practical (bidding per
  packet), but shows
• balance of demand and supply leads to market
  equilibrium
• stable system just based on economics suffices
  for congestion control
• Problem link cannot know bids in advance ? no
  guarantees

58
TCP in heterogeneous environments

• Timeout interpreted as congestion
• was bad idea in times of error-prone networks
• seems reasonable in times of fibre networks
• absolutely insane for wireless links!
• TCP over long fat pipes large bandwidthdelay
  product
• long time to reach equilibrium, MD problematic!
• Different implementations / extensions
• TCP SACK - explicit notification of missing
  segments
• TCP over wireless - checksum error ? do not
  immediately drop packet ? not interpreted as
  congestion
• TCP over sat (also "Interplanetary Internet")
  larger windows, ..

59
Wireless TCP

• Various experimental enhancements
• split connection at base station
• monitor connection at base station, buffer
  interfere (Snoop TCP)
• Note ECN is not affected by link noise!

60
UDP and UDP Lite

• UDP IP 2 features
• Ports identify communicating instances with
  similar IP address(transport layer)
• Checksum Adler-32 covering the whole packet
• checksum field 0 no checksum at all! bad idea!
  (why?)
• ? solution UDP Lite (length checksum
  coverage)
• advantage e.g. video codecs can cope with bit
  errors, but standard UDP throws a whole packet
  away!
• critical apps depending on UDP Lite can depend
  on lower layers
• usefulness often, link layers do not hand over
  corrupt data
• Usage of UDP unreliable data transmission (DNS,
  SNMP, real-time streams, ..)

61
End2end real-time data transfer

• Assumption no special service available at
  application level
• (Definition of Internet "real-time" softer than
  usual)
• Different requirements
• reliable service may not be needed (no
  retransmission)
• Timely transmission important
• Different treatment
• no retransmission / waiting for ACKs
• no sliding window (stop go behaviour not
  feasible)
• but
• some kind of flow control still needed
• synchronization necessary
• often Multicast

62
Real Time Protocol (RTP)

• Can be used on top of anything, but usually UDP
• adds a timestamp
• supports payload type identification
• adds a counter to preserve the original packet
  order
• Intermediate systems
• Translator converts encodings, filtering
• Mixer synchronizes multiple streams based on RTP
  timestamps
• Real Time Control Protocol (RTCP)
• Receiver reports provide feedback (packet loss,
  interarrival jitter)
• Sender reports similar to RRs, issued if receiver
  isalso a sender

Regular network end nodes ? overlay network model!
63
Other related protocols

• XTP
• rate-based protocol for multimedia
  transport(instead of allowing a certain amount
  of data, inform the sender about the allowed data
  rate)
• no time synchronization / timestamps
• Multicast (overlay)
• Application level protocols (mainly for ease of
  use)
• Real Time Streaming Protocol (RTSP)
• Application level protocol supporting VCR-like
  commands
• Play, Forward, Rewind, Pause, Stop, Record
• Just signaling - no payload transmission (usually
  done with RTP/UDP)
• Not connection-oriented, can be used on top of
  TCP or UDP
• Hypertext Transport Protocol (HTTP)
• Can be used to control streaming multimedia
  (browser plug-ins, ..)

64
Data encoding / application layer signaling

• RTP content type definitions encapsulate various
  formats
• H.323 protocol suite
• big, covers various layers (e.g., H.263 video)
• encoding formats for video, audio
• originally designed for ISDN (CBR)
• SDP, SAP, SIP
• newer than H.323, improved scalability
• session signaling similar to http request (ascii)
• designed to co-exist with RTP/RTCP in the
  Internet
• MPEG
• Fluctuating data stream designed for efficient
  encoding
• Semantics in MPEG7

Compare DivX downloads ? streaming video
65
Multimedia adaptation

• Mistake
• adaptation schemes assume arbitrary data stream
  scalability
• Problem
• Data streams show fluctuations (example MPEG I-,
  B-, P-frames)
• Solution
• Special CBR design for communication - H.261
  designed for ISDN
• not possible or feasible in all cases
• Problem
• compression usually not deterministic - size
  depends on content!
• real-life distance learning example40kbps
  enough for streaming video (Smartboard) audio
  (speech), but speech suffers dramatically if
  teacher visible

66
Fairness

• ATM ABR Max-Min-fairness
• A (..) allocation of rates is max-min fair iff
  an increase of any rate (..) must be at the cost
  of a decrease of some already smaller rate.
• One resource mathematical definition satisfies
  "general" understanding of fairness - resource is
  divided equally among competitors
• Usually requires knowledge of flows in routers
  (switches) - scalability problem!
• Internet
• TCP dominant, but does not satisfy
  Max-Min-fairness criterion!
• Ack-clocked - flows with shorter RTT react sooner
  (slow start, ..)and achieve better throughput
• Therefore, Internet definition of fairness
  TCP-friendliness"A flow is TCP-compatible
  (TCP-friendly) if, in steady state, it uses no
  more bandwidth than a conformant TCP running
  under comparable conditions."

67
Proportional Fairness
F. Kelly Network should solve a global
optimization problem (maximize log utility
function) Max-Min-fairness suboptimalS1 S2
S3 c/2
All link capacities c

• Proportional fairness
• An allocation of rates x is proportionallyfair
  iff, for any other (..) allocationy, we have
  (roughly approximated by
  AIMD!)

Proportionally fair allocationS1 c/3, S2 S3
2c/3
68
How to be TCP-friendly

• TCP-friendliness can be achieved by emulating the
  behaviourof TCP (or the desired parts of it)
• Simplified TCP AIMD (additive incr. ? ,
  multiplicative decr. ?)
• 0 lt ? , 0 lt ? lt 1 -gt stable and fair
  congestion control
• ? 4 x (1 - ?2) / 3 -gt TCP-friendly
  congestion control
• ? 1, ? 1/2 -gt TCP
• AIMD mechanisms for multimedia applications RAP,
  LDA
• Different approaches
• TCP Emulation At Receivers (TEAR)TCP
  calculations (cwnd calculation, fast recovery,
  ...) moved to receiver, do not ack every packet,
  smooth sending rate
• Binomial congestion control TCP-friendly,
  nonlinear control law

69
GAIMD congestion control

• Relationship between ? and ? for TCP-friendliness

more aggressive responsive
TCP
smoother
70
Equation based congestion control

• Based on TCP steady-state response function-
  gives upper bound for transmission rate T
  (bytes/sec)

• well known example TFRC - TCP-friendly rate
  control protocol
• smooth sending rate

71
Evaluation of TCP-friendly mechanisms

• Infocom 2001 paper by Yang, Kim, Lam examines
  simulations ofTCP (Reno) vs. GAIMD (?0.31,
  ?7/8) vs. TFRC vs. TEAR(bigger is better)
• Network simulator "ns-2
• Well known scenario compete on a single
  bottleneck link ("dumbbell")

72
...and performance?

• Parameters not very performance oriented
• Smoothness not very interesting if large buffers
  are available (one-way streaming)
• Despite efforts to identify non-adaptive
  flowsThe more you send, the more you get.
• Probably most interesting question beside
  smoothnessHow closely does the mechanism follow
  the available bandwidth?
• measurable parameter packet loss ratio

73
Not-so-TCP-friendly solutions

• Overcome rate fluctuationslimit encodings (e.g.
  2 or 3 qualities), let user decide
• Cross-media-adaptationchoose from video, audio,
  single pictures, text(e.g. MPEG7)
• Limit by bottleneck bandwidth
• often "last mile" - e.g. RealMedia
• better determine actual bottleneck via packet
  pair approach
• If wireless link involved small packets, UDP
  Lite
• Another possibility send more (do FEC) in
  response to packet loss
• (very network-unfriendly behaviour, but may yield
  less data loss)

Network stability some adaptation better than
none!
Example Jedi Knight 2 !
74
Some thoughts

• How TCP-friendly are 8 web browsers?
• Congestion Manager congestion control for all
  flows in OS core
• MulTCP Emulate multiple TCPs to provide
  differentiated services
• How TCP-friendly are short-lived flows?
  (web-traffic, ..)
• How to convince Internet multimedia app.
  programmers to implement TCP-friendly congestion
  control?
• Solution Datagram Congestion Control Protocol
  (DCCP)
• Well-defined framework for TCP-friendly
  congestion control
• Sender app chooses an appropriate congestion
  control mechanism
• Core OS implementation of mechanisms
• Lots of additional features nonces, partial /
  separate checksums (distinguish corruption ?
  congestion), ...

75
TCP header
76
TCP Connection Management
heavy solid linenormal path for a client The
heavy dashed linenormal path for a
server Light linesunusual events

• Service provided by TCPCongestion
  controlled,reliable, sequentially ordered data
  streambetween two entities(port no. IP addr).
• What else could we want?

77
Multihoming

• Original Internet intention robust communication
  network (military)
• Trend provide telephone-like service
• Phones have been around for a long time
• Easy to use, always works
• Metaphor (SIP) call somebody on the net
• Paths rarely change provide a circuit
• http//www.yahoo.com request for information
  (yahoo web site)
• Metaphor call yahoo wrong!
• Better dissemination of information (mirrors,
  web caches), transparent to the user ? stress
  reduction, faster service, location dependent
  services
• (Application Layer) Multihoming use the net for
  distributed data retrieval

78
Multihoming /2

• Location dependent services
• Adapt content based on language, culture
• find people with similar interest in this place
  (mobile client)
• Exclude surfers from certain locations (law
  enforcements, ..)
• New issue geographical location tracking(DNS
  extensions, time measurements, ..)
• Problem efficient information dissemination
• Routing

79
Application Level Framing

• TCP byte stream oriented protocol
• Application may want logical data units
  (chunks)
• Byte stream inefficient when packets are lost

• ALF app chooses packet size chunk size
• packet 2 lost no unnecessary data in packet
  1,use chunks 3 and 4 before retrans. 2 arrives
• 1 ADU (Application Data Unit) multiple chunks
  -gt ALF still more efficient!

80
Multiple Data Streams

• Application may use multiple logical data
  streams(e.g. pictures in a web browser)
• Common solution multiple TCP connections
• separate flow / congestion control (Congestion
  Manager?)

Chunk 2
Chunk 1
Chunk 2
Chunk 1
Unnecessary retransmission!
81
SCTP (Stream Control Transmission P.)

• Stream-based, connection oriented, reliable
  transmission protocol (TCP features)
• Decoupling of reliable and ordered delivery
• Unordered delivery eliminate head-of-line
  blocking delay
• Support for multiple data streams (per-stream
  ordered delivery)
• Application Level Framing
• TCP congestion control (SACK ECN)
• Hearbeat keep-alive mechanism

82
SCTP /2

• Multihoming at transport layer! (layer 3
  functionality)
• Goal robustness
• switch hosts upon routing failure
• eliminate effect of routing reconvergence time
• CRC-32 checksum instead of Adler-32
• TCP connection ? SCTP association
• 2 IP addresses, 2 port numbers ? 2 sets of IP
  addresses, 2 port numbers
• DoS protecton avoid TCP SYN attacks
• 4-way handshake no state at receiver of INIT
  message
• state in digitally signed INIT-ACK message
  (cookie)

83
Unicast / Broadcast / (overlay) Multicast
84
Multicast issues

• Required for applications with multiple receivers
  only
• video conferences, real-time data stream
  transmission, .. ? different data streams than
  web surfing, ftp downloads etc!
• Issues
• group management
• protocol required to join / leave group
  dynamically Internet Group Management Protocol
  (IGMP)
• state in routers hard / soft (lost unless
  refreshed)?
• who initiates / controls group membership?
• congestion control
• scalability (ACK implosion)
• dealing with heterogeneity of receiver groups
• fairness
• Multicast congestion control mechanism
  classification
• sender- vs. receiver-based, single-rate vs.
  multi-rate (layered),
• reliable vs. unreliable, end-to-end vs.
  network-supported

depends on content!
85
3. Quality of Service

• "Managed unfairness"
• Service rules behaviours
• IntServ / RSVP, DiffServ, QoS Routing, Traffic
  Engineering, MPLS, MP?S, IPv6, Internet2 QBone

86
QoS and network layers
87
QoS and network layers /2

• QoS fundamentally an end-to-end issue...
• QoS spec. must not be violated at any layer
• QoS request may originate from (almost) any layer
• QoS provisioning may be demanded at (almost) any
  layer
• There is no overall framework -demand for QoS
  often leads to layer violation

88
QoS below IP

• LAN Medium Access Control (MAC) Layer
• CSMA/CD (Ethernet) behaviour practically
  unpredictable(collisions lead to Binary
  Exponential Backoff, calculations too
  complicated)
• Token passing schemes bandwidth / delay
  predictable
• WAN ATM-Layer (ATM has its own 3-dimensional
  model)
• ATM was the first serious QoS attempt - "ATM to
  the desktop"
• Constant cell size of (548) bytes enables Time
  Division Multiplexing-gt predictable data rate!

89
ATM Services
CBR (Constant Bit Rate) emulates a leased line
RT-VBR (Real-time Variable Bit Rate) for rt-streams w/ varying bandwidth such as MPEG
NRT-VBR (Non-real-time Variable Bit Rate) similar to RT-VBR, but more jitter is tolerated
ABR (Available Bit Rate) Cheap service - you do what you are told, get what is available and achieve a small cell loss ratio
UBR (Unspecified Bit Rate) Cheap, too no promises - best used by IP
90
ATM Services and Switching



91
ATM ABR

• ATM Explicit Rate Feedback (part of Available
  Bit Rate (ABR) service)
• RM (resource management) cells
• sent by sender, interspersed with data cells
  bits in RM cell set by switches
• NI bit no increase in rate (mild congestion),
  (EF)CI bit like Internet ECN
• two-byte ER (explicit rate) field may be lowered
  by congested switch
• sender send rate thus minimum supportable rate
  on path!

• Problem ATM failed (scalability? too much
  complexity in switches?)
• Experimental Internet approaches
• Multilevel ECN (two bits), eXpress Control
  Protocol (XCP), CADPC/PTP (my own)

92
ATM and reality

• ATM to the desktop dead
• Most often used for high-speed Internet links
  (backbone)
• Suboptimal for various reasons
• Cell size does not match packet sizes
• IP provides datagram service, no use for CBR
  etc.
• IP mostly used with UBR or ABR service in case
  of ABR,TCP is a control loop on top of a control
  loop!

93
Typical QoS requests
ATM Peak / Sustained / Minimum Cell Rate, Cell Delay Variation Tolerance, Cell Transfer Delay, Cell Error Rate, Cell Loss Ratio, ..
Layer 4 (distributed Multimedia app) Throughput, End2end Delay, Residual Error Rate (not (yet?) on the Internet!), Connection Establishment Delay / Failure Probability, ..
Layer 7 Transmission Security, Data Encoding Completeness, ..
Human Layer Perceived quality - "does it look good?", "does it feel controllable?", fun factor, ..
94
"Human Layer" QoS requirements
95
QoS Architectures

• Heidelberg QoS Model, OMEGA, int-serv, XRM
  (hierarchical), QoS-A and Tenet (3-dimensional),
  OSI, TINA, MASI, ..
• Various concepts related to layers (OSI, QoS-A),
  related to specific implementations (int-serv),
  ..
• Architectures identify fundamental concepts of
  QoS specification, provisioning, control and
  management
• No overall agreement on a single architecture

96
Generic QoS-capable router
InputInterfaces
OutputInterfaces
Meter
Policing / Admission Control Marking
Queuing Scheduling / Shaping
PacketClassification
SwitchFabric
Building blocks of modern QoS architectures
97
QoS router building blocks

• Packet Classification
• Group packets according to header properties
• Multiple fields (MF classification) needed to
  detect individual flowsip source / destination,
  protocol and port numbersproblems packet
  fragmentation (port numbers),header compression,
  encryption (IPSec)
• Meter
• Monitor traffic characteristics (e.g., does flow
  741 hold its promises?), provide information to
  other block(s)
• Policing
• Drop packets if certain conditions are fulfilled
• Admission Control
• React (not necessarily drop packets) if certain
  conditions are fulfilled
• Marking
• Mark packets (change header) if certain
  conditions are fulfilled
• for later special treatment - maybe not even in
  the same router

98
QoS router building blocks /2

• Switch(ing) Fabric
• Do a query on the routing table, decide where to
  send the packet
• Queuing
• If a packet cannot be delivered immediately
  (congestion),put in queue(s) for later delivery
• Decision which queue? Active queue management?
• Scheduling
• When to take a packet from which queue (e.g.,
  round robin)
• Shaping
• Adjust traffic characteristics if certain
  conditions are fulfilled(usually implemented in
  scheduling)
• Useful even without QoS provisioning Do not
  exceed max. promised quality - customers will get
  accustomed and complain!

99
Integrated Services (IntServ)

• Notionhard guarantees desired, per-flow
  resource reservation needed
• Two services defined
• Guaranteed Serviceguaranteed bandwidth, firm
  bounds on end-to-end queuing delaysto be used
  by real-time applications
• Controlled Loadclosely approximates the
  behaviour seen when there is (almost) no
  congestion to be used by elastic applications
• Architecture, Services / Reservation signaling
  protocol("Resource Reservation Protocol" - RSVP)
  design separated

100
IntServ per-hop requirements

• Classification
• per-flow context established via multifield
  classification
• flow context used to drive token-bucket metering

TokenGenerator
Marker, Policer, ..
Token
Token
Threshold
Token
Token
Token
Packet
Packet

• implemented as byte counter goal detect
  various degrees of burstiness
• several thresholds (also empty) with
  associated treatment possible!

IntServ traffic specification contains token
generation rate, bucket size
101
IntServ per-hop requirements /2

• IntServ token bucket metering leads to remarking
  or dropping(admission control)
• Multiple queues, one for each flow
• Implementation virtual queues - only one real
  queue per service
• Scheduler takes packets based on priorities
  (airline analogy)
• e.g., 1, 1, 2, 1, 1, 2, .. but not priority
  queuing (q1 until empty) - may cause starvation
  of q2!

• No bandwidth guarantees because of packet
  sizes!
• Solution Weighted Fair Queuing (WFQ), Class
  Based Queuing (CBQ)

102
Resource ReserVation Protocol (RSVP)

• Signaling - routers must know which flows to
  choose
• state in routers is established via PATH messages
  from sender
• Sender advertises allowed traffic spec via adspec
  messages
• Receivers initiate reservation (resv messages
  containing flow spec.)
• Multicast support, state merging

103
IntServ / RSVP discussion

• RSVP requires support by all routers(if
  unsupported, RSVP is tunneled - but no more hard
  guarantees)
• Scaling per-flow state not feasible!RSVP
  protocol not scalable either (maybe due to bad
  implementation)
• Strict guarantees per customer complicated
  accounting
• Solution "softer" QoS, no per-flow state in core
  routers - DiffServ

Best-Effort IntServ/RSVP DiffServ
QoS-Guarantees none flow-based aggregated
Configuration none dynamicend2end staticedge2edge
104
Differentiated Services (DiffServ)

• Edge routers Classifier / Meter / Marker /
  Shaper / Dropper
• Core routers static forwarding according to
  DiffServ-class, implementation may vary
• SLA Service Level Agreement between DS Domains

105
DiffServ terminology

• SLA contains non-technical aspects
• Service Level Specification (SLS)
• Parameters which determine theservice provided
  by a DS domain
• contains Traffic Conditioning Spec. (TCS),and
  other properties such as encryptionand routing
  constraints
• DiffServ Codepoint (DSCP) - IPv4 Precedence / TOS
  Bytes
• DSCP mapped to Per Hop Behaviour (PHB)
• how are packets treated in the core?
• Aggregated flows with same DSCP Behaviour
  Aggregate (BA)
• Distinguish PHB specification / implementation
• PHB Group PHBs that call for similar spec. /
  implementation

106
DiffServ details

• Edge routers MF and BA classificationbased on
  signaling, metering .. or ideas such as simply
  UDP / TCP
• Expedited Forwarding (EF) PHB
• "Virtual Leased Line" Service
• Aggregated flows must not exceed peak bandwidth
• Ingress Router Policing (dropping) Egress
  Router shaping
• Small delay - real time apps simple service
  model
• Unused bandwidth used by best-effort traffic!
• Assured Forwarding (AF) PHB Group
• Supports bursty flows
• Packets are marked with AF Class and Drop
  Precedence
• non-conforming packets are remarked

107
DiffServ details /2

• DiffServ does not define
• End2end service models
• Implementation details (PHBs, traffic
  conditioners, ..)
• But hints
• As in ATM ABR, "open" spec. leads to a lot of
  research work
• Implementation examples
• schedulers for PHB WFQ, CBQ, WRR (Weighted Round
  Robin), ..
• policers for drop precedence Weighted RED, RIO -
  RED variants which drop according to priorities
• shapers for traffic conditioningLeaky Bucket -
  enforces CBR, may drop!
• meters for drop precedence marking
• Token Bucket(s) with various thresholds("A
  Single Rate Three Color Marker")

108
DiffServ extensions / ideas

• IntServ over DiffServ
• may be good idea fine granularity of IntServ /
  RSVP signaling at edge routers end systems,
  scalability through DiffServ core
• IntServ flows are aggregated for DiffServ
• DiffServ does not participate in RSVP signaling
• IntServ treats DS Domains (EF PHB!) as a leased
  line
• Bandwidth Broker
• additional network nodes for signaling and
  negotiation
• translation SLS -gt TCS
• explicit communication with edge routers, e.g.
  via COPS
• Open specification brought some chaos, tooRed /
  green / blue packets, assured / premium service,
  Gold / Silver / Bronze olympic services .. what
  is real?

109
QoS Routing

• IntServ and DiffServ assume shortest-path
  routing!Not always optimal some flows may
  prefer a "long, fat pipe"
• Solution classify / meter, then forward
  according to requirements
• Knowledge of a path's QoS properties additional
  routing metrics(increases routing protocol
  traffic!)
• Problems
• scalability / oscillation - if QoS Routing is
  done for many sourcesquality reduced by own
  payload! use old path again?
• when / how often is QoS measured / calculated?
• QoS Routing not yet a real issue in IETF(WG only
  produced framework, OSPF QoS extensions
  experimental)

110
Traffic Engineering

• Static configuration administrators want to move
  some traffic
• based on long-term measurements

• IP-in-IP tunneling example
• B encapsulates packets (new srcB, dstC), C
  removes new header

111
From traffic engineering to MPLS

• Layer violation
• If you are tunneling along a fixed path and your
  network is ATM, you could just as well set up a
  VC for the path - faster forwarding!
• Automatic variant Ipsilon IP Switching
• Switches identify flow (MF classification),
  establish ATM-VC "Short-Cut"
• Does not scale well - fine granularity
• Better Multiprotocol Label Switching (MPLS)
• Not just (but mostly) ATM, even LANs!
• based upon separation of forwarding and control
  functionality in routers
• Label put short info. (from layer 2) in front of
  IP (like IP encapsulation)
• Label Switching Routers (LSR) forward on Label
  Switched Path (LSP)
• At destination remove label, forward IP packet
  normally

112
Multi Protocol Label Switching (MPLS)

• Forwarding Equivalence Class (FEC)
• Group of packets with similar expected treatment
  (usually same label)
• Various forms of classification possible (MF, ..)
• What if labeled packets are labeled again?
• Labels are stacked (push, pop, swap poppush,
  connects two LSPs )

113
MPLS details

• Label designed for speed
• 32 bit
• S1 this is the last label
• TTL is the only IP header field that MUST be
  treated at each hop
• Labels distributed via Label Distribution
  Protocol (LDP)
• MPLS applications
• Traffic engineering (like IP-in-IP tunneling)
• Split load by establishing more LSPs for one FEC
• Virtual Private Networks (VPNs)
• First aid if links go down (switch to different
  LSP)
• QoS support

114
QoS support with MPLS

• Obvious choices
• IntServ over MPLS
• set up a label switched RSVP tree
• RSVP-TE protocol RSVP extension MPLS-specific
  information in messages (LABEL_REQUEST in PATH,
  LABEL in RSV, ..)
• DiffServ over MPLS
• CR-LDP protocol LDP extended with traffic
  parameters
• Group packets according to DSCP
• Set up E/L - LSPs via LDP or RSVP
• E-LSP EF and AF1 packets transmitted on same
  LSP, but differentqueues (based on experimental
  bits)
• L-LSP EF and AF1 packets on different LSPs,
  different queues

115
Multi Protocol Lambda Switching (MP?S)

• Wavelength Division Multiplexing (WDM)
• frequency multiplexing for optical tranmission
  media
• optical packet switches - switches based on
  colours
• problems contention (signals with same colour
  overlap), ...
• IP over WDM difficult to realizeeasier if
  connection oriented
• Achieved via MPLS (Wavelength label) -gt MP?S
• MP?S switches can be connected via ?'s for
  default-routing and signaling (similar to MPLS
  lt-gt ATM VC)
• Even heavier layer violation!

116
Internet Protocol Version 6 (IPv6, IPNG)

• Different addresses (much bigger! but makes
  migration hard)
• Some header fields removed
• Multicast - IGMP now part of ICMP
• New optional header extensions(IPSec problematic
  for MF classification!)
• QoS support
• DiffServ field / flow label instead of ToS /
  precedence...for easier flow classification (no
  further semantics defined)

117
Main IPv6 Header
Optional extension headers
Fragmentation only in hosts!
118
Transition From IPv4 To IPv6

• Not all routers can be upgraded simultaneous
• no flag days
• How will the network operate with mixed IPv4 and
  IPv6 routers?
• Several approaches
• Dual Stack some routers with dual stack (v6, v4)
  can translate between formats (example path
  v6 ? v6_2_v4 ? v4_2_v6 ? v6)
• Tunneling IPv6 carried as payload in IPv4
  datagram among IPv4 routers
• IPv6/IPv4 network address and protocol
  translation serious recent efforts
• Short-term solution Network Address Translator
  (NAT)
• assumption at time t, only x out of y hosts
  communicate with outside world? use unique
  translation tables ( state!) to map x local to
  x global addresses
• NAT extension NAPT (Network Address / Port
  Translator)
• map local ip addr. / (tcp or udp) port no. pair
  to globally unique ip address / port no.
• advantage 1 globally unique ip address can be
  used by several local hosts at once
• disadvantage problems with specific port numbers

119
Some NAT trouble (there is more!)

• Problem realm-specific IP address in payload
• Solution per-app treatment by ALG (Application
  Level Gateway)
• Problem IPSec encryption at lower layer
  (transparent to the app)
• Solution implement lower-layer decryption
  encryption in ALG
• ...
• IP fragmentationsemantics of IP address /port
  number pair lost atIP layer -gt wrong reassembly!

ignored by IP layer!

• NATs break the end-to-end model
• complicated functions (state, ..) in the network
• special per-application functionality at lower
  layer

120
Internet2 Qbone Initiative

• Internet2 - http//www.internet2.edu
• partnership of over 130 universities, 40
  corporations and 30 other organizations (numbers
  of fall '99)
• One Internet2 objective (there are others)
• engineer scalable, interoperable, and
  administrable interdomain QoS to support advanced
  networked applications
• Qbone - http//www.internet2.edu/qos/qbone
• an interdomain testbed to enable Internet2 QoS
  objective
• driven by input and work from
• Internet2 QoS Working Group
• Internet2/Qbone Bandwidth Broker Advisory Council
• To participate, you must make your network part
  of Internet2!

121
IP QoS lessons learned
122
Some QoS rules

• Scalability above everything!
• especially avoid per flow state
• avoid state alltogether
• consider hierarchical structures for state
  aggregation
• QoS guarantees need a consistent end2end service
  model
• If hard guarantees are impossible, consider
  "softer" QoS
• Consider interactions with end system congestion
  control!
• Layer violations may be necessary
• Either "manage unfairness" or be fair (Internet
  TCP-friendly)

123
The Future
124
History again