End-to-End Estimation of Available Bandwidth Variation Range

1
End-to-End Estimation of Available Bandwidth
Variation Range

• Constantine Dovrolis
• Joint work with Manish Jain Ravi Prasad
• College of Computing
• Georgia Institute of Technology

2
Probing the Internet

• Several network parameters are important for
  applications and transport protocols
• Delay, loss rate, capacity, congestion, load, etc
• Internet routers do not provide direct feedback
  to end-hosts
• Due to scalability, simplicity administrative
  issues
• Except SNMP, ICMP
• Alternatively
• Infer network state through end-to-end
  measurements

3
End-to-end bandwidth estimation

• Bandwidth in data networks refers to throughput
  (bits/sec)
• Capacity maximum possible throughput w/o cross
  traffic
• Available bandwidth (or residual capacity)
  capacity cross traffic
• Bandwidth estimation measurement techniques
  statistical analysis to infer bandwidth-related
  metrics of individual links and end-to-end
  network paths
• Objectives
• Accuracy application-specific but typically
  within 10-20
• Estimation latency within a few seconds
• Non-intrusiveness cross traffic should not be
  affected
• Scalability important when monitoring many paths
  (not covered in this talk)

4
Why to measure bandwidth?

• Large TCP transfers and congestion control
• Bandwidth-delay product estimation
• TCP socket buffer sizing
• Streaming multimedia
• Adjust encoding rate based on avail-bw
• Intelligent routing systems
• Overlay networks and p2p networks
• Intelligent routing control multihoming
• Content Distribution Networks (CDNs)
• Choose server based on least-loaded path
• SLA verification interdomain problem diagnosis
• Monitor path load and allocated capacity
• End-to-end admission control
• Network spectroscopy
• Several more..

5

• Definitions and problem statement

6
Capacity

• Maximum possible end-to-end throughput at IP
  layer
• In the absence of any cross traffic
• For maximum-sized packets

• If Ci is capacity of link i, end-to-end capacity
  C defined as
• Capacity determined by narrow link

7
Average available bandwidth

• Per-hop average avail-bw
• Ai Ci (1-ui)
• ui average utilization
• A.k.a. residual capacity
• End-to-end avg avail-bw A
• Determined by tight link
• ISPs measure average per-hop avail-bw passively
• 5-min averaging intervals

8
Avail-bw variability

• Avail-bw has significant variability
• Variability depends on averaging timescale t
• Larger timescale, lower variance
• Variation range
• Range between, say, 10th to 90th percentiles
• Example
• Path-1 variation range 10Mbps, 90Mbps
• Path-2 variation range 20Mbps, 20Mbps
• Which path would you prefer?

9
The avail-bw as a random process

• Instantaneous utilization ui(t) either 0 or 1
• Link utilization in (t, tt)
• Averaging timescale t
• Available bandwidth in (t, tt)
• End-to-end available bandwidth in (t, tt)

10
Problem statement

• Avail-bw random process, measured in timescale t
  At(t)
• Assuming stationarity, marginal distribution of
  At
• Ft(R) Prob At R
• Ap pth percentile of At, such that p Ft(Ap)
• Objective Estimate variation range AL, AH for
  given averaging timescale t
• AL and AH are pL and pH percentiles of At
• Typically, pL 0.10 and pH 0.90

11

• Probing methodology

12
Probing a network path

• Sender transmits periodic packet stream of rate R
• K packets, packet size L, interarrival T L/R
• Receiver measures One-Way Delay (OWD) for each
  packet
• D(k) tarv(k) - tsnd(k)
• OWD variations ?(k) D(k1) D(k)
• Independent of clock offset between
  sender/receiver
• With stationary fluid-modeled cross traffic
• If R gt A, then ?(k) gt 0 for all k
• Else, ?(k) 0 for all k

13
Self-loading periodic streams

• Increasing OWDs means RgtA
• Non-increasing OWDs means RltA

14
Example of OWD variations

• 12-hop path from U-Delaware to U-Oregon
• K100 packets, A74Mbps, T100µsec
• Rleft 97Mbps, Rright34Mbps

15

• Percentile sampling
• estimation algorithms

16
Percentile sampling

• Given R and t, estimate Ft(R)
• Ft(R) is also referred to as the rank of rate R
• Assume that Ft(R) is inversible
• Sender transmits a periodic packet stream of rate
  R
• Length of stream measurement timescale t
• Receiver classifies the stream, based on measured
  one-way delay trends, as
• Type-G if At R
• I(R) 1 with probability Ft(R)
• Type-L if At gt R
• I(R) 0 with probability 1-Ft(R)

17
Percentile sampling (cont)

• Send N packet streams, and classify each packet
  stream as
• Type-G if At R
• I(R) 1 with probability Ft(R)
• Type-L if At gt R
• I(R) 0 with probability 1-Ft(R)
• Number of type-G streams
• Unbiased estimator for the rank of rate R

18
How many streams do we need?

• Larger N ? longer estimation duration
• Smaller N ? larger variance in estimator I(R,N)/N
  
• Choose N so that
• I(R,N)/N within Ft(R) r
• r maximum percentile error
• PN(p-r) lt I(R,N) lt N(pr) gt 1-e
• where p Ft(R) and e small
• I(R,N) Binomial (N, p) assuming independent
  sampling
• With N40-50 streams, the maximum percentile
  error r for 10th-90th variation range is about
  0.05

19
Non-parametric estimation

• It does not assume specific avail-bw distribution
• Iterative algorithm
• Stationarity requirement across iterations
• N-th iteration probing rate Rn
• Use percentile sampling to estimate percentile
  rank of Rn
• To estimate the upper percentile AH with pH
  Ft(AH)
• fn I(Rn,N)/N
• If fn is between pHr, report AH Rn
• Otherwise,
• If fn gt pH r, set Rn1 lt Rn
• If fn lt pH - r, set Rn1 gt Rn
• Similarly, estimate the lower percentile AL

20
Non-parametric algorithm

• Parameter b
• Upper bound on rate variation in successive
  iterations
• Tradeoff between accuracy and responsiveness
• Larger b
• Faster convergence
• Larger oscillations

21
Validation example (non-parametric)

• Testbed experiments using real Internet traffic
  traces

b0.05
b0.15

• Non-parametric estimator tracks variation range
  within 10-20
• Optimal selection of b depends on traffic
• Traffic spikes/dips may not be detected if b is
  too small
• But larger b causes larger MSRE

22
Parametric estimation

• Assume Gaussian avail-bw distribution
• Justified assumption for large degree of traffic
  multiplexing
• And/or for long averaging timescale (gt200msec)
• Gaussian distribution completely specified by
• Mean m and standard deviation st
• pth percentile of Gaussian distribution
• Ap m st f-1(p)
• Sender transmits N probing streams of rates R1
  and R2
• Receiver determines percentiles ranks
  corresponding to R1 and R2
• m and st can be then estimated by solving
• R1 m st f-1(p1)
• R2 m st f-1(p2)
• Variation range is then calculated from
• AH m st f-1(pH)
• AL m st f-1(pL)

23
Parametric algorithm

• Variation range estimate
• Non-iterative algorithm
• More appropriate under non-stationary conditions

• Probing rates do not need to follow variation
  range
• Less intrusive probing

24
Validation example (parametric)
Gaussian traffic
non-Gaussian traffic

• Parametric algorithm is more accurate than
  non-parametric algorithm, when
• traffic is good match to Gaussian model
• in non-stationary conditions

25
Comparison of the two algorithms
Non-parametric t 40msec
Parametric t 250msec

• Non-parametric algorithm
• Stationarity assumption is more critical
  (iterative algorithm)
• Can be used with any cross traffic distribution
• Parametric algorithm
• Provides variation range estimate at end of each
  round
• Accurate when underlying traffic close to Gaussian

26

• Avail-bw variability factors

27
A sample measurement from the Internet

• Path from Georgia Tech to University of Ioannina,
  Greece
• Average avail-bw increases over this 2-hour
  period
• Variation range decreases as average avail-bw
  increases

28
Objectives and methodology

• Examine effect of following factors on avail-bw
  variability
• Load at tight link
• Degree of multiplexing at tight link
• Averaging time scale
• Single-hop simulation topology with TCP traffic
• Monitore load at tight link
• Examine variation range width V
• V AtH - AtL
• Compare V with Coefficient of Variation (CoV)
• CoV standard deviation (at time scalet) over
  average avail-bw
• V Absolute variability metric
• CoV Relative variability metric

29
Tight Link Utilization

• Variation range width V shows non-monotonic
  behavior
• V increases in low/medium load, due to
  increasing variance in input traffic (tight link
  rarely saturated)
• V decreases in heavy load due to clamping by
  tight link capacity
• CoV increases monotonically with load

30
Statistical Multiplexing

• Conventional wisdom
• Keeping the load constant, higher degree of
  multiplexing makes the traffic smoother
• Two models for increasing degree of multiplexing
• Capacity Scaling
• Increase capacity of tight link and
  proportionally increase number of flows
• Average flow rate remains constant
• Flow Scaling
• Increase number of flows and proportionally
  decrease average flow rate
• Capacity of tight link remains constant

31
Capacity Scaling

• Variation range width V increases with capacity
  scaling
• CoV decreases with capacity scaling
• Conventional wisdom true for relative
  variability (CoV) but not for absolute variation
  range (V)

32
Flow Scaling

• Variation range decreases in both absolute and
  relative terms

33
Measurement Timescale

• Avail-bw variability decreases with averaging
  time scale
• Rate of decrease depends on correlation
  structure of avail-bw process
• Observed decrease rate consistent with scaling
  process in the 50-500ms (Hurst parameter0.7)

34

• Summary and future work

35
Future work

• Applications of bandwidth estimation
• Overlay routing and multihoming path selection
  algorithms, avoidance of oscillations,
  provisioning
• Interdomain performance problem diagnosis
• TCP throughput prediction (see ACM Sigcomm05)
• Internet traffic analysis
• Use of bw-estimation to explain traffic
  burstiness in short time scales (see ACM
  Sigmetrics05)
• Examine validity of single-bottleneck assumption
• Examine congestion responsiveness of Internet
  traffic
• New estimation problems
• Detect maximum possible shared available
  bandwidth among set of network paths