Experimental and Analytical Evaluation of Available Bandwidth Estimation Tools

1
Experimental and Analytical Evaluationof
Available Bandwidth Estimation Tools
Cesar D. Guerrero and Miguel A.
Labrador Department of Computer Science
Engineering University of South Florida
2
Outline

• Motivation
• Problem
• Testbed Description
• Analytical Model
• Target Tools
• Performance Evaluation
• Conclusions

3
MotivationWhy to evaluate available bandwidth
tools?

• Available bandwidth ? to improve network
  applications performance.
• Applications ? different time, accuracy, and
  overhead from estimators.
• Evaluation ? determine whether a tool is suitable
  for an application.

4
ProblemWhat issues do we want to solve?

• Topology
• Link capacities
• Packet loss rate
• Delay

• Evaluate tools over the same variable network
  conditions
• Analytical model to have a theoretical value to
  compare with

5
Testbed DescriptionArchitecture
Low cost

• Client and server hosting bandwidth estimation
  tools
• Intermediate nodes hosting a packet shaper and a
  traffic generator
• Phython applications running in all the machines
  to automatically perform experiments.
• Internet connected

6
Analytical ModelJackson Network
8
1
2
3
4
5
6
7
Client
Server

• Eight M/M/1 queues model input and output packet
  flows.
• The Jackson model gives the average arrival rate
  to a node
• ?j ?j S?i?ij
• The available bandwidth is the minimum non
  utilized capacity of the queues associated to the
  links
• A mini1,3,5,7 (Ai) mini1,3,5,7 (1-?i)

7
Target ToolsEstimation Approaches
Probe Rate Model

• Pathload.
• TOPP
• Pathchirp
• PTR

8
Target ToolsPathload
Figure copied from the paper Pathload A
Measurement Tool for End-to-end Available
Bandwidth by M. Jain and C. Dovrolis

• Fleet of probing streams are sent to fill the
  available bandwidth.
• The one-way delay increases when the rate of the
  probing traffic is higher that the available
  bandwidth.
• In the gray region, the tool reports the
  available bandwidth

9
Target ToolsIGI

• Estimates the cross traffic as a function of the
  amount of traffic inserted between a packet pair.
• Available bandwidth is given by the average rate
  of the packet train when the initial packet gap
  is equal to the output gap.

Figure copied from the paper Evaluation and
Characterization of Available Bandwidth Probing
Techniques by N. Hu and P. Steenkiste
10
Target ToolsSpruce

• Probing packets are sent with an intra-pair gap
  (?in) equal to the narrow link transmission time
  of a 1500B packet (to guarantee that the pair
  will be in the queue at the same time)
• Cross traffic is measured using the dispersion of
  the probing packets (?out) calculated at the
  receiver.
• It requires a previous calculation of the tight
  link capacity (C)

11
Performance EvaluationExperiments

• Metrics accuracy, time, overhead
• 28 network scenarios link capacities from 1 to
  10 Mbps and from 10 to 100 Mbps
• Each scenario with four cross traffic loads 0,
  25, 50, and 75 of the capacity
• Every estimation was performed 35 times
• Accuracy plots have a 95 confidence interval

11760 experiments
12
Performance EvaluationAccuracy with 75 of the
capacity as cross traffic
Estimated available bandwidth / total bandwidth
(capacity)
Pathload
IGI
Spruce
13
Performance EvaluationRelative Error
Pathload
IGI
Spruce
14
Performance EvaluationConvergence Time
Pathload
IGI
Spruce
15
Performance EvaluationOverhead
Probing traffic / total bandwidth (capacity) in
the tight link
Pathload
IGI
Spruce
16
Conclusions

• Main contributions
• Low cost and flexible testbed to evaluate
  estimation tools in a controlled network.
• Analytical model to fairly compare the tools
  accuracy with a theoretical value.
• Regarding to the tools evaluation
• Pathload is the most accurate tool but the
  slowest to converge
• IGI is the fastest tool but the least accurate
• Spruce is the least intrusive tool with
  intermediate accuracy and convergence time.

17
Experimental and Analytical Evaluationof
Available Bandwidth Estimation Tools
Cesar D. Guerrero cguerrer_at_cse.usf.edu Miguel A.
Labrador labrador_at_cse.usf.edu Department of
Computer Science Engineering University of
South Florida