NetQuest: A Flexible Framework for Internet Measurement

1
NetQuest A Flexible Framework for Internet
Measurement

Lili Qiu Joint work with Mike Dahlin, Harrick
Vin, and Yin Zhang UT Austin
2
Motivation
Server
Sprint
Server
CW
AOL
ATT
UUNet
SBC
Qwest
Earthlink
Server
Server
3
Motivation (Cont.)
AOL
CW
Sprint
ATT
UUNet
SBC
Qwest
Earthlink
4
Motivation (Cont.)

• Applications are performance-aware
• Server selection
• Fault diagnosis
• Traffic engineering
• Overlay networks
• Peer-to-peer applications
• Internet large decentralized
• Network measurement is important to
• ISPs
• Enterprise and university networks
• Application and protocol designers
• End users

5
Key Requirements

• Scalable work for large networks (100 10000
  nodes)
• Flexible accommodate different applications
• Multi-user design
• Multiple users interested in different parts of
  network or have different objective functions
• Augmented design
• Conduct additional experiments given existing
  observations, e.g., after measurement failures
• Differentiated design
• Different quantities have different importance,
  e.g., a subset of paths belong to a major customer

Q Which measurements to conduct to estimate the
quantities of interest?
6
What We Want

• A function f(x) of link performance x
• We use a linear function f(x)Fx in this talk

Ex. 1 average link delay f(x)
(x1x11)/11 Ex. 2 end-to-end delays
Apply to any additive metric, eg. Log (1 loss
rate)
x2
3
2
x4
x1
x3
x5
x6
5
4
1
x10
x7
x8
x11
7
6
x9
7
Problem Formulation

• What we can measure e2e performance
• Network inference
• Given e2e performance, infer link performance
• Infer x based on yFx, y, and F
• Design of measurement experiments
• State of the art
• Probe every path (e.g., RON)
• Rank-based approach sigcomm04
• Select a best subset of paths to probe so that
  we can accurately infer f(x)
• How to quantify goodness of a subset of paths?

8
Bayesian Experimental Design

• Notations
• D a measurement design (eg., a subset of paths
  to probe)
• I an inference algorithm
• U(D,I) utility function for design D and
  inference I
• A good design maximizes the expected utility
  under the optimal inference algorithm

9
Design Criteria

• Let , where
  is covariance matrix of x
• Bayesian A-optimality
• Goal minimize the squared error
• Bayesian G-optimality
• Goal minimize the worst-case squared error
• Bayesian D-optimal
• Goal maximize the expected gain in Shannon
  information

10
Flexibility

• Multi-user design
• New design criteria a linear combination of
  different users design criteria
• Augmented design
• Ensure the newly selected paths in conjunction
  with previously monitored paths maximize the
  utility
• Differentiated design
• Give higher weights to the important rowsof
  matrix F

11
Evaluation Methodology

• Data sets
• NLANR traces
• RTT, loss, traceroute measurements between pairs
  of 140 universities in Oct. 2004
• Resilient overlay network (RON)
• RTT and loss among 12-15 hosts in March May
  2001
• Accuracy metric

12
Evaluation Results (Cont.)
All pairwise
Rank-based
13
Evaluation Results (Cont.)
All pairwise
Rank-based
14
Summary of Other Results

• Bayesian experimental design can support
• Multi-user design
• Augmented design
• Differentiated design
• Inference accuracy also depends on
• Inference algorithms
• Prior information

15
Summary

• Our contributions
• Bring Bayesian experimental design to network
  measurement
• Develop a flexible framework to accommodate
  different design requirements
• Experimentally show its effectiveness
• On-going work
• Build a toolkit
• Gain operational experience
• Develop applications
• anomaly detection
• performance knowledge plane

16
Thank you!