DARRYL VEITCH

1
ACTIVE MEASUREMENT IN NETWORKS
DARRYL VEITCH The University of Melbourne
MetroGrid Workshop GridNets 2007 19 October 2007
2
A RENAISSANCE IN NETWORK MEASUREMENT

• Not just monitoring
• Traffic patterns link, path, node,
  applications, management
• Quality of Service delay, loss, reliability
• Protocol dynamics TCP, VoIP, ..
• Network infrastructure routing, security, DNS,
  bottlenecks, latency..

• Knowing, understanding, improving network
  performance

• Data centric view of networking
• Must arise from real problems, or observations
  on data
• Abstractions based on data
• Results validated by data
• In fact the scientific method in networking
• Not just getting numbers, Discovery

3
THE RISE OF MEASUREMENT

• Papers between 1966-87 ( P.F. Pawlita,
  ITC-12, Italy )
• Queueing theory several thousand
• Traffic measurement around 50

• Now have dedicated conferences
• Passive and Active Measurement Conference
  (PAM) 2000
• ACM Internet Measurement Conference (IMC) 2001
  ...

4
ACTIVE VERSUS PASSIVE MEASUREMENT

• Typical Passive Aims
• At-a-point or Network Core
• Link utilisation, Link traffic patterns, Server
  workloads
• Long term monitoring
• Dimensioning, Capacity Planning, Source
  modelling
• Engineering view Network
  performance

• Typical Active Aims
• End-to-End or Network edge
• End-to-End Loss, Delay, Connectivity,
  Discovery ..
• Long/short term monitoring Network health
  Route state
• Internet view Application
  performance and robustness

5
(No Transcript)
6
ACTIVE PROBING VERSUS NETWORK TOMOGRAPHY

• Active Probing
• Typically over a single path
• Use tandem FIFO queue model
• Exploit discrete packet effects in
  semi-heuristic queueing analysis
• Typical metrics link capacities, available
  bandwidth

• Network Tomography
• Typically network wide multiple destinations
  and/or sources
• Simple black box node/link models, strong
  assumptions
• Classical inference with twists
• Typical metrics per link/path
  loss/delay/throughput

7
TWO ENTREES

• Loss Tomography
• Removing the temporal independence assumption
• Arya, Duffield, Veitch, 2007

• Active Probing
• Towards (rigorous) optimal probing
• Baccelli, Machiraju, Veitch, Bolot, 2007

8
TOMO - GRAPHY
Tomos section
Graphia writing
9
EXAMPLES OF TOMOGRAPHY

• Atom probe tomography (APT)
• Computed tomography (CT) (formerly CAT)
• Cryo-electron tomography (Cryo-ET)
• Electrical impedance tomography (EIT)
• Magnetic resonance tomography (MRT)
• Optical coherence tomography (OCT)
• Positron emission tomography (PET)
• Quantum tomography
• Single photon emission computed tomography
  (SPECT)
• Seismic tomography
• X-ray tomography

10
COMPUTED TOMOGRAPHY
11
NETWORK TOMOGRAPHY

• Began with Vardi 1996
• Network Tomography estimating
  source-destination traffic intensities from link
  data

• Classes of Inversion Problems
• End-to-end measurements ? internal metrics
• Internal measurements ? path metrics

• The Metrics
• Link Traffic (volume, variance) (Traffic
  Matrix estimation)
• Link Loss (average, temporal)
• Link Delay (variance, distribution)
• Link Topology
• Path Properties (network kriging)
• Joint problems (use loss or delay to infer
  topology)

12
THE EARLY LITERATURE (INCOMPLETE!)

• Traffic Matrix Tomography
• ATT ( Zhang, Roughan, Donoho et al. )
• Sprint ATL ( Nucci, Taft et al. )

• Loss/Delay/Topology Tomography
• ATT ( Duffield, Horowitz, Lo Presti, Towsley
  et al. )
• Rice ( Coates, Nowak et al. )
• Evolution
• loss, delay ? topology
• Exact MLE ? EM MLE ? Heuristics
• Multicast ? Unicast (striping )

13
LOSS TOMOGRAPHY USING MULTICAST PROBING
multicast probes
...
Loss rates in logical Tree
Loss Estimator
...
14
THE LOSS MODEL

• Stochastic loss process on link acts
  deterministically on probes arriving to
• Node and Link Processes
• loss process on link
• probe bookkeeping process for node

1
1

0
0
1
1
0
0

Link loss process
1

0
0
0
Bookkeeping process
15
ADDING PROBABILITY LOSS DEPENDENCIES

• Independent Probes
• Spatial
• loss processes on different links independent
• Temporal
• losses within each link independent

Model reduces to a single parameter per link, the
passage or transmission probabilities
16
FROM LINK PASSAGE TO PATH PASSAGE PROBABILITIES
Path probabilities only ancestors matter
Sufficient to estimate path probabilities
17
ACCESSING INTERNAL PATHS
Estimation joint path passage probability,
single probe
Aim estimate

Obtain a quadratic in
1
0
1
0
1
1

• Original MINC loss estimator for binary tree
• Cáceres,Duffield, Horowitz, Towsley 1999

1
1
1
1
1
1
0
0
0



10 / 19
18
OBTAIN PATHS RECURSIVELY
Estimation joint path passage probability,
single probe

10 / 19
19
TEMPORAL INDEPENDENCE HOW FAR TO RELAX ?

• Before
• Spatial
• link loss processes independent
• Temporal
• link loss processes Bernoulli
• Parameters link passage probabilities

• After
• Spatial
• link loss processes independent
• Temporal
• link loss processes stationary, ergodic
• Parameters joint link passage probabilities
  over index sets
• Full characterisation/identification
  possible!

20
TARGET LOSS CHARACTERISTIC

• Loss run-length distribution ( density, mean
  )

Pr
1
2
3
Loss-run length


Loss/pass bursts

• Importance
• Impacts delay sensitive applications like VoIP
  (FEC tuning)
• Characterizes bottleneck links

21
ACCESSING TEMPORAL PARAMETERS GENERAL PROPERTIES

• Sufficiency of joint link passage probabilities

e.g.

• Mean Loss-run length

• Loss-run distribution

7
22
JOINT PASSAGE PROBABILITIES

• Joint link passage probability

• Joint path passage probability

23
ESTIMATION JOINT PATH PASSAGE PROBABILITY
No of equations equal to degree of node k
computed recursively over index sets
0 for
24
ESTIMATION JOINT PATH PASSAGE PROBABILITY

• Estimation of in general trees
• Requires solving polynomials with degree equal
  to the degree of node k
• Numerical computations for trees with large
  degree
• Recursion over smaller index sets

• Simpler variants
• Subtree-partitioning
• Requires solutions to only linear or
  quadratic equations
• No loss of samples
• Also simplifies existing MINC estimators
• Avoid recursion over index sets by considering
  only
• subsets of receiver events which imply

25
SIMULATION EXPERIMENTS

• Setup
• Loss process
• Discrete-time Markov chains
• On-off processes pass-runs geometric, loss-runs
  truncated Zipf
• Estimation
• Passage probability
• Joint passage probability for a pair of
  consecutive probes
• Mean loss-run length
• Relative error

26
EXPERIMENTS

• Estimation for shared path in case of
  two-receiver binary trees

Markov chain
On-off process
27
EXPERIMENTS

• Estimation for shared path in case of
  two-receiver binary trees

Markov chain
On-off process
28
EXPERIMENTS

• Estimation of for larger trees

Trees taken from router-level map of ATT network
produced by Rocketfuel (2253 links, 731
nodes) Random shortest path multicast trees
with 32 receivers. Degree of internal nodes from
2 to 6, maximum height 6
29
VARIANCE

• Estimation for shared path in case of tertiary
  tree

Standard errors shown for various temporal
estimators
30
CONCLUSIONS

• Estimators for temporal loss parameters, in
  addition to loss rates
• Estimation of any joint probability possible for
  a pattern of probes
• Class of estimators to reduce computational
  burden
• Subtree-partition simplifies existing MINC
  estimators
• Future work
• Asymptotic variance
• MLE for special cases (Markov chains)
• Hypothesis tests
• Experiments with real traffic

31
OPTIMAL PROBING IN CONVEX NETWORKS
FRANÇOIS BACCELLI, SRIDHAR MACHIRAJU, DARRYL
VEITCH, JEAN BOLOT
32
PREVIOUS WORK

• The Case Against Poisson Probing
• Zero-sampling bias not unique to Poisson (in
  non-intrusive case)
• PASTA talks about bias, is silent on variance
• PASTA is not optimal for variance or MSE
• PASTA is about sampling only, is silent on
  inversion

• Probe Pattern Separation Rule select
  inter-probe (or probe pattern)
• separations as i.i.d. positive random variables,
  bounded above zero.
• Example Uniform (i.i.d.) separations on
  0.9µ, 1.1µ
• Aims for variance reduction
• Avoids phase locking (leads to sample path
  bias)
• Allows freedom of probe stream design

10
33
LIMITATIONS

• Results derived in context of delay only
• No optimality result (expected to be highly
  system and traffic dependent)

10
34
NEW WORK

• Extended all results to loss case (loss and
  delay in uniform framework)
• For convex networks, have universal optimum for
  variance
• But sample-path bias
• Give family of probing strategies with
• Zero bias and strong consistency
• Variance as close to optimal as desired
  (tunable)
• Fast simulation
• Consistent with spirit of Separation Rule

• But are networks convex?
• Some systems when answer is known to be Yes
• virtual work of M/G/1
• loss process of M/M/1/K
• Insight into when No
• Real data when answer seems to be Yes

10
35
TWO PROBLEMS SAMPLING AND INVERSION

• Sampling
• For end-to-end delay of a probe of
  size x
• Only have probe samples

• Inversion
• From the measured delay data of perturbed
  network, may want
• Unperturbed delays
• Link capacities
• Available bandwidth
• Cross traffic parameters at hop 3
• TCP fairness metric at hop 5 .

Here we focus on sampling only, do so using
non-intrusive probing
10
36
THE QUESTION OF GROUND TRUTH

• Non-intrusive Delay
• Delay process using zero sized
  probes still meaningful
• Each probe carries a delay samples available
  

• Non-intrusive Loss
• Losses with x0 are hard to find..
  meaningful but useless
• Lost probes are not available (where? How?)
• Probes which are not lost tell us ? about
  loss?

Cannot base non-intrusive probing on virtual
(x0) probes
10
37
GENERAL GROUND TRUTH

• Approach
• Define end-to-end process directly
  in continuous time
• Must be a function of system in equilibrium -
  no probes, no perturbation!
• Non-intrusive probing defined directly as a
  sampling of this process
• Interpretation what probe would have seen
  if sent in at time .

• Note
• Process may be very general, not just isolated
  probes but probe patterns!
• Applies equally to loss or delay
• This is the only way, even virtual probes can
  be intrusive!

10
38
LOSS GROUND TRUTH EXAMPLES

• 1-hop
• FIFO, buffer size K bytes, droptail
• If packet based instead, becomes independent of
  x !

• 2-hop
• Depends on x

• Other examples
• Packet pair jitter
• Indicator of packet loss in a train
• Largest jitter in a chain, or number lost

10
39
NIPSTA NON-INTRUSIVE PROBING SEES TIME AVERAGES

• Strong Consistency
• Empirical averages seen by probe samples converge
  to true expectation
• of continuous time process, assuming
• Stationarity and ergodicity of CT
• Stationarity and ergodicity of PT (easy)
• Independence of CT and PT (easy)
• Joint ergodicity between CT and PT

Result follows just as in delay work, using
marked point processes, Palm Calculus and Ergodic
Theory.

• Not Restricted to Means
• Temporal quantities also, eg jitter
• Probe-train sampling strategies

10
40
OPTIMAL VARIANCE OF SAMPLE MEAN
Covariance function
Sample Mean Estimator
Periodic Probing
10
41
PERIODIC PROBING IS OPTIMAL !
Compare integral terms If R convex, then can
use Jensens inequality
No amount of probe train design can beat periodic!

• Problem
• Periodic is not mixing, so joint ergodicity not
  assured (phase lock)
• If occurs, get sample path bias (estimator
  not strongly consistent)

10
42
GAMMA RENEWAL PROBING
Gamma Law

• Gamma Renewal Family
• As increases, variance drops at constant
  mean
• Poisson is beaten once
• Process tends to periodic as
• Sensitivity to periodicities increases however

• Proof
• Again uses Jensens inequality
• Uses a conditioning trick and a technical
  result on Gamma densities

10
43
TEST WITH REAL DATA FOR MEAN DELAY
10
44
EXAMPLE WITH REAL DATA
10
45
EXAMPLE WITH REAL DATA
10
46
CONCLUSION TO A MORNING

• Active measurement continues to develop!
• Grid applications will no doubt lead to
  interesting new problems

Merci de votre attention