IP Performance Metrics: Metrics, Tools, and Infrastructure

1
IP Performance MetricsMetrics, Tools, and
Infrastructure

• Guy Almes
• January 30, 1997

2
Outline

• Background
• IETF IPPM effort
• IPPM Activities outside IETF Scope
• Technical Approaches
• One Example of Measurement Infrastructure

3
Background

• Internet topology is increasingly complex
• Load grows (even) faster than capacity
• The relationship among networks is increasingly
  competitive
• Result users dont understand the Internets
  performance and reliability

4
IP Performance Metrics Objective

• enable users and service providers (at all
  levels) to have an accurate common understanding
  of the performance and reliability of paths
  through the Internet.

5
Example Internet Topology
6
IETF IPPM Effort

• BOF Apr-95 at Danvers
• Within Operational Requirements / Benchmarking
  Methodology WG
• Initial Meeting Jul-95 at Stockholm
• Framework Document Jun-96
• Definitions of specific metrics Dec-96

7
Jun-96 Framework Document

• Importance of careful definition
• Good properties related to measurement
  methodology
• Avoidance of bias and of artificial performance
  goals
• Relationship to dynamics between users and
  providers

8
Terminology for Paths and Clouds

• host, link, and router
• path lt h0, l1, h1, , ln, hn gt
• subpath
• cloud graph of routers connected by links
• exchange links that connect clouds
• cloud subpath lt hi, li1, , lj, hj gt
• path digest lt h0, e1, C1, , Cn-1, en, hn gt

9
Three Fundamental Concepts

• Metric a property of an Internet component,
  carefully defined and quantified using standard
  units.
• Measurement Methodology a systematized way to
  estimate a metric. There can be multiple
  methodologies for a metric.
• Measurements the result of applying a
  methodology. In general, a measurement has
  uncertainties or errors.

10
Metrics and the Analytical Framework

• The Internet has a rich analytical framework
  (A-frame)
• There are advantages to any notions described
  using the framework
• can leverage A-frame results
• have some hope of generality, scaling
• Well specify metrics in A-frame terms when
  possible

11
Such metrics are called analytically-specified
metrics

• Examples
• Propagation time of a link
• Bandwidth of a link
• Minimum bandwidth along a path
• The introduction of an analytical metric will
  often require the refinement of A-frame concepts
• These refinements form an hierarchy

12
Empirically-specified Metrics

• Some key notions do not fit into the A-frame
• Example flow capacity along a path while
  observing RFC-1122 congestion control
• The only realistic way to specify such a metric
  is by specifying a measurement methodology (cf.
  treno)

13
Measurement Strategies

• Active vs Passive measurements
• Hard vs Soft degree of Cooperation
• Single Metric with multiple methodologies

14
Two Forms of Composition

• Spatial Composition
• e.g., Delay metric applied to router vs path vs
  subpath
• Temporal Composition
• e.g., Delay metric at T compared to delay at
  times near T

15
Progress at the San Jose IETF

• Dec-96
• One-way Delay
• Flow capacity
• Availability
• Revisions to the Framework Document

16
Framework Revisions

• Clock Issues
• Synchronization, Accuracy, and Resolution
• Singletons, Samples, and Statistics
• Generic Type P Packets

17
Motivation of One-way Delay

• Minimum of delay transmission/propagation delay
• Variation of delay queueing delay
• Large delay makes sustaining high-bandwidth flows
  harder
• Erratic variation in delay makes real-time apps
  harder

18
SingletonType-P-One-way-Delay

• (src, dst, T, path)
• either undefined or a duration
• undefined taken as infinite
• duration in seconds

19
Sample Type-P-One-way-Delay-Stream

• (src, dst, first-hop, T0, Tf, lambda)
• sequence of ltT, delaygt pairs
• Poisson process (lambda) used to generate T values

20
Statistics

• Minimum of a Sample
• Percentile
• Median

21
Measurement Technologies

• Active Tests
• Passive Tests
• Advantages/disadvantages of each
• Policy implications of each

22
Active Tests

• Both at edge and at/near exchange points
• Extra traffic
• No eavesdropping
• Delay, packet loss, throughput across specific
  clouds

23
Example of Active Tests
24
Active Test Results
25
Passive Tests

• Only at the edge -- privacy caution
• No extra traffic
• Throughput
• Also, non-IPPM measurements on nature of use

26
Example of Passive Tests
27
Passive Test Results
28
The Surveyor Infrastructure

• Collaborating organizations
• Advanced Network Services
• (23) Common Solutions Group universities
• Active tests
• ongoing tests of delay, packet loss
• occasional tests of flow capacity
• Passive tests
• some tests to characterize Internet use

29
The Surveyor Infrastructure

• Key ideas
• database / web server to receive results
• use of GPS to synchronize clocks
• need for measurement machines both at campuses
  and at/near exchange points

30
Database/Web Server

• Measurement machines upload their results to
  the database server
• These results are stored so that queries can be
  made at a later time
• Users interrogate the server using ordinary web
  browsers
• These interrogations are analyzed using the
  database and the results returned to the user via
  the Web

31
Surveyor Infrastructure
32
Ongoing Tests
33
Uploading Results
34
Reporting Analysis
35
Policy Implications for Asia / Pacific

• Better understanding of cost vs performance
  tradeoffs
• Cooperation among users / providers
• Sharing of test results
• Value of cooperating in sharing results even of
  imperfect tests
• Value of supporting very accurate NTP at
  exchanges and campuses