Network Topology Inferences

1
Network Topology--- Inferences

• Prof. Gao
• ECE697A Fall 2003
• Advanced Computer Networks

2
Outline

• Topology Inferences
• AS level connectivity
• From Routing Tables
• From Routing dynamics
• Combined with other resources
• Router Level connectivity within an AS
• Rocketfuel
• Link weight inferences
• IP prefix topology
• Clustering network prefixes

3
AS Level Connectivity

• From Routing Tables
• Prefix NextHop Metric
  AS Path
• 6.1.0.0/16 216.18.63.137 0
  6539 2914 668 1455 i
• 217.75.96.60 0
  16150 8434 3257 1239 701 668 1455 i
• 157.130.182.254 1065 0
  19092 701 668 1455 i
• 194.85.4.249 0
  3277 8482 2578 702 701 668 1455 i
• 134.55.20.229 0
  293 668 1455 i
• 64.50.230.1 0
  4181 3356 701 668 1455 i
• 203.194.0.5 0
  9942 1 3356 701 668 1455 i
• 213.200.87.254 80 0
  3257 1239 701 668 1455 i
• 216.140.8.59 20 0
  6395 701 668 1455 i
• 216.140.14.186 8 0
  6395 701 668 1455 i
• 216.140.2.59 23 0
  6395 701 668 1455 i
• 216.191.65.126 0
  15290 701 668 1455 i
• Get AS level Connectivity
• 6539 lt-gt 2914 2914 lt-gt 668 668 lt-gt
  1455

4
AS Level Connectivity

• From BGP dynamics (Update Messages)
• TIME 05/11/03 101420
• TYPE BGP4MP/MESSAGE/Update
• FROM 192.205.31.33 AS7018
• TO 193.0.0.1 AS12654
• ORIGIN INCOMPLETE
• ASPATH 7018 6453 8297 8319 8319 16084
• NEXT_HOP 192.205.31.33
• ANNOUNCE
• 217.67.32.0/20
• Get AS level Connectivity
• 7018 lt-gt 6453 6453 lt-gt 8297 8297 lt-gt
  8319

5
Completeness

• AS level connectivity is not complete
• From only one or two BGP routing tables or BGP
  dynamics
• Combined with other resources
• More BGP routing tables
• Internet Routing Registries
• Traceroute

6
Several Resources

• Route Server
• Supply IP (forwarding or BGP) routing tables
• Looking Glass
• A piece of software running as a CGI on an ISP's
  web server
• Allows external users to look at routing and
  network behavior within their network.
• Internet Routing Registry (IRR)
• Database storing routing policies for most ASes

7
Completeness of AS level Connectivity

• From Oregon Routeviews
• Combined with all public available route servers
  (RS)
• Combined with looking glasses (LG)
• Combined with RIPE (RIPE)

8
Difficulty on AS Level Connectivity

• Hard to get a complete AS-level connectivity
• Some AS-level links do not appear in routing
  tables
• Aggregation in BGP practices
• Network topology is dynamic
• Misconfiguration could affect the accuracy of
  AS-level topology inferences

9
Router Level Connectivity

• Question
• Instead of inferring AS-level connectivity, how
  to infer router level connectivity in an ISP?
• Tools traceroute

10
Traceroute
Solid Dot Point-of-Presence (POP) POP consists
of backbone and access routers
11
Challenge (1)

• Mapping ISP router-level topology
• traceroute
• From where ?
• To where ?
• In order to get complete topology
• All-to-all approach to do traceroute
• Tons of measurements
• Unimaginable and impossible
• How to reduce the measurements?

12
Challenge (2)

• Alias Problem
• One router has several interfaces
• Interfaces have different IP addresses
• Output of traceroute can only record interface IP
  addresses
• How to map multiple interface IP addresses to one
  router?
• Otherwise, overestimate the number of routers

13
Challenge (3)

• How to determine whether the routers belong to
  the ISP being mapped
• Their geographical locations
• Their roles in the topology

14
Rocketfuel

• Features
• Selective Measurements
• Directed probing
• Path reductions
• Alias Resolution
• Mercators IP address-based method
• Checking similarities between responses
• Router Identification and Annotation
• Rely on DNS to identify routers that belong to
  the ISP

15
Selective Measurements (1)

• Directed Probing
• Using BGP tables to choose where to traceroute
• Trace to dependent prefixes
• Dependent prefixes originated by the measured
  ISP or its singly-homed customers
• Trace from server located in a dependent prefix
• Trace to IP addresses that are likely to transit
  the ISP based on AS path

16
Selective Measurements (2)

• Example for directed probing
• from routing table
• 1.2.3.0/24 13 4 2 5 6 9 10 5
• 11 7 5
• 4.5.0.0/16 3 7 8
• 7 8
• We want to map topology in AS 7
• 4.5.0.0/16 is a dependent prefix
• Do measurements
• From any traceroute server to 4.5.0.0/16
• From traceroute server in 4.5.0.0/16 to any
  prefix
• From traceroute server inside AS 11 to 1.2.3.0/24

17
Selective Measurements (3)

• Path Reductions
• Identify probes that are likely to have identical
  paths inside the ISP
• Ingress Reduction
• Egress Reduction
• Next-hop AS reduction

18
Selective Measurements (4)

• (a) Ingress reduction Only T1 or T2 does
  measurement to P
• (b) Egress reduction T does measurement only to
  P1 or P2
• (c) Next-hop AS reduction T does measurement
  only to P1 or P2

19
Alias Router

• Alias influence to topology
• Interface 1 and 2 are alias

20
Alias Resolution

• Mercator Approach
• Assumption
• Router is configured to send UDP port
  unreachable response
• Response from the same interface
• Method
• Choose several potentially aliased IP addresses
• Send probes to high-numbered UDP port with TTL of
  255
• Two aliases will respond with same interface IP
  address
• Efficient, but depends on configurations in router

21
Rocketfuel --- Alias Resolution

• Identify peculiar similarities between responses
• TTLs in responses
• ICMP rate limiting
• IP identifier
• x-y lt200
• Imply the same router

22
Router Identification and Annotation

• Relying on DNS to router identification
• Routers are often numbered from the ASs address
  space
• att.net, sprintlink.net etc
• Most ISPs have naming convention for their
  routers
• S1-bb11-nyc-3-0.sprintlink.net
• Sprints Router in New York City
• P4-0-0-0.r01.miamfl01.us.bb.verio.net
• Verio Backbone router in Miami, Florida

23
Mapping Engine
24
ISP Maps
25
Summary on Rocketfuel

• New Internet mapping technique to directly
  measure router-level ISP topologies
• Reduces the number of required traces compared to
  all-to-all traces
• Identifies the similarity of responses for alias
  resolution

26
Usages for Rocketfuel

• Now we have router-level topology in an ISP
• More information link weights?
• Connectivity is not enough to derive best path
• Link weights in an ISP is not public

27
Motivation

• Inferring link weights provides
• a simple, concise, and useful model of
  intra-domain routing.
• Common routing protocols choose least-cost paths
  using link weights
• It extends router-level ISP maps
• Includes connectivity
• Link weights consistent with routing

28
Inferred And Actual Link Weights

• Inferred Link Weights
• Pro
• Might not be exactly match for actual link
  weights
• But, valuable in characterizing observed routing
• Con
• Do not affect the observed paths
• But, may mis-predict the paths selected during
  failures

29
Problem Formulation

• Input
• Network topology
• Routing as a set of chosen paths (weight
  shortest)
• Output
• Assign weights for each link, so that
• Weight shortest paths match chosen paths
• Not a unique solution
• Scaling all weights with same factor
• Changing the weight of a link within bounds

30
Constraint-based Method

• To approximate ISP link weights
• based solely on end-to-end measurements
• Approach
• Basic solution
• Reducing the number of constraints
• Dealing with shortcoming in measurement data
• Applicability

31
Basic Solution

• Two key observations
• Chosen path has less weight than any other path
  between two nodes
• If multiple chosen paths exist, they have equal
  weight

32
Basic solution

• Chosen paths between A-G
• ADG and ABEG
• Similar constraints for chosen paths between all
  other node-pairs
• Linear programming
• Shortcomings
• An exponential number of constraints in large
  networks

33
Reduce Constraints
I
Y
X

• Chosen Paths
• S-M-N-D, S-A-I, I-B-D
• Weight Constrains
• SMND lt SAI IBD
• SAI lt SXI
• IBD lt IYD
• SMND lt SXI IYD will be redundant
• All alternative paths from S to D through I,
  except path SAIBD
• Actual number of constraints is much lower than n3

B
A
S
D
M
N
34
Shortcomings In Measurement Data

• Two shortcomings from using traceroute
  measurements
• Some paths might not be shortest path due to
  transient events (e.g. failures)
• Not all chosen paths can be observed

35
Approach to resolve the first shortcoming

• To associate error variables with constraints
• Associate with error variables for observed paths
• 1. Wad Wdg - eadg Wab Wbe Weg - eabeg
• 2. Wad Wdg - eadglt Wac Wcg
• Similar constraints for all node-pairs
• Using the simplex algorithm to solve the
  constraint system
• Minimize the weighted sum of the error variables

36
Approach to resolve the second shortcoming

• Pair-wise Completeness
• The fraction of vertex-pairs between which at
  least one path was observed
• Two techniques to improve pair-wise completeness
• Path symmetry
• Reverse path of a shortest path is shortest
• Optimal substructure property
• Any subset of a shortest path is shortest

37
Applicability

• Applicable for any network with weighted shortest
  path routing with a single setting of link
  weights
• the preferred path between two nodes is dependent
  only on the weights

38
Data source completeness

• Maps collected by Rocketfuel have high pair-wise
  completeness

39
Evaluation

• Criteria
• Fraction of observed paths that had least cost
• Fraction of dominant paths that had least cost
• Dominant paths most commonly seen paths
• Fraction of node-pairs between which the routing
  was fully characterized
• Load balancing
• Two or more least cost paths

40
Other Topology

• What we have known
• AS-level topology in the Internet
• Router-level topology in the ISPs
• However, In the Internet
• Routes are based on the source/destination IP
  addresses
• The logical relationship between prefixes in an
  ISP

41
Prefix Topology Inferences

• From BGP Routing dynamics
• Main Idea
• Clustering of Prefixes based upon similarities
  between their frequency of update
• Creating a hierarchy linking the prefixes within
  an AS

42
Clustering Mechanism

• Input
• Time series of routing updates
• Ordered by timestamp
• Clustering Mechanism
• Group the prefixes that are frequently updated in
  the same time window

43
Distance Metric

• Correlation between two update streams
• Up(t) 1 if p updated during interval t and 0
  otherwise

44
Correlation

• Based on Up(t) calculate correlation between two
  prefixes as follow

45
Single-Linkage Clustering

• Computes pair-wise distance correlation
• Stores them in sorted order
• Iterates prefix-pair from closest to farthest
• When encounter new node in a pair, join it to its
  neighbor or neighbors cluster if the
  neighborhood already clustered
• If prefixes are in different clusters, merge the
  clusters

46
Single-Linkage Clustering
47
Data Collection
Genuity
Northeast exchange
Collected 70 Million BGP announcements
48
Data Collection cont.

• Performed clustering on
• 2338 prefixes announced by AS 701 (UUNET)
• 1310 prefixes announced by AS 7018 (ATT)
• Time Window 30 seconds

49
Results

• UUNET ends up with 6 clusters after 2.3 million
  comparisons
• ATT ends up with 5 clusters after 800k
  comparisons

50
Evaluation

• Are the clustered IP Addresses adjacent to each
  other ?
• use IP address similarity
• Are the Prefixes routed to the same destination ?
  
• DNS based POP comparison
• How deep into the network do the prefixes share a
  path ?
• Ratio of shared to unshared path length

51
Examples

• Prefixes 200.50.192.0/19 and 196.3.153.0/24
• Appear to have little to do with each other
• However
• Their traceroutes stay together for 10 hops, to
  New York.
• From whois data,, both are in the Caribbean
• one in Jamaica, the other in Haiti.

52
Examples

• Prefixes 199.230.128.0/23 and 204.154.48.0/21
• Located about 45 miles away from each other in
  Illinois
• However, traceroute does not reveal this
• From historical data of BGP updates
• One uses backup link to UUNET
• One uses default link as before

53
Implication
Many BGP Updates occur for multiple prefixes at
the POP level

• Retaining 95 POP Level accuracy
• UUNET reduces clusters from 2337 to 1200
• ATT reduces clusters from 1310 to 900

54
Summary

• Prefixes Clustering
• One or more prefixes have almost similar
  properties on their frequency of update
• Groups of prefixes sharing a common BGP AS path
• How many prefixes sharing a common AS path in the
  Internet?

55
Policy Atoms

• Broido and Claffys definition
• Groups of prefixes sharing a common BGP AS path
  at any Internet backbone router.
• Prefixes missing in any routing table were not
  considered
• In this paper
• A group of prefixes p, for any index i and j
• Prefixes Pi, Pj belonging to p
• Route to Pi equals to route Pj for any router A

56
Policy Atoms

• If some prefixes are missing from the view point
  of some Internet routers
• They are put into different atoms

57
Routing Tables
58
Example of Calculation of Policy Atoms
59
Calculation of Policy Atoms

• RIPE Database for 13 peer routers was used to
  calculate policy atoms
• Two ways to calculate atoms
• Snapshot Method
• Uses routing table information (snapshot).
• Could not capture changes at the time of
  snapshots
• Quiet Period
• For each snapshot, tracking updates for next 4
  hours
• No updates in each 1000 seconds, atom was
  calculated

60
Statistics of Atoms

• 90 ASes have 4 atoms
• 90 ASes have 12 prefixes
• 90 atoms have 6 prefixes

61
Number of Atoms

• General statistics for ASes and atoms
• Average number of atoms calculated was 25k
• of atoms is much closer to of ASes (12.5k)
  than to of prefixes (115k)

62
Stability of Atoms
Implication Pretty stable for both mechanisms
63
Atoms Created by Policy
85 of the atoms are created between the source
AS and its immediate peers Implies policy atoms
are created by policy
64
Application of Atoms

• Compressing BGP update traffic
• Good correlation of atom structure to BGP update
  traffic
• Possible to compress BGP update traffic
• Atoms are real entities
• Group prefixes to ID of the atom
• Could save bandwidth in the Internet routing
  updates

65
Further Reading

• On Characterizing Routing Table Growth, ,
• T.Bu, Lixin Gao, and Don Towsley
• GlobalInternet 2002

66
Summary

• Inferences for AS-level topology
• Combine more routing tables and resources
• Inferences for Router-level topology
• Rocketfuel
• Inferences for link weight
• Logical relationships on network prefixes
• Prefixes clustering
• Policy Atoms