Scalable and Accurate Identification of ASLevel Forwarding Paths

1
Scalable and Accurate Identification of AS-Level
Forwarding Paths

• Z. Morley Mao
• University of Michigan, Ann Arbor

Joint work with David Johnson, Jennifer Rexford,
Jia Wang (ATT-Research), and Randy Katz (UC
Berkeley)
2
IP Forwarding Path

• Path packets traverse through the Internet

Internet
IP traffic
destination
source

• Why important?
• Characterize end-to-end network paths
• Discover the router-level Internet topology
• Detect and diagnose reachability problems

3
Example Traceroute Output (Berkeley to CNN)
Hop number, IP address, DNS name
1 169.229.62.1 2 169.229.59.225 3
128.32.255.169 4 128.32.0.249 5 128.32.0.66
6 209.247.159.109 7 8 64.159.1.46 9
209.247.9.170 10 66.185.138.33 11 12
66.185.136.17 13 64.236.16.52
inr-daedalus-0.CS.Berkeley.EDU soda-cr-1-1-soda-br
-6-2 vlan242.inr-202-doecev.Berkeley.EDU gigE6-0-
0.inr-666-doecev.Berkeley.EDU qsv-juniper--ucb-gw.
calren2.net POS1-0.hsipaccess1.SanJose1.Level3.net
? ? pos8-0.hsa2.Atlanta2.Level3.net pop2-atm-P0-2
.atdn.net ? pop1-atl-P4-0.atdn.net www4.cnn.com
4
Autonomous System Forwarding Path

• Example Pinpoint forwarding loop responsible AS

Internet
IP traffic
destination
source
5
Border Gateway Protocol (BGP)
Signaling path control traffic
Origin AS
d pathA B C
d pathB C
prefix d

• BGP path may differ from forwarding AS path
• Routing loops and deflections
• Route aggregation and filtering
• BGP misconfiguration

6
Map Traceroute Hops to ASes
Traceroute output (hop number, IP)
1 169.229.62.1 2 169.229.59.225 3
128.32.255.169 4 128.32.0.249 5 128.32.0.66
6 209.247.159.109 7 8 64.159.1.46 9
209.247.9.170 10 66.185.138.33 11 12
66.185.136.17 13 64.236.16.52
Need accurate IP-to-AS mappings (for network
equipment).
7
Possible Ways to Get IP-to-AS Mapping

• Routing address registry
• Voluntary public registry such as whois.radb.net
• Used by prtraceroute and NANOG traceroute
• Incomplete and quite out-of-date
• Mergers, acquisitions, delegation to customers
• Origin AS in BGP paths
• Prefix198.133.206.0/24, ASpath1239 2914 3130
• Public BGP routing tables such as RouteViews
• Used to translate traceroute data to an AS graph
• Incomplete and inaccurate but usually right
• Multiple Origin ASes (MOAS), no mapping, wrong
  mapping

8
Refining Initial IP-to-AS Mapping

• Start with initial IP-to-AS mapping
• Mapping from BGP tables is usually correct
• Good starting point for computing the mapping
• Collect many BGP and traceroute paths
• Signaling and forwarding AS path usually match
• Good way to identify mistakes in IP-to-AS map
• Successively refine the IP-to-AS mapping
• Find add/change/delete that makes big difference
• Validation explain these edits by operational
  realities

9
BGP and Traceroute Data Collection
Initial mappings from origin AS of a large set
of BGP tables
(Ignoring unstable paths)
10
Experimental Methodology
200,000 destinations d0, d1, d2, d3, d4,
d200,000
For each di -Traceroute path -BGP path
11
Measurement Data Eight Vantage Points

• Sweep the routable IP address space
• 200,000 IP addresses
• 160,000 prefixes
• 15,000 destination ASes

12
Assumptions

• IP-to-AS mapping
• Mappings from BGP tables are mostly correct.
• Change slowly
• BGP paths and forwarding paths mostly match.
• 70 of the BGP path and traceroute path match

13
Reasons BGP and Traceroute Paths Differ

• IP-to-AS mapping is inaccurate (fix these!)
• Internet eXchange Points (IXPs)
• Sibling ASes owned by the same institution
• Unannounced infrastructure addresses
• Forwarding and signaling paths differ (study
  these!)
• Forwarding loops and deflections
• Route aggregation and filtering
• Traceroute inaccuracies (dont overreact to
  these!)
• Forwarding path changing during measurement
• Address assignment to border links between ASes
• Outgoing link identified in time exceeded
  message

14
Extra AS due to Internet eXchange Points

• IXP shared place where providers meet
• E.g., Mae-East, Mae-West, PAIX
• Large number of fan-in and fan-out ASes

A
E
A
E
F
B
F
B
D
G
C
G
C
Traceroute AS path
BGP AS path
Physical topology and BGP session graph do not
always match.
15
Extra AS due to Sibling ASes

• Sibling organizations with multiple ASes
• E.g., Sprint AS 1239 and AS 1791
• AS numbers equipment with addresses of another

A
E
A
E
F
B
D
H
F
B
D
G
C
G
C
Traceroute AS path
BGP AS path
Sibling ASes belong together as if they were
one AS.
16
Weird Paths Due to Unannounced Addresses
12.0.0.0/8
A
B
C does not announce part of its address space in
BGP(e.g., 12.1.2.0/24)
C
Fix the IP-to-AS map to associate 12.1.2.0/24
with C
17
Optimization Framework

• Start with initial IP-to-AS map A(x)
• IP address x maps to A(x), a set of ASes
• Compute traceroute IP to AS mapping
• For each traceroute-BGP path pair
• Dynamic programming to minimize mismatch
• Iterative refinement
• Modify A(x) depending on a small set of rules
• Terminate when no further modifications

18
Rules for Modifying the IP-to-AS Mapping

• Computing match statistics across paths
• Focusing on path pairs with at most two errors
• Example rules
• Create a mapping A(x) is null
• Assign to the AS y that appears in the most
  matchings
• Replace a mapping A(x) has one entry
• If an AS y not in A(x) accounts for gt 55 of
  matchings
• Delete from a mapping A(x) has multiple entries
• If an AS y in A(x) accounts for lt 10 of
  matchings
• Algorithm converges in less than ten iterations

19
Optimization Results

• Metric Mismatch ratio
• Percentage of traceroute-BGP path pairs with a
  mismatch
• Modified 2.9 of original mappings

Robustness
20
Validating the Changes to the Mapping

• ATTs tier-1 network (AS 7018)
• Dump of configuration state from each of the
  routers
• Explains 45 of 54 changes involving AS 7018
• E.g., customer numbered from ATT addresses
• E.g., Internet exchange point where ATT connects
• Whois query on prefix or AS
• Look for exchange point or Internet exchange
• Look for ASes with similar names (Sprintlink vs.
  Sprintlink3)
• List of known Internet eXchange Points
• Explains 24 of the MOAS inferences
• Total of 38 IXPs contributed to mapping changes

21
Validation Exploring the Remaining Mismatches
BGP path B C Traceroute path B C D

• Route aggregation
• Traceroute AS path longer in 20 of mismatches
• Different paths for destinations in same prefix
• Interface numbering at AS boundaries
• Boundary links numbered from one AS
• Verified cases where ATT (AS 7018) is involved

BGP path B C D Traceroute path B D
22
Contributions

• Problem formulation
• AS-level traceroute tool for troubleshooting
• Compute an accurate IP-to-AS mapping
• Optimization approach
• Compute matchings using dynamic programming
• Improve mapping through iterative refinement
• Measurement methodology
• Traceroute and BGP paths from many locations
• Validation of our results
• Changes to the IP-to-AS mappings
• Remaining mismatches between traceroute and BGP

23
Future Work on AS Traceroute

• Lower measurement overhead
• Avoid traceroute probes that would discover
  similar paths
• Work with BGP routing tables rather than live
  feeds
• Limiting the effects of traceroute inaccuracies
• Catch routing changes through repeat experiments
• Use router-level graphs to detect AS boundaries
• Detect routers using outgoing link in time
  exceeded
• Public AS traceroute tool
• Periodic data collection and computation of
  IP-to-AS mapping
• Software to apply mapping to traceroute output
• Network troubleshooting
• Analyze valid differences between forwarding and
  signaling paths
• Use the AS traceroute tool to detect and
  characterize anomalies

24
Comparison of IP-to-AS Mappings
Comparing BGP and Traceroute AS paths for various
IP-to-AS mappings

• Whois unmapped hops cause half of mismatches
• BGP tables mostly match, as our algorithm
  assumes
• Refined mapping change 2.9 of original mapping
• Robust to reducing of probes and introducing
  noise

25
Systematic optimization

• Dynamic-programming and iterative improvement
• Initial IP-to-AS mapping derived from BGP routing
  tables
• Identify a small number of modifications that
  significantly improve the match rate.
• 95 match ratio, less than 3 changes, very
  robust

26
Traceroute Measuring the Forwarding Path

• Time-To-Live field in IP packet header
• Source sends a packet with a TTL of n
• Each router along the path decrements the TTL
• TTL exceeded sent when TTL reaches 0
• Traceroute tool exploits this TTL behavior

destination
source
Send packets with TTL1, 2, 3, and record
source of time exceeded message
27
Matching Function and Unavoidable Error

• Matching function m for BGP/traceroute pair
• Traceroute path t1, t2, , tn of n IP addresses
• BGP path b1, b2, , bl of l AS numbers
• Matching associate IP hop ti with AS hop bm(i)
• Find the matching m that minimizes error
• Number of traceroute hops with bm(i) not in A(ti)
• Dynamic programming algorithm to find best m

t 1 2 3 4 5 6 7
8
b A B C
28
Initial Analysis of BGP and Traceroute Paths

• Traceroute paths initial mapping A from BGP
• Unmapped hops match no ASes (1-3 of paths)
• MOAS hops match any AS in the set (10-13 of
  paths)
• hops match any AS (7-9 of paths)
• BGP paths discard 1 of prefixes with AS paths
• Routing changes based on BGP updates
• Private AS numbers (e.g., 65100)
• Empty AS paths (local destinations)
• Apparent AS-level loops from misconfiguration
• AS_SET instead of AS sequence

29
Validating the Changes to the Mapping

• ATTs tier-1 network (AS 7018)
• Dump of configuration state from each of the
  routers
• Explains 45 of 54 changes involving AS 7018
• E.g., customer numbered from ATT addresses
• E.g., Internet exchange point where ATT connects
• Whois query on prefix or AS
• Look for exchange point or Internet exchange
• Explains 24 of the changes to the mappings
• Look for ASes with similar names (Sprintlink vs.
  Sprintlink3)
• Explains many of the changes to the mappings
• List of known Internet eXchange Points
• Explains 24 of the MOAS inferences
• Total of 38 IXPs contributed to mapping changes