InterDomain TrafficEngineering

1
Inter-Domain Traffic Engineering
Principles, Applications and Case Studies Profess
or Lixin Gao Presented by Karim Mattar
2
Motivation for this Presentation

• Outlining some of the methods and concepts on how
  to "improve" inter-domain connectivity.
• Depending on who you are, "improve" will have
  different meanings.
• - Tier-I ISP catching people that are forcing
  traffic onto you via routes that you did not
  announce.
• - Small ISP finding the best people to peer
  with or buy connectivity from.
• Finding ways to reduce impact of failure in peer
  or transit networks. Making sure neighbouring
  networks do not have trouble reaching you.
  Increasing reliability.

3
Presentation Outline

• Importance of Defining Your Inter-Domain TE Goals
• Making Appropriate Measurements for Inter-domain
  TE
• Applying Collected Data to Address Your Goals
• Redefining Your Goals
• TE Examples
• Real Live Network Examples
• Who is doing this stuff?
• Questions?

4
Definition of Inter-Domain TE Goals

• Important to define your goals clearly. What do
  you want to accomplish?
• Do not measure/collect data blindly
• After you define your goals, the procedure is as
  follows
• Measure goal specific data
• Analyze the data
• Refine your goals based on collected data
• Take action to improve your network

5
Examples of Goals

• Need to decrease the amount of outgoing traffic
  on some of my peering links. (congestion
  management)
• Need to expand my inter-domain peering links
  cluefully (expand and grow). Who should I connect
  to?
• Need to find some people to provide my services
  to (sales). Profit is always a motive.

6
Tip Be Prepared to Adjust Your Assumptions

• Be prepared to adjust your assumptions based on
  measured data. Real data measurements could be
  very unexpected.
• What you planned to do, and what you end up doing
  may change substantially.
• The positive side, however, is that you should
  have a better view of your network once network
  data becomes available.

7
Inter-domain TE Measurement

• Once you define your goal need to measure data
  that will help solve the problem.
• The basic idea is to make sure you get good and
  problem/goal specific data.

8
Types of Data to Measure

• Routing Data
• BGP
• Traffic Data
• Netflow, Flow-export data
• Active Measurement Performance Data
• - Ping /Traceroute /One-way delay

9
Assumed Network Model

• Hierarchical Network Model.
• Ingress/Egress Network services are separated
  from the transit services.
• Other network models also exist.

10
Hierarchical Network Model
Core Network Services
Core1
Core2
Peer1
Peer2
LocalASN
RemoteASN
AS2
AS3
AS3
AS4
AS9
11
Routing Data

• Different data sets are available for
  measurement
• IBGP (used for analyzing outbound traffic)
• Want to see traffic exit off of my network
• Route-Reflection (used for analyzing inbound
  traffic)
• Destination address inside your network will map
  to a route and IBGP will not communicate this
  information.
• EBGP (used to see neighbor's view of you)
• Choose the right one to measure based on your
  needs/goals

12
Routing Data In/Outbound
Core Network Services
IBGP vs. Route-Reflection
Core1
Core2
Collector
Peer1
Peer2
Data
LocalASN
RemoteASN
AS2
AS3
AS3
AS4
Routes
AS9
13
Routing Data In/Outbound

• When your goal is to analyze the outbound
  traffic, and your measurement point is the exit
  point for traffic, IBGP should be used.
• Routes are always external, and thus always
  propagated
• When your goal is to analyze the inbound traffic,
  and your measurement point is the entry point for
  traffic, Route-Reflection must be used.
• Only way to get internal routes cleanly

14
BacktrackingTypes of Data to Measure

• We already talked about Routing data
• Routing Data
• BGP
• Now moving on with Traffic Data
• Traffic Data
• Netflow, Flow-export
• Active Measurement Performance Data
• - Ping /Traceroute /One-way delay

15
Traffic Accounting Data

• Also Known As
• Flow-export
• NetFlow
• Cflow

16
What is Flow Measurement?

• Packet and Byte counters
• Measured from strategic routers per input
  interface
• Which interfaces should be analyzed depends on
  your defined goals/needs.

17
Flow Data Inbound - Easy
Core Network Services
Core1
Core2
Collector
Peer1
Peer2
Data
LocalASN
RemoteASN
AS2
AS3
AS3
AS4
Routes
AS9
18
Flow Data Outbound - Easy
Core Network Services
Core1
Core2
Collector
Peer1
Peer2
Data
LocalASN
RemoteASN
AS2
AS3
AS3
AS4
Routes
AS9
19
Flow Data Outbound - Harder
AS2
AS4
Core
Core
AS6
Core
Core
Core
AS3
20
Flow Data Outbound - Harder

• Flow-export data is inbound only. Therefore all
  potential feeder links in a non-hierarchical
  structure must be accounted for in order to catch
  all potential outbound traffic.
• Problem How do you know what data coming into a
  core link is really outbound traffic? Can
  double-count!
• If each edge router or core box is running a
  different policy, the problem is magnified.

21
Tips on Using Flow Data

• Micro-management of networks based on flows is
  bad. Making macro-management decision based on
  micro-level flows is very difficult, time
  consuming and usually inefficient.
• Macro-management of networks based on flows is
  good. Looking at traffic trends over a long
  period of time is easier and is much more
  efficient in terms of managing your network.

22
Operational Challenge (1)

• Measuring data can sometimes be very difficult.
• On the other hand, anything can be measured in
  such a way that is a lot better than not
  measuring it at all.

23
Operational Challenge (2)

• Challenge of running Access lists (ACLs) vs.
  exporting data.
• If any of the interfaces on a router is running
  an access list (ACL) then you cannot export data
  from any of the interfaces.

24
Operational Challenge (3)

• On high-speed interfaces, the best you can
  realistically do is sample at some ratio less
  than 11
• If you need to compare samples, make sure the
  samples are normalized
• This does NOT mean multiplying by the interval!

25
Operational Challenge (4)

• The total benefit experienced through the
  process of network measurement is far more than
  the harm experienced through planning and
  engineering a network without network
  measurements.

26
Performance Data

• Active measurement
• Round-trip / one-way delay
• Link utilization
• Helps in goal selection and re-selection
• Helps answering the everlasting question, is what
  I am doing making things better or worse?

27
Applying Data to your Goals

• What to do with all this data?
• How do you apply the Traffic Accounting Data to
  the Routing data?
• Traffic Load per ltsomethinggt
• attribute or route
• Focus is on traffic stats (byte and packet rates)
  per AS-PATH

28
Traffic Statistics per AS-PATH

• Measuring the traffic load per AS-PATH allows us
  to creates a tree of traffic relationships.
• Allows us to address the intermediate ASs along
  a path instead of the traditional first or last
  AS.
• Gives us information about "TO (source/sink) and
  "THROUGH (transit) values instead of just "TO"
  values.

29
Measuring Traffic Load Per AS Path
Link 1
AS9995
AS1234
AS1
AS101
We measure the traffic load on Link 1 which is
carrying a total load of (X Y Z) bits/sec.
AS1 -gt AS 101.Traffic Load is X
bits/sec
AS1 -gt AS101 -gt AS1234Traffic Load is Y
bits/sec
AS1 -gt AS 101 -gt AS1234 -gt AS9995.Traffic Load
is Z bits/sec
30
Data Aggregation Over Time

• Aggregate data over timeframes (macro-level view)
• Long term averages
• Short term benchmarks
• Short term here means long term. Really
  depends on what you are trying to do.
• Monitoring the impact of a policy change over a
  few days
• Micro-management of networks based on flows is
  BAD.

31
Data Aggregation Across Interfaces

• Aggregate across the set of interfaces that
  represent your problem statement
• What interfaces am I interested in?
• One Specific Interface on a router that is
  connected to a certain neighboring AS.
• One specific Router. Want to aggregate the flows
  on all the interfaces on a router to see if
  router is performing a certain task.
• All Routers in a Specific Domain.
• Some of the interfaces on a router. M of N
  interfaces.

32
What to do with all this?

• The next question naturally is, what does one do
  once they have all this data?
• How can you now control your network to improve
  performance?

33
Assumptions about your Routing Architecture

• Routes to external networks are in BGP
• Your IGP tells you how to find the NEXT_HOP
  addresses in BGP
• We select exit points for traffic based on BGP
  path selection

34
Fixing Outbound Traffic

• Examples of Some General policies
• prefer peering links over expensive transit links
• prefer private peering links over public peering
  links
• Examples of Some Specific policies
• temporarily avoid NAP X for traffic to AS Y
• prefer AS C to reach remote network D

35
Tweakable Knobs for
Fixing Outbound Traffic

• LOCAL_PREF
• MED
• AS_PATH
• Different router vendors have a different BGP
  path selection tiebreaker list. You must choose a
  set of knobs that gives you the kind of control
  that your policy requires.

36
Tips for Controlling Your Outbound Traffic

• Make only small changes at one time.
• Pause and see what happens.
• Refine your changes and repeat the process.

37
Fixing Inbound Traffic

• Controlling inbound traffic flow involves trying
  to influence the BGP path selection decisions in
  other networks that you have no control over.

38
Tweakable Knobs for
Fixing Inbound Traffic

• AS_PATH stuffing also known as AS prepending
• AS_PATH pollution
• Prepending an AS name other than your own
• Selective Prefix Announcement
• Announcing a longer prefix on certain routes

39
Be Considerate of Other Networks

• Some tweakable knobs have a negative impact on
  other networks. Some examples include
• - Ignoring the MED attribute
• - AS path Stuffing
• - AS path pollution

40
TE Example (1)

• Who are the top consumers of my network
  resources?
• Top sources of traffic
• Top sinks of traffic
• Asymmetry

41
TE Example (2)

• Peering Optimisation
• Appropriate network expansion
• Offloading the expensive peer
• Settlement of fees and traffic ratios
• Avoiding congestion
• Maximize route availability. Having alternate
  routes.

42
TE Example (3)

• Theft-over-IP. How can you detect when peers are
  stealing from you?
• Peers dumping traffic at you for routes you
  didnt send them
• Extremely rude
• Catch them in the act

43
Real Live Network Example 1

• A network peers with a large regional ISP in
  several places. The peering links are getting
  congested.
• Who are the top talkers and top listeners that we
  reach via this peer?
• Maybe we can peer with them directly
• Look at traffic loads per AS-PATH

44
Analysis of Problem

• Analyze the relative byte count through and to
  networks reached via the peer in question.
• We get a ranked list of candidate networks who
  peering with might prove to be useful.

45
Real Live Network Example 2

• AS R wants to peer with you.
• AS R suggests that a private peering link should
  be set up right away, since they claim that you
  send them around 140M of traffic on a regular
  basis.
• What is the main question that you need to ask?
  Can we confirm those numbers before we dedicate a
  port to them?

46
Analysis of Problem

• You currently reach AS R through your peer AS T
• You have interface byte and packet counters
• Measure traffic flows to AS T and calculate AS
  Rs traffic as a proportion of all the outgoing
  traffic to AS T.

47
Who is doing this stuff?

• Yahoo! - Jeffrey Papen (TUNDRA Tool)
• Peering Analysis, Capacity Planning, Performance
  Analysis
• Features
• Custom macros for AS analysis
• Source and Destination AS bandwidth details
• Transit AS (hop counts) bandwidth summary data
• Bandwidth forecasting peering merit analysis
• Billing formulas for cost/benefit budget analysis
• Also
• Analyze internal usage for Charge Back Billing
• POP-to-POP Network Performance Analysis (latency
  / loss)
• DOS attack detection

48
Destination vs. Transit Traffic to AS701
49
Any Questions?