Intra-domain Traffic Engineering

1

• Intra-domain Traffic Engineering

2
Outline

• Introduction to Internet Traffic Engineering
• IP-based Traffic Engineering
• MPLS-based Traffic Engineering

3
Do IP networks manage themselves?

• In some sense, yes
• TCP senders send less traffic during congestion
• Routing protocols adapt to topology changes
• But, does the network run efficiently?
• Congested link when idle paths exist?
• High-delay path when a low-delay path exists?
• How should routing adapt to the traffic?
• Avoiding congested links in the network
• Satisfying application requirements (e.g., delay)
• essential questions of Traffic Engineering

4
Traffic Engineering

• What is Traffic Engineering?
• Control and optimization of routing, to steer
  traffic through the network in the most effective
  way
• Internet traffic engineering is defined as that
  aspect of Internet network engineering dealing
  with the issue of performance evaluation and
  performance optimization of operational IP
  networks. Traffic Engineering encompasses the
  application of technology and scientific
  principles to the measurement, characterization,
  modeling, and control of Internet traffic
• RFC 3272 Overview and Principles of Internet TE

5
Traffic Engineering

• Two fundamental approaches
• IP-based Traffic Engineering
• MPLS-based Traffic Engineering

6
IP-based Traffic Engineering

• Using traditional routing protocols
• Routers flood information to learn topology
• Routers determine next hop to reach other
  routers
• Path selection based on link weights (shortest
  path)
• Link weights configured by network operator

7
Approaches for setting the link weights

• Conventional static heuristic
• Proportional to physical distance
• Cross-country links have higher weights
• Minimizes end-to-end propagation delay
• Inversely proportional to link capacity
• Smaller weights for higher-bandwidth links
• Attracts more traffic to links with more capacity
• Can we do better?

8
Approaches for setting the link weights

• A traffic engineered approach
• Collect measurements of traffic and topology
• Tune the weights based on the offered traffic
• Network management system sets the link weights
• Acting on network-wide view of traffic and
  topology
• Directly minimizes metrics like max link
  utilization

9
TE approach Measure, Model and Control
measure
control
Operational network
10
IP-based Traffic Engineering

• Topology
• Connectivity and capacity of routers and links
• Traffic matrix
• Offered load between points in the network
• Link weights
• Configurable parameters for routing protocol
• Performance objective
• Balanced load, low latency,
• Question Given the topology and the traffic
  matrix, which link weights should be used?

11
IP-based Traffic Engineering

• Instrumentation
• Topology monitoring of the routing protocols
• Traffic matrix fine-grained traffic measurement
• Network-wide models
• Representations of topology and traffic
• What-if models of shortest-path routing
• Network optimization
• Efficient algorithms to find good configurations
• Operational experience to identify key
  constraints

12
Topology/Routing

• Router configuration files
• Daily snapshot of network assets configuration
• Software to parse the router config commands
• Network-wide view of topology routing policies
• Also useful for detecting configuration mistakes
• Routing monitors
• Online monitoring of routing protocol messages
• Real-time view of routes via neighboring ASes
• Real-time view of paths within the AS
• Software for aggregating and querying the data
• Also useful for detecting and diagnosing anomalies

13
Formalizing the Optimization Problem

• Input graph G(R,L)
• R is the set of routers
• L is the set of unidirectional links
• cl is the capacity of link l
• Input traffic matrix
• Mi,j is traffic load from router i to j
• Output setting of the link weights
• wl is weight on unidirectional link l
• Pi,j,l is fraction of traffic from i to j
  traversing link l

14
Formalizing the Optimization Problem

• A solution to the optimization problem cannot
  freely allocate the traffic load between two
  routers
• The traffic between two routers can only flow
  along a shortest weighted path (routing
  protocols work this way)
• This constraint implies that we achieve a
  sub-optimal solution to the general routing
  problem
• In case multiple shortest weighted paths exist,
  traffic can be split among them
• In practice, this is allowed by OSPF by means of
  the ECMP (Equal Cost Multi-Path) function

15
Formalizing the Optimization Problem

• To enforce the optimal solution, it is necessary
  to split traffic along multiple shortest paths

Values of Pi,j,l
16
Complexity of the Optimization Problem

• NP-complete optimization problem
• No efficient algorithm to find the link weights
• Even for simple objective functions
• What are the implications?
• Have to resort to searching through weight
  settings

17
Optimization based on local search

• Start with an initial setting of the link weights
• E.g., same integer weight on every link
• E.g., weights inversely proportional to capacity
• E.g., existing weights in the operational network
• Compute the objective function
• Compute the all-pairs shortest paths to get
  Pi,j,l
• Apply the traffic matrix Mi,j to get link loads
  ul
• Evaluate the objective function from the ul/cl
• Generate a new setting of the link weights

repeat
18
Making the search efficient

• Avoid repeating the same weight setting
• Keep track of past values of the weight setting
• or keep a small signature of past values
• Do not evaluate setting if signatures match
• Avoid computing shortest paths from scratch
• Explore settings that changes just one weight
• Apply fast incremental shortest-path algorithms
• Limit number of unique values of link weights
• Do not explore 216 possible values for each
  weight
• Stop early, before exploring all settings

19
Application to ATT's backbone network

• Performance of the optimized weights
• Search finds a good solution within a few minutes
• Much better than link capacity or physical
  distance
• Competitive with multi-commodity flow solution
• Optimal routing possible with more flexible
  routing protocols (e.g. MPLS)
• How ATT changes the link weights
• Maintenance every night from midnight to 6am
• Predict effects of removing link(s) from network
• Reoptimize the link weights to avoid congestion
• Configure new weights before disabling equipment

20
To learn more...

• Overview papers
• B. Fortz, J. Rexford, M. Thorup, Traffic
  engineering with traditional IP routing
  protocols, IEEE Communication Magazine, October
  2002
• Traffic measurement
• R. Caceres, N. Duffield, A. Feldmann, et al.,
  Measurement and analysis of IP network usage and
  behavior, IEEE Communications Magazine, May 2000
• A. Feldmann, A. Greenberg, C. Lund, N. Reingold,
  J. Rexford, F. True, Deriving traffic demands
  for operational IP networks Methodology and
  experience, IEEE/ACM Transactions on Networking,
  June 2001
• Topology and configuration
• A. Feldmann, J. Rexford, IP network
  configuration for intradomain traffic
  engineering, IEEE Network Magazine,
  September/October 2001
• Intradomain route optimization
• B. Fortz, M. Thorup, Internet traffic
  engineering by optimizing OSPF weights, Proc. Of
  IEEE INFOCOM 2000

21
Traffic Engineering

• Two fundamental approaches
• IP-based Traffic Engineering
• MPLS-based Traffic Engineering

22
Multi-Protocol Label Switching

• Multi-Protocol
• Encapsulate a data packet
• Could be IP, or some other protocol (e.g., IPX)
• Put an MPLS header in front of the packet
• Actually, can even build a stack of labels
• Label Switching
• MPLS header includes a label
• Label switching between MPLS-capable routers

IP packet
MPLS header
23
Forwarding Equivalence Class (FEC)

• Rule for grouping packets
• Packets that should be treated the same way
• Identified just once, at the edge of the network
• Example FECs
• Destination prefix
• Longest-prefix match in forwarding table at entry
  point
• Useful for conventional destination-based
  forwarding
• Src/dest address, src/dest port, and protocol
• Five-tuple match at entry point
• Useful for fine-grain control over the traffic
• Sent by a particular customer site
• Incoming interface at entry point
• Useful for virtual private networks

24
LSP (Label Switched Path) Setup

• Protocols used to automatically establish LSPs
• LDP (Label Distribution Protocol RFC 3036)
• RSVP (with the objects needed to request/map
  labels)
• Exploit IP routing tables information to
  establish paths and bind FECs to LSPs
• CR-LDP (Constraint-based Routing LDP RFC 3212)
• RSVP-TE (TE extension to RSVP RFC 3209)
• Support establishment of explicit paths

25
Explicit routing

• LSPs can be established along paths other than
  the shortest path
• More flexibility than destination-based IP
  routing
• In an MPLS network, it is (in theory) possible to
  implement the optimal solution to the general
  routing problem (multi-commodity flow problem)

26
TE with constraint-based routing

• Explicit routing fine-grained FEC definition
• Allows to select a suitable path for each flow
• We can select a path that meets the QoS
  requirements of the flow
• Knowledge of network resource usage is required
• Extend OSPF to disseminate the extra information
  (OSPF-TE RFC 3630)
• Maximum bandwidth, maximum reservable bandwidth,
  unreserved bandwidth, TE metric, etc.
• The MPLS ingress LSR (Label Switching Router)
  computes the constraint-based path and
  establishes the corresponding explicit LSP

27
TE with constraint-based routing

• Given the network topology and the flow
  requirements, how to select a path?
• We assume that flow requirements are expressed in
  terms of requested bandwidth B between a source s
  and a destination d
• The residual bandwidth on each link is R(l)
• A feasible path is such that R(l)gtB on all the
  links
• If we prune all the links such that R(l)ltB, all
  the paths between s and d in the pruned topology
  are feasible
• What feasible path can we choose?
• The goal is to optimize resource usage

28
Widest Shortest Path (WSP)

• WSP selects the path with the minimum hop count
  among all feasible paths
• In case several such path exist, the one the
  maximal residual bandwidth is selected
• The residual bandwidth of a path is the minimum
  among the residual bandwidths of its links
• R. Guerin, D. Williams, and A. Orda, QoS routing
  mechanisms and OSPF extensions, in Proc. IEEE
  Globecom, 1997

29
Other TE algorithms

• Most of TE algorithms
• prune links with insufficient bandwidth
• assign a cost to each link in the pruned topology
• select the least cost path
• Different techniques to assign costs
• The cost may be an increasing function of link
  load
• Slightly loaded links are preferred
• Heavily loaded links are discouraged

30
Minimum Interference Routing (MIRA)

• Given the knowledge of ingress-egress pairs, MIRA
  selects the path which minimizes the interference
• Interference on an ingress-egress pair (s,d) due
  to routing a flow between some other
  ingress-egress pair is defined as the decrease in
  the maximum network flow between s and d
• The minimum interference path between a
  particular ingress-egress pair is the path which
  maximizes the minimum maxflow between all other
  ingress-egress pairs
• This problem is NP-hard
• Set link weights proportional to maxflow
  reduction
• K. Kar, M. Kodialam, and T. Lakshman, Minimum
  Interference Routing of Bandwidth Guaranteed
  Tunnels with MPLS Traffic Engineering
  Applications, IEEE Journal on Selected Areas in
  Communications, vol. 18, no. 12, pp. 2566-2579,
  December 2000.

31
Simple Minimum Interference Routing (SMIRA)

• The interference on an ingress-egress pair is
  evaluated by means of a k-shortest-path-like
  computation instead of a maxflow computation
• The set of k paths is determined by first
  computing the widest-shortest path between s and
  d
• Then, all the links along this path with a
  residual bandwidth equal to the bottleneck
  bandwidth of the path are pruned
• The second path is the widest-shortest path in
  the pruned topology
• This procedure is repeated until either k paths
  are found or no more paths are available
• The cost of links belonging to the set of k paths
  is increased proportionally to the weight of the
  path and the ratio of bottleneck bandwidth to
  residual bandwidth.
• I. Iliadis and D. Bauer, A New Class of Online
  Minimum-Interference Routing Algorithms,
  Networking 2002, LNCS 2345, pp. 959-971, 2002.