Exterior Routing 201

1
Exterior Routing 201

• Howard C. Berkowitz
• hberkowi_at_nortelnetworks.com hcb_at_clark.net
• [phone] ESN 451-5819

2
Agenda

• What's the problem?
• Formal and informal clue
• ISP service offerings
• Quirks, Defnitions, and Issues
• ISP External Scenarios
• POP and other infrastructure
• Router requirements
• Playing in the Club
• Turning it On

If there's time...full employment for
consultants path selection
3
What is the Problem to be Solved?
4
Good little boys and girls read RFC1771 and live
happily ever after
5

• Noah.
• Noah.
• (yawn) MMMmmmmhp?
• Noah.
• Yeahh?
• Build an ISP.

6
ISPs Facing End User

• Entry
• Basic Internet Access
• Hosting
• Availability and QoS
• Dealing with specialized access providers (DSL,
  CATV, etc.)
• Dealing with content providers
• Voice services?

• Improvement for Users
• Improving capacity
• Improving availability
• Adding services
• Perceptions of end-to-end SLA

7
Before the Animals
Downlinks
Uplinks
Management
Routers
Facilities
User Hosts
HVAC
Staff
8
Load the Ark
From Virtual Hosts
Traffic
Policies
Policies
Traffic
Traffic
From Upstreams
From Downstreams
Traffic
AAA
From Users
9
Quirks, Definitions and Issues
10
I said "peer," not "peer"

• Peer relationship 1
• Basic BGP session
• Peer relationship 2
• Mulual benefit customers reach one another
• No monetary exchange
• Each advertises customer routes
• Transit Provider relationship
• Customer pays for service
• Full routes available to customer

11
(C) O'Leary Museum and Library Association Ltd.
Inc.
12
Closest Exit RoutingHot potato
dest
src

• Paths are not optimized end-to-end
• Paths are optimized for each AS

13
Asymmetrical Routing

• No guarantee that traffic leaving your AS at one
  point
• Will return at the same point
• Remember
• Each AS in both directions makes decisions on its
  information

14
ISP Scenarios
15
Basic Internet Access ISP
To 70-90 of customers Default route To 5-10
of customers Partial routes To 10 of
customers Full routes
From customers Few public routes ??? VPN
16
Bilateral Peering
eBGP Relationship Exchange of customer routes
only Some aggregation No infrastructure
routes Highest bandwidth requirement
"Tier 1 Provider" Does not buy transit
service from anyone Has default-free
routers Gets all routes from
bilateral/multilateral peering Total RIB
size of 1.3-1.5 DefaultFreeZone (D)
17
Large Content Provider

• Sometimes bandwidth limited
• Provider may be default free
• Often high touch processing limited
• Possible SLA and VPN agreements

May participate in content distribution, caching
18
Multilateral Peering
eBGP Relationships Depending on exchange rules
Exchange of customer routes only Most
common case Some aggregation No
infrastructure routes Some ISPs buy transit
services Can receive full routes
Private peerings Largest carriers tend to avoid
due to congestion ISPs can peer with route
server rather than a mesh of ISPs May be
done to reduce BGP peers Or simply for
statistics collection
19
Special Case Local Exchanges

• Entry
• Who's in charge?
• Connectivity
• Facilities
• Allow content providers?
• Allow end users?
• Peering model?
• Supplementary services?

• Improvements

20
POP and Other Internal Design
21
Typical Basic POP Implementation
Gigabit Ethernet
Frame Interfaces
ATM Interfaces
Router Fabric
32x/30
/18
/18
LAN Switch
ISP Core Router 1
ISP Core Router 2
Frame DS3
Full DS3
2x/25
Customer Site Routers
Customer Site Router
Management Servers
Access Server
Dedicated Customers
Dedicated Customers
PSTN
1 per POP
25 per POP
Dialup Customers
450 users per POP
22
Transit Provider POP, Intra-POP
Design Alternatives 1. POP is a route
reflector cluster Core is higher-level
cluster 2. Each POP is a private or
public AS Full mesh iBGP or route
reflectors inside POP Confederation
between POPs 3. IGP within POP
Controlled redistribution inside POP to
BGP Prefer intra-POP of same metric
POP Router
POP Router
Access Router
Access Router
23
POP Confederations
Public AS
POP AS65000
POP AS65111
POP AS65222
24
POP Reflectors
Public AS
POP AS65000
POP AS65111
POP AS65222
25
Open Access/Specialized Access
ISP 1
Subscribers
ISP 2
Tunnel Server
Layer 1/2 Fabric
ISP 3
Internal Routed Network
Content Servers
26
Tunneled Addressing
Voice Provider 1
Access Gateway
Internal Routing Switching
CLE
Data Provider 1
Enterprise VPN NAS
DHCP DNS
CLE
Data Provider 2
CLE
Access OAM address space
ISP address space
L2TP, Differv High
VoIP
Data 1
Data 2
L2TP, Differv High
VPN
27
Router Requirements

• Big part of the solution...but not all.

28
Routing Paradigms
Enterprise
Edge
Core
Number of Interfaces
Number of Routes
Forwarding Bandwidth
Hello Processing
Policy Analysis
QoS Awareness
L4/7 Processing
29
Observations on Routing Table Size

• Global default-free table continues to grow
  exponentially
• 96509 routes as of Tony Bates' CIDR report
  2/11/2001
• Let the default routing table size be D
• Large provider often has 1.3 to 1.5 D active
  routes
• additional routes are more-specific customer
  internal
• may also have substantial numbers of inactive
  routes

30
Growth in Global Routing Table Size
736K
368K
184K
Sep 01
Sep 02
Sep 03
Sep 04
85K public
31
Growth in Typical Tier 1 Routing Table
Size(external customer, not infrastructure)
1104
552
276
Sep 01
Sep 02
Sep 03
Sep 04
85K public 42K internal
32
Observation More than Routes

• Customer routes
• Paths per route
• Route validity

33
Convergence

• Global routing system
• Intra-AS
• Single Router

34
Single Router Convergence

• Initialization
• Time to add new route
• Time to add better route
• Time to withdraw route
• Time to withdraw and replace route
• Parameters
• Matrix number of peers versus
• Routes advertised
• Routes accepted

• Performance Modifiers
• Route filtering
• Route flapping
• Packet vs. route filtering

draft-berkowitz-bgpcon-0x.txt
35
Distinguish among cases

• Failover of link or router between customer and
  provider
• Rerouting to intranet/adjacent provider resources
• Rerouting to arbitrary internet destnation

More multihoming in next tutorial
36
S-T-R-E-T-C-H
37
Joining the Club
38
More than Just Addresses, Protocol...
Address Registry
Route Registry
Allocate
ISP with Prefixes
Routing System
Directories
Routing Registry
Customer
Maintainer objects
Configs
SWIP
NAT
Route objects
DNS
Reverse DNS
AS objects
Hosts
39
Complexity

• BGP itself is fairly simple
• Additional attributes it carries are more complex
• Policy actions taken inside router (BGP sender or
  receiver) far more complex than the protocol
  itself

40
"BGP Transmits Policies"
Wrong!
41
Operational Relationships 1Addresses and
Delegation
Address authority
Reverse DNS
DNS
Address delegation
Prefixes
Hosts
42
Obtain routable address space

• Apply to registry
• RIPE, APNIC, ARIN
• If immediate need for /19 or /20
• Obtain addresses from upstream ISP
• If /19 or /20 cannot be justified
• Registry needs
• Network design
• Justification for address space

43
Origination vs. Advertising
AS65000
AS65000
128.0.0.0/19
AS64444
192.0.0.0/16
AS 65000
128.0.0.0/20
192.0.0.0/16 AS64444 an AS65000 Customer
/23 POP Dialups
/23 Internal
/23 Customers
/23 Customers
/25
/25
/25
/25
32 /30
32 /28
/24
/24
/25
/25
44
Aggregating your Own Traffic
AS65000
128.0.0.0/19
Suppress more specific routes unless required by
multihoming
45
Advertising with NO-EXPORT
AS63333 64.0.0.0/12
Assigns 64.0.0.0/22
Assigns 64.0.4.0/22
AS62222
AS61111
Advertises 64.0.0.0/22 NO-EXPORT
Advertises 64.0.4.0/22 NO-EXPORT 96.1.0.0/16
AS61000 96.1.0.0/16
46
Aggregation is better than Aggravation

• Blackhole routes for your blocks
• Avoid more-specifics
• Use NO-EXPORT when controlling load to upstream
• Encourage customers to aggregate
• Proxy aggregation hard to administer
• Understand which blocks you can advertise
• And do ingress/egress filtering

47
Preparing for Address Request (1)

• Address requirements of services are you offering
• Dynamic addressing
• Dialup
• Residential broadband
• Private addressing
• Enterprises homed only to you
• Dialup/broadband not offering servers
• Globally addressable

48
Prepare for Address Request (2) An ISP Topology
Switch
POP1 1 internal LAN
POP2 1 internal LAN
POP3 1 internal LAN
POP4 1 internal LAN
100 Dial Ports
8 small LANs
1 med. LAN
100 Dial Ports
8 small LANs
1 med. LAN
100 Dial Ports
8 small LANs
1 med. LAN
100 Dial Ports
8 small LANs
1 med. LAN
49
Establishing an AS (1)AS Number Request

• In request to AS number registry
• Administrative and technical contacts
• Autonomous system name
• Router description
• Deployment schedule
• Networks (by name) connected by the router(s)
• Internet addresses of the routers

50
Establishing an AS (2)Registering in Routing
Registry

• Minimum requirements
• Maintainer object
• AS object
• Route object (s)

51
Establishing an AS (3)Operational deployment

• Build configuration
• Policy implementation
• Ingress/egress filtering
• Establish security procedures
• Start BGP connections

52
Routing Registry Objects

• Basic
• AS
• Route
• Maintainer
• Additional
• Inter-AS Network
• Community
• Router

Refinements
53
Operational Relationships 3Registries, Domains,
etc.
Address authority
Reverse DNS
Route objects
AS
DNS
Address delegation
Prefixes
Hosts
54
Autonomous System

• Basis of exterior routing
• AS originate routes for some prefixes they want
  to be visible
• AS advertise routes to one another
• Advertisement may not contain all addresses
• Not all advertisements need be accepted

55
Current AS DefinitionRFC 1930

• Connected group of IP CIDR blocks
• Run by one or more network operators
• Single routing policy
• announced to the general Internet
• announced with BGP-4

56
AS Number

• 16 bit number
• 32 bit under discussion
• Numbers assigned by registries
• Routing policy should be stored in registry
• ISPs can mirror routing registry -- place for
  sensitive data
• Private ASNs
• 64512 through 65535
• Private AS stripping, confederations

57
Operational Relationships 2Addesses and
Autonomous Systems
Address authority
Reverse DNS
AS
DNS
Address delegation
Prefixes
Hosts
58
Full Employment for Consultants Policies are
inside Routers

• Advertising Policies
• Outbound to other AS
• BGP advertisement sources
• Outbound route filters
• Route must be in internal routing table
• Acceptance Policies
• Inbound AS filters
• Inbound route filters

59
Stop! What are you going to Advertise?

• Routes Assigned/Allocated to You
• Routes Assigned/Allocated to Customers
• Routes for which you provide Transit

60
Advertising Affects

• The way the world sees you/sends to you
• Binary
• Routes to which you provide routing
• Quantitative Preferences
• Multi-Exit Discriminators to your Neighbors
• AS Path Manipulation to all

61
Routes Eligible to Advertise

• Are reachable by your IGPor static routes
• Unless they are black holes
• Which conceptiually are reachable
• Do not advertise
• Spoofed source addresses
• Your internal addresses
• RFC1918 space
• Known rogues?
• RBL?

62
Stop! What are you going to Accept?

• It depends
• Only those routes you will do something about
• Otherwise default

63
Do Not Accept

• RFC1918 source or destination
• Unexpected sources not assigned/allocated to
  peers
• Your internal addresses from peers

64
Turning it On
65
BGP Configuration Overview

• Plans and policies first!
• Define system of BGP speakers
• Specific BGP speaker configuration
• Identifier
• BGP process
• Neighbors
• NLRI to advertise
• Filters and other policy mechanisms

Cisco commands used as examples
66
Policy Implementation Flow
67
Policy vs. Protocol Flow
AS1 R1
AS21 R1
AS1 R1
AS21 R1
AS1 R2
AS21 R2
All equivalent from a policy standpoint!
68
BGP Configurations

• Know global information (AS, policies, etc.)
• Establish router ID
• Create BGP process
• Identify internal and external peers

69
Router ID and loopback interface
interface loopback 0 ip address 192.168.0.1
255.255.255.0
70
Refining the Configuration

• Single and Multiple Links
• to a Single Provider

71
The BGP Tunnel
Serial 0
Serial 0
Loop 0
Loop 0
ebgp-multihop needed when neighbor is not on same
subnet
72
Load Balancing 1IP Level to Single Provider
Router
Customer AS
Provider AS
Serial 0
Serial 0
Loop 0
Loop 0
73
Load Balancing 1Multiple Routers
Customer AS
Provider AS
74
Another Non-BGP AlternativeOSPF Routing Domain
ISP 1
Static routes
D1-A0 ASBR1
D1-A0 ASBR2
Default Route (0.0.0.0/0) Metric Type 1 Equal
Metrics
75
Multiple OSPF Defaults
Static routes
D1-A0 ASBR1
D1-A0 ASBR2
Default Route (0.0.0.0/0) Metric Type 2
Higher Metric to ISP 2 (Backup)
76
Blackhole Route

• Establish static route to your block(s)
• ip route 1.2.3.4 255.255.240.0 null0
• Redistribute/import into BGP
• Suppress more-specific prefix advertising

77
Effects of Blackholing

• No route flapping outside your AS
• If your internal routes go up or down
• Incoming traffic for specific routes that are
  down
• Doesnt match any internal route
• Automatically discarded without concerning anyone
  else

78
BGP Path Selection
79
Next Hop Access
Advertised route via R1
R1
X
R2
Advertised route via R2
80
ScopeMED vs. Local Preference vs. Weight
AS1
AS2
Local Preference
Weight
Weight
MED
81
Administrative Weight (Cisco extension)
Advertised route via R1
R1
X
R2
Advertised route via R2
Rules in this router set R1 weight to 100, R2
weight to 500
82
Weight examplefor load sharing
Default local preference 200
Primary ISP
Backup ISP
Default local preference 500 All routes
AS_Backup local preference 100
83
Tiebreaker for Equal WeightLocal Preference
Advertised route via R1, local preference 100
R1
R2
Advertised route via R2, local preference 500
84
Local Preference example for load sharing
Default local preference 200
Primary ISP
Backup ISP
Default local preference 500 All routes
AS_Backup local preference 100
85
Prefer locally originated routes
Advertised route via R1
R1
R2
Locally defined via R2
86
AS Path
87
Shortest AS Path (Cisco extension)
R1
AS
AS
AS
AS
Route
R2
AS
AS
Route
88
Full Employment For ConsultantsInterpreting AS
Path

• Default assumption local preference set based
  on AS_PATH
• Cisco considers it as part of the algorithm

89
AS Path Prepending

• Applies to routes you advertise
• Makes them less attractive to others
• Increases AS_PATH length
• your AS put in the path twice

90
Limitations of Prepending
6
91
External Paths Preferred
Route Learned from iBGP
R1
R2
Route Learned from eBGP
92
Lowest MED
Remote AS
MED500
R1
R2
MED100
93
Full Employment For ConsultantsWeight, Local
Preference MED

• HIGHER value wins
• Weight
• Local preference
• LOWER value wins
• MED
• Cisco default route with no MED preferred
• IETF route with no MED least preferred

94
Full Employment For ConsultantsScope of MED

• Default assumption
• MEDs only compared between exits to the same
  adjacent AS
• Alternate always-compare-MED
• Useful at exchange points, possibly private
  peerings
• Cisco knob

95
Closest Neighbor
IGP metric to R1500
R1
R2
IGP metric to R1100
96
Lowest BGP router ID
R1 1.1.1.1
R2 2.2.2.2