Title: Design of a Diversified Router: Model and System Overview
1Design of aDiversified RouterModel and System
Overview
Jon Turner, John DeHart, Fred Kuhnsjon.turner_at_wu
stl.edu, jdd_at_arl.wustl.edu, fredk_at_arl.wustl.edu ht
tp//www.arl.wustl.edu/arl
2Outline
- What is NOT covered in these slides
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
3Not Covered
- Control
- Installation
- Configuration
- Initialization
- Monitoring
-
- End Host
- Substrate/Meta Model
- Details about how PlanetLab works.
- These are very important topics but for now we
will talk about the Data Path in the Network.
4Outline
- What is NOT covered in these slides
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
5System Architecture
- General purpose blades.
- shared blades run Plab OS
- no change to current apps
- also support dedicated blades
- use separate blade server to preserve ATCA slots
for NPs - NP blades.
- support dedicated PEs
- control from Vserver on PE/GP
- shared PE options
- shared NP for fast path
- shared NP with plugins
- 10 GE fabric switch
- VLANs used to isolate metarouters
- uplinks for connecting to multiple chasses
- Good ratio of PEs to LC 31
compute blade with disk
Radisys7010
Radisys 7010 with RTM
1 GE for control 10 Gb/s for data
6Sharing NPs
- Motivation
- MRs with limited IO bandwidth cant make full
use of NP blade - extreme (but perhaps common) case 25 MRs with
100 Mb/s IO - useful to allow such MRs to share a PE
- Packet flow
- ingress LC terminates external tunnel and adds
internal header - MR fast path determines outgoing metalink and
queues packet - selected packets directed to Vserver on shared GP
blade - injected packets placed on metalink queues in PE
- Configuration
- single NP blade hosts two shared NPs
- each NP engineered for 5 Gb/s throughput
- divide TCAM between NPs
7Shared PE Processing
- code constraints
- finite, acyclic flow graph
- bounded path length
- restricted memory accesspacket buffer, control
block, output area - thread-safe
- inputs (in regs)
- buffer pointer
- output MI
- ctl blk pntr
- lookup result pntr
- output
- buffer offset
use MR to find parser code and MR control block
header includes MRMI
- inputs (in regs)
- buffer pointer
- MI
- ctl blk pntr
- output pntr
- output
- 16B lookup key
per metalink queues with static rates
- input MRlookup key
- output
- output MI
- qid (supplied by substrate)
- lookup result
- stats index
- Stats module not shown inputs from DeMux,
Lookup and QM. - Potential for plugins using spare MEs
- one ME per MR, static SRAM regions,
8Managing MR-Specific Data
- MR control software runs on separate PE.
- xScale provides interface through control daemon.
- enable/disable MR (input packets discarded when
disabled) - read/write MR control block
- block read/writes to from memory area using
offsets - read/write filters
- substrate fills in hidden fields (MR in key, qid
in result) - read statistics
- read/write code segments
- cryptographic hash used to verify
compliance-checking - only when MR is disabled
9Line Card Functions
- Substrate functions only
- Ingress
- terminate tunnels (VLAN, plain IP, IP tunnel,
MPLS) - map physical interface tunnel id MLI to MRMI
- map MRMI to destination blade and qid
- queue packets in queues with static rates (paced)
- provision for xScale to respond to ARP requests
- Egress
- arriving packets include MRMI(next-hop-metanet-a
dr) - monitor rates for each MR/MI pair raise alarm
if threshold exceeded - if MI is designated as multipoint,
- map next-hop-metanet-adr to MAC adr and write
into blank spot in buffer - if no matching entry, pass buffer pointer to ARP
handler in xScale - xScale handles ARP, fields reply and enqueues
packet when reply received - if no reply, send exception notification MR
exception interface - place packet in outgoing per-MI queue
10Possible Extensions
- Queueing in shared PE
- allow MR to define multiple queues per MI with
queue weights, per queue space allocation and per
MI space allocation - substrate implements fair queueing scheduler
- WDRR or stratified round-robin
- packet discarded if queue using more than its
space allocation and more than its share of the
shared space - implement when users demand it or spare time
available - Limited Plugins
- replace Header Formatter with parallel block of
MEs, one per MR - code restrictions (static and run-time checks)
- access to restricted address ranges
- restricted use of bus bandwidth resources
- instructions inserted to increment usage counters
- xScale (or perhaps dedicated ME) checks counters
periodically - MRs using excessive bus bandwidth are disabled
11Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
12Schedule
- Two time horizons
- techX Project
- End of 2006
- Future Diversified Router Project
- Outstanding new grant request under which we do
the rest of the work - JDDs Proposal techX End of Year Target
- Data Path
- Default IPv4 MR
- We should be able to use N instances of this to
show multiple MRs running in a Substrate Router. - Legacy PlanetLab nodes in a Blade Server
- By Legacy we mean receiving/sending plain IP
packets, no Meta/Substrate functionality present. - Basic LC Substrate functionality
- IP Tunnel Substrate Link termination
- Plain IP handling
- MetaLink Loopback Block
- Control?
- Install an MR
- Configure an MR?
- Routing Protocols?
13Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
14Substrate/MetaNet Model
- The Substrate should provide a view to Meta Nets
that their Meta Routers are directly connected
via Physical Links - A Meta Interface emulates a Physical Interface
- A Meta Link emulates a Physical Link
- The Substrate should provide isolation between
Meta Nets
Meta Net A
MR_A
MR_A
MR_A
MI
MI
MI
MI
MI
MI
Meta Net B
MR_B
MR_B
MI
MI
MI
MI
Meta Net C
MR_C
MR_C
MR_C
MI
MI
MI
MI
MI
MI
15Substrate/MetaNet Model
- Substrate Links connect Substrate Router Peers
- A Substrate Link is terminated at a Physical
Interface of a Substrate Router. - Meta Links connect Meta Router Peers
- A Meta Link is terminated at a Meta Interface on
a Meta Router
MetaLink 1
Substrate Link
MetaLink 2
MetaLink 3
MR_A
MR_A
MR_A
MI
MI
MI
MI
MI
MI
MR_B
MR_B
MI
MI
MI
MI
MR_C
MR_C
MR_C
MI
MI
MI
MI
MI
MI
Substrate Router X
Substrate Router Z
Substrate Router Y
16Mapping the Model onto Hardware
Meta
Dedicated NP
Dedicated GP
Shared NP
Shared GP
Switch
LC
Loopback
Substrate
17Mapping the Model onto Hardware
- Line Card
- Typically supports multiple Physical Interfaces
- Our Radisys blade when implementing a LC would
have an RTM with 10 1-GE physical interfaces. - Substrate Link Termination
- Including IP Tunnel Substrate Link termination
- Mux/Demux of Meta Links into/out of Substrate
Links. - Pass through Meta Links
- Plain IP
- Switch
- Substrate Switch
- Meta Routers on shared blades do not have a view
of the switch. - All switching functions are provided by the
Substrate - Traffic from all MRs is isolated from other MRs,
including dedicated blade MRs, via the use of
VLANs - Meta Switch
- An MR on a dedicated blade has a view of a
Meta-Switch including - Port its blade is on
- Portions of LCs that it has access to.
- Loopback (next slide)
- NP Network Processor Blade
18Mapping the Model onto Hardware
- Line Card
- Switch
- Loopback
- Provides MRxMIi connectivity to MRyMIj within
Substrate Router - More on this later when we provide details on the
components - NP Network Processor Blade
- Dedicated entire blade is dedicated to one MR.
- No Substrate functionality present.
- Shared blade is shared among several MRs.
- Substrate functionality provides resource sharing
and isolation - GP General Purpose Processing Blade
- May be realized by a blade in the ATCA chassis or
in a blade server directly connected to switch
blade which resides in ATCA chassis. - The switch blade has front panel Ethernet
interfaces (uplinks) that could be connected to
the blade server for this purpose. - Dedicated entire blade is dedicated to one MR.
- No Substrate functionality present.
- Shared blade is shared among several MRs.
- Substrate functionality provides resource sharing
and isolation.
19Model Layering and Abstractions
- Two Types of Traffic
- Legacy
- MAC layer
- Legacy Layer (i.e. non-Substrate, typically IPv4)
- Substrate
- MAC layer
- Substrate layer
- Meta layer
- Figures below are LAYERS, not Packet formats
20Layer Attributes
- Attributes at the different layers
- Physical/MAC
- Type (Ethernet, ATM, )
- Wireless (1/0)
- Type (P2P, MA, )
- Substrate
- MNid
- MRid
- MLI
- Link Type
- Link Model (P2P, Multi-Access)
- ARP support
- Meta
- MetaLink
- MI
- MnAddr
- What is exposed to the MR?
- Type Ethernet, Wireless,
- P2P vs. Multi-Access
21Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
22Traffic at a Substrate Router
- For now assume traffic is on wired Ethernet.
- Substrate
- EtherType Substrate
- Each frame has a Substrate Header
- MLI Meta Link Identifier
- Identifies MRMI to which this frame should be
delivered - Len
- Line Card
- sends Meta Frame to the Blade hosting the MR
indicated by the MLI - includes in the Meta Header the MRs MI that this
frame is for. - IP
- Other
23Traffic at a Substrate Router
- Substrate
- IP
- EtherType IP
- Substrate Tunnels
- IP Proto Substrate
- Each frame has a Substrate Header immediately
following IP Header - MLI Meta Link Identifier
- Identifies MRMI to which this frame should be
delivered - Len
- Line Card sends Meta Frame to Blade hosting MR
indicated by MLI - Included in Meta Header is the MRs MI that this
frame is for. - Plain IP Control Protocols
- Contains no Substrate Header
- ICMP
- Substrate on LC may need to process some ICMP
messages. - ICMP or a subset of ICMP messages sent to XScale
on LC. - Depending on processing in Substrate/XScale, ICMP
messages may be forwarded on to MR(s). - BGP Border Gateway Protocol (inter-domain
routing protocol) - Treated like Plain IP Data described below Sent
to default IP MR.
24Traffic at a Substrate Router
- Substrate
- IP
- Substrate Tunnels
- Plain IP Control Protocols
- Plain IP Data
- Contains no Substrate Header
- Line Card forms Meta Frame and sends it to
default IP MR - A unique MI for the default IP MR is associated
with each Physical Interface. - Via the default IP MR, plain IP traffic may be
destined for - Other MRs on this Substrate Router via the
Loopback - Other MRs on other Substrate Routers via Meta
Links carried on Substrate Links - The Default IP MRs on peer Substrate Routers
could exchange route information so they know how
to get to other Substrate Routers. - Legacy (Pure IP) Planet Lab Nodes on this
Substrate Router. - Substrate Control Entities on XScale?
- Egress as Plain IP?
- Policy based option
- Other?
- Other
- ARP
25Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
26IP Address Assignment
Substrate Router(A.B.C.0 A.B.C.255)
PlanetLab Blades
X.Y.Z.1
1
U.V.W.1
2
- Consider a Substrate Router (SR) with the
following characteristics - 3 physical interfaces that connect to three
Internet Routers - Interface 1 ?? X.Y.Z.1
- Interface 2 ?? U.V.W.1
- Interface 3 ?? R.S.T.1
- Blade server that contains 6 Legacy (Plain IP)
PlanetLab Nodes - A Default IPv4 MR
- Block of IP Addresses A.B.C.0 A.B.C.255
R.S.T.1
3
27IP Address Assignment
Substrate Router(A.B.C.0 A.B.C.255)
PlanetLab Blades
X.Y.Z.1
A.B.C.1
A.B.C.128
X.Y.Z.2
1
U.V.W.1
A.B.C.129
U.V.W.2
2
- Assign addresses to the SR as follows
- Each Physical Interface has a Substrate IP Tunnel
Address - Interface 1 ?? A.B.C.128 (A.B.C.0x80)
- Interface 2 ?? A.B.C.129 (A.B.C.0x81)
- Interface 3 ?? A.B.C.130 (A.B.C.0x82)
- Default IPv4 MR has 4 IP Addresses
- To X.Y.Z.1 out Interface 1 X.Y.Z.2
- To U.V.W.1 out Interface 2 U.V.W.2
- To R.S.T.1 out Interface 3 R.S.T.2
- To PlanetLab Nodes A.B.C.1
R.S.T.1
A.B.C.130
R.S.T.2
3
28IP Address Assignment
Substrate Router(A.B.C.0 A.B.C.255)
PlanetLab Blades
X.Y.Z.1
A.B.C.1
A.B.C.128
X.Y.Z.2
1
U.V.W.1
A.B.C.129
U.V.W.2
2
- Route announcements
- Out Interface 1
- A.B.C.128/32
- A.B.C.0/25
- Out Interface 2
- A.B.C.129/32
- A.B.C.0/25
- Out Interface 3
- A.B.C.130/32
- A.B.C.0.25
R.S.T.1
A.B.C.130
R.S.T.2
3
29BGP Notes
- RFC 4271 BGP-4
- BGP is defined as an inter-Autonomous System (AS)
routing protocol. - Autonomous System A set of routers under a
single technical administration - BGP Speaker A router that implements BGP
- BGP Identifier an IP Address assigned to a BGP
Speaker. Same for every local interface and BGP
peer. - A router may also have separate IP Addresses for
its interfaces. - UPDATE message used to exchange route information
- NEXT_HOP attribute defines IP address of the
router that SHOULD be used as the next hop to the
destinations listed in the UPDATE message. - Would this potentially be used in ARP requests?
- BGP uses TCP between peers.
- From RFC 4271, Section 3
- In the context of this document, we assume that a
BGP speaker advertises to its peers only those
routes that it uses itself (in this context, a
BGP speaker is said to use a BGP route if it is
the most preferred BGP route and is used in
forwarding).
30Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- MetaLink Loopback Block
- Switch
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
31Meta Link Loopback
- We would like to be able to route Plain IP pkts
to an MR - We can achieve this by using a default IPv4 MR
- But we dont want the IPv4 MR to have to know
anything about other MRs and their
Meta-Interfaces - That is a Substrate function
- So, we need a Substrate function to translate
from - MRxMIi to MRyMIj
- We would also like to be able to route Plain IP
pkts to Legacy PlanetLab Nodes residing on Blades
in a Substrate Router - Legacy PlanetLab Nodes have no Substrate.
- Again we can use the default IPv4 MR to achieve
this. - But the packets cannot have a Substrate Header or
Meta Header when they arrive at the PlanetLab
Nodes. - So, we need a Substrate function to strip these
headers. - We would also like to be able to do similar
things with IP Pkts in Substrate Frames on
MetaLinks which terminate at the default IPv4 MR - We will implement a Meta Link Loopback block
- Provides Internal Meta Link Loopbacks at the
Substrate level. - Probably will reside on one or more Line Cards.
32Meta Link Loopback
Physical interface n on LC A
Substrate Router
Def IPv4 MR
LC A
n
m
Physical interface m on LC A
MR Y
n
LC B
Physical interface n on LC B
MR X
Loopback
PLab
Switch
33Meta Link Loopback Plain IP to/from MR Y
Substrate Router
Def IPv4 MR
IPAm
VLAN a used.
IP
IPBn
a
IP
Ab MI b associated with MR A used
LC A
n
IPYk
IP
m
IPAm
Plain IP
IP
MR Y
Yk
n
IPBn
Y
Plain IP
LC B
IP
MR X
Loopback
IPYk
IP
Yk
Y
PLab
Switch
34Meta Link Loopback Plain IP to/from PLab
Substrate Router
Def IPv4 MR
IPAm
VLAN a used.
IP
IPBn
a
IP
IPPi
IP
Ab MI b associated with MR A used
LC A
n
m
Plain IP
IP
IPAm
MR Y
n
IPBn
Plain IP
LC B
IP
MR X
IPPi
Loopback
PLab
Plain IP
P
Plain IP
P
Switch
35Meta Link Loopback General
IPAn
Substrate Router
Def IPv4 MR
IP
IPAm
VLAN a used.
IP
IPBn
a
IP
IPPi
IP
IPXj
Ab MI b associated with MR A used
LC A
n
Substrate
IPAn
IP
IPYk
IP
IP
m
Plain IP
IP
IPAm
MR Y
Yk
n
IPBn
Y
Plain IP
LC B
IP
MR X
Xj
X
IPPi
IP
Loopback
IPXj
IPYk
Yk
Y
Xj
X
PLab
Plain IP
P
Plain IP
P
Switch
36Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- MetaLink Loopback Block
- Switch
- NP Blades
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
37Switch
- Switch Blade Specs
- Promentum ATCA-2210
- http//www.radisys.com/products/ds-page.cfm?produc
tdatasheetsid1191 - 20-port 10GE fabric switch
- 14 10GE links to user slots
- 4 10GE links for external connections (up/cross
links) on front panel - 24-port 1GE Base switch
- 14 1GE links to users lots
- 1GE link to redundant switch blade
- 1 10GE and 4 1GE links for external connections
(up/cross links) on front panel - Wire-speed L2 and L3 switching
- 4K IEEE 802.1Q VLANs
- Etc
- Traversing the Switch
- Switching is based on Ethernet Destination
Address - Isolation is based on VLAN.
- One VLAN will be assigned to each MetaNet present
on a Substrate Router. - All switch traffic for a MetaNet will be required
to use its assigned VLAN. - Frames from a MetaNet will only be transmitted to
a port which is allowed to receive the specified
VLAN.
38Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- MetaLink Loopback Block
- Switch
- NP Blades
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
39NP Blades Radisys 7010
40NP Blades Radisys 7010
10Gb Per Port Ethernet Switch Blade
ATCA Backplane
16 /
FIC
NPUA
SPI-4 Switch
/ 16
1 /
10G MAC
10Gb Per Port Ethernet Switch Blade
/ 1
16 /
NPUB
/ 16
- SPI-4 Switch supports 6 Interfaces
- Each Interface can support up to 16 SPI-4 Ports
- Each Port on an Interface can be switched to any
other Port on any other Interface - Radisys 7010 uses 4 of the 6 Interfaces
- 10G MAC Chip (IXF18105) supports and uses only 1
SPI-4 Port on its SPI-4 Interface - Therefore, all traffic from 10G MAC Chip must go
to one NPU, either NPUA or NPUB. - This should be fine for our LC in the future
- One NPU will be for Ingress and one for Egress
- For our NPE Blades, this will be a bit of pain.
- We can direct all traffic to NPUA and then have a
block right after RX that splits the traffic for
NPUA and NPUB - Traffic for NPUA continues on to the processing
blocks on NPUA - Traffic for NPUB would go directly to a TX block
to be transmitted to NPUB via the SPI Switch.
(Does not go to 10G MAC chip) - Still need to work out whether this will actually
work or not.
41NP Blades Radisys 7010
- Radisys 7010
- Two IXP2850 NPs
- 1.4 GHz Core
- 700 MHz Xscale
- Each with
- 3x256MB RDRAM, 533 MHz
- 3 Channels of QDR II SRAM, 8MB each, 200 MHz
- 16KB of Scratch Memory
- 16 Microengines
- 8KB instruction store
- 640 32-bit words of local memory
- Network Search Engine on 4th QDR channel
- Shared between NPs
- IDT75K72234
- 18Mb TCAM
- SPI-4 Switch for interconnections
42Line Cards Radisys 7010 with RTM
- Radisys 7010 with RTM
- Same board as NP blade.
- Two IXP2850 NPs
- One used for Ingress
- One used for Egress
- Rear Transition Module (RTM)
- Connects via ATCA Zone 3
- 10 1GE Physical Interfaces
- Supports Fiber or Copper interfaces.
43Radisys 7010 Blade
44Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
45Substrate Links
- 2 Types of Substrate Links
- Point-to-Point (P2P)
- Connects two peer Substrate Routers
- May be realized via
- Directly Connected (P2P-DC)
- One physical link connects an Interface on one
Substrate Router to an Interface on another
Substrate Router - Can use VLANs or not.
- Tunnel (P2P-Tunnel)
- Two Substrate Routers have a Tunnel defined that
connects an interface on each router. - Such a tunnel comprises exactly one Substrate
Link. - Can use VLANs or not.
- Multi-Access Network (P2P-VLAN0)
- Substrate Routers with interfaces connected to a
Multi-Access Network all use a predefined VLAN
(VLAN0). - VLAN0 is defined for each Multi-Access Network
and my be different on each one. - Each of these Substrate Routers is pre-configured
with the Ethernet addresses of the other
Substrate Routers connected to the Multi-Access
Network. - By using the Ethernet address of peers, pairs of
Substrate Routers have defined Substrate Links. - With N Substrate Routers, there are at most
(N(N-1))/2 Substrate Links - Multi-Access
46Substrate Links
- 2 Types of Substrate Links
- Point-to-Point (P2P)
- Multi-Access
- Supports Unicast, Multicast and Broadcast traffic
from Meta Routers. - Interfaces on Substrate Routers connected to a
Multi-Access network are NOT necessarily
pre-configured with each others Ethernet
addresses. - ARP may be used over the Multi-Access network to
do the translation from MetaNet Address to
Ethernet Address. - ARP protocol and tables are managed by XScale.
- Can use VLANs or not, but cannot use VLAN0.
47Substrate Links
- Substrate Link Restrictions
- P2P-DC does not co-exist on a physical interface
with any of the other types of SL. - P2P-Tunnel, P2P-VLAN0 and Multi-Access may all
co-exist on the same physical interface. - A physical interface may have
- At most one P2P-DC SL
- XOR
- One or more P2P-Tunnel SLs
- One or more P2P-VLAN0 SLs
- At most one Multi-Access SLs
48IP Tunnel Substrate Links
- Each IP Tunnel Substrate Link is terminated at a
single Physical Interface on a Substrate Router. - The only route TO this end of the tunnel is from
the outside into the associated physical
interface. - Nothing inside the Substrate Router routes TO
this end of the Tunnel - From inside the Substrate Router packets are just
sent INTO the tunnel so they come out the other
end. - Included in the configuration of an IP Tunnel
Substrate Link is the Address (IP or MAC?) of the
next/first hop for the route to the other end of
the tunnel. - What happens if the routes in the Internet change
and this next/first hop is no longer the way to
go? - Physical Interface will receive ICMP No Route to
Host or ICMP Redirect messages. - ICMP messages should be sent to the XScale on the
Line Card. - XScale should then nullify (mark as down, ) the
IP Tunnel Substrate Link and all
MetaLinks/MetaInterfaces associated with that
Substrate Link.
49Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- Focus is on wired Ethernet
- LC Rx/Tx Design Implementation
- Common Router Framework (CRF)
- Functional Blocks for implementing a Router
50Point-to-Point Directly Connected (P2P-DC)
LC
LC
MR
MR
MetaLinks
MR
MR
- P2P should not need the TypeSubstrate since it
is a point-to-point physical link and should only
be carrying Substrate traffic. - Configuration/Policy/Implementation option?
51Point-to-Point via Tunnel (P2P-Tunnel)
Tunnel Substrate Link
MR
MR
LC
LC
MR
MR
MetaLinks
P2P-Tunnel SL Packet With VLANs
P2P-Tunnel SL Packet Without VLANs
- This example shows a Substrate Link in an IP
tunnel over Ethernet. - We could also envision other tunnel types
- MPLS
- ATM
- Sonet?
- Etc
- Ether/MAC layer encap ? Tunnel Protocol
- Tunnel layer encap ? Substrate
52MPLS Tunnels
- Weve looked at MPLS just enough to convince
ourselves that we can do something, but not
enough to understand all the implications. - Some of the MPLS RFCs
- 3031 Multiprotocol Label Switching Architecture
- 3032 MPLS Label Stack Encoding
- 3037 LDP Applicability
- LDP Label Distribution Protocol
- 4221 Multiprotocol Label Switching (MPLS)
Management Overview - MPLS uses a 32-bit field which contains
- Label (20b)
- Experimental (3b)
- S-Bit (1b) 1 indicates bottom of label stack
- TTL (8b)
53Point-to-Point via Multi-Access Network
(P2P-VLAN0)
- For this example, on each LC pictured, one
interface is being used.
Ethernet Switch
LC
MR-1
MR-2
LC
MR-2
LC
MR-1
MR-4
MR-2
M3
LC
MR-4
MR-2
MR-3
54P2P-VLAN0 Packet Format
P2P-VLAN0 SL Packet (VLAN0)
- VLANVLAN0 provides isolation from other traffic
on the Multi-Access network.
55Multi-Access
- One MLI used per MetaNet on a Multi-Access
Substrate Link. - We may want to use the MnID for the MLI for each.
Ethernet Switch
LC
MR-1
MR-3
LC
MR-3
LC
MR-1
MR-2
MR-2
LC
MR-1
MR-2
56Multi-Access Packet Formats
Multi-Access SL Packet with VLANs (VLAN?0)
Multi-Access SL Packet without VLANs
57ARP for IP Packet Format
- Ethernet Header
- Dst Addr
- Src Addr
- Type 0x0806 (ARP)
- Hard Type Type of hardware address
- 1 ? Ethernet
- Proto Type Type of protocol address being mapped
- E.g. for IP 0x0800 (same as EtherType for IP)
- Hard/Proto Size size in bytes of address fields
- OP Operation
- 1 ARP Request
- 2 ARP Reply
- 3 RARP Request
- 4 RARP Reply
- Sender Eth Address
- Sender IP Address
- Target Eth Address
- Target IP Address
58ARP for MetaNet Multi-Access Packet Format
- Ethernet Header
- Dst Addr
- Src Addr
- Type 0x0806 (ARP)
- Hard Type Type of hardware address
- 1 ? Ethernet
- Proto Type Type of protocol address being mapped
- Hard/Proto Size size in bytes of address fields
- OP Operation
- 1 ARP Request
- 2 ARP Reply
- 3 RARP Request
- 4 RARP Reply
- Sender Eth Address
- Sender MetaNet Address (MNidMnAddr)
- Target Eth Address
- Target MetaNet Address (MNidMnAddr)
59Other Access Technologies
- ATM?
- Sonet?
- Wireless
- How would we characterize a wireless access
network? - How would we provide substrate links on a
wireless network? - Multi-Access network without any mechanism for
isolation and differentiation of P2P and
Multi-Access Links - Two possibilities
- We do not support P2P on such a network
- We use one bit in the MLI (or one bit somewhere
in some header) to indicate P2P vs. Multi-Access.
60Summary Ethernet Pkt Formats with VLANs
Ethernet Hdr
Tunnel Hdr (IPv4)
Substr. Hdr
Meta
Ethernet Trailer
61Summary Ethernet Pkt Formats without VLANs
P2P-Tunnel SL Pkt Without VLANs
P2P-VLAN0 SL Pkt
Multi-Access SL Pkt without VLANs
P2P-DC SL Pkt Without VLANs
DstAddr (6B)
DstAddr (6B)
DstAddr (6B)
Ethernet Hdr
SrcAddr (6B)
SrcAddr (6B)
SrcAddr (6B)
TypeIP (2B)
TypeSubstrate (2B)
TypeSubstrate (2B)
MLI (2B)
MLI (2B)
LEN (2B)
LEN (2B)
Tunnel Hdr (IPv4)
Meta Frame
Meta Frame
MLI (2B)
Substr. Hdr
LEN (2B)
Meta Frame
Meta
PAD (nB)
PAD (nB)
PAD (nB)
Ethernet Trailer
CRC (4B)
CRC (4B)
CRC (4B)
62Outline
- JSTs original slides
- Schedule
- Model
- Traffic Types
- IP Addressing
- Components
- Switch
- MetaLink Loopback Block
- LC
- Substrate Link Types
- Packet Formats
- LC Rx/Tx Design Implementation (Next set of
slides) - Common Router Framework (CRF)
- Functional Blocks for implementing a Router
63Extra
- The next set of slides are for templates or extra
information if needed
64Text Slide Template
65Image Slide Template
66Multicast Alternatives
- Provide support in CRF for Multicast
- Allow MRs to provide code to interpret Lookup
Result. - Define a default Unicast format for a Lookup
Result - A pure Unicast MR could use this exclusively and
would not need to provide any code to interpret
Lookup Result. - Type bit indicates Default format
- CRF Specific Data
- QID
- Stats Index
-
- MR Related Data Used by CRF
- MI
-
- MR Specific Data Opaque to CRF
- Whatever the MR developer wants
- Define an MR-specific format
- Type bit indicates MR-Specific
- MR supplies a block of code to interpret result
(next slide)
67Core Components
- Functionality
- Initialization
- Table management
- Route/Filter tables
- L2 Tables
- MR State Table (in Demux)
- MetaRouter Installation/Removal
68Core Components (sample App)
69Core Components
MetaRouter Management Core
MR Specific Core Components ???
MR Code Management
GP PE?
MR Lookup Tbl Mgmt
MR State Tbl Mgmt
MR L2 Tbl Mgmt
RX Core
DeMux Core
Parse Core
Lookup Core
Hdr Fmt Core
QM Core
TX Core
Portability Framework Core Components
Infrastructure
Portability Framework Resource Manager
XScale
Micro- Engines
DeMux
Parse
Lookup
HeaderFormat
Rx
Tx
70System Configuration
NumGP
NumLC
NumPE
Blade Server
e.g. 10 GE Interfaces Per LC
Switch Blades (2)
- NumLC NumPE NumGP 2 (switch) lt 14
- 14 of slots in a full ATCA Chassis
- Good ratio of PEs to LCs is probably 31
- Blade Server may take the place of the GP Blades
- Connected through Uplinks on Switch Blade(s)
71MetaLinks
- Types of MetaLinks
- May be directional and asymmetric.
- Normal
- Frames enter with a Substrate Header(MLILEN) and
Meta Frame - Frames are sent to a MetaRouter on a Processing
Engine - Frames leave with a Substrate Header(MLILEN) and
Meta Frame - Gateway
- Frames either
- Enter an interface as Legacy and get delivered to
an MR as Meta Frames - OR
- Leave an MR as Meta Frames and get sent out an
interface as Legacy - Pass-Through
- Frames enter with a Substrate Header and Meta
Frame and leave with a Substrate Header and Meta
Frame without going to a MetaRouter. - i.e. they go LC to LC without visiting a
Processing Engine. - No need for Legacy Pass-Through
72MPLS Tunnels
- Weve looked at MPLS just enough to convince
ourselves that we can do something, but not
enough to understand all the implications. - Some of the MPLS RFCs
- 3031 Multiprotocol Label Switching Architecture
- 3032 MPLS Label Stack Encoding
- 3037 LDP Applicability
- LDP Label Distribution Protocol
- 4221 Multiprotocol Label Switching (MPLS)
Management Overview - MPLS uses a 32-bit field which contains
- Label (20b)
- Experimental (3b)
- S-Bit (1b) 1 indicates bottom of label stack
- TTL (8b)
- There are some special labels defined, for
instance - IPv4 NULL Label
- when at bottom of stack indicates that the
encapsulated packet is IPv4 and should be routed
as such. - How would we use it?
- We could define a similar special label for
Substrate. - Our Substrate Routers would need to participate
in the MPLS label distribution protocols and
such. - The sending Substrate Router would push the
special Substrate label onto the bottom of the
stack and then push the first routing label onto
the stack.
73OLD
- The rest of these are old slides that should be
deleted at some point.