Analyzing Single Buffered Routers

1
Analyzing Single Buffered Routers
Sundar Iyer, Rui Zhang, Nick McKeown (sundaes,
rzhang, nickm)_at_stanford.edu High Performance
Networking GroupDepartments of Electrical
Engineering Computer Science, Stanford
University
2
What is an Ideal Router?
R
R
NR
1
1
NR
R
R
N
N
BW N2R
Departing Packets
Arriving Packets
Interconnect
Memory
Output Queued Switch

• Output Queued switches are ideal but not
  practical
• It minimizes the delay faced by a packet
• Can give QoS guarantees
• The bandwidth to each output is NR, the total
  bandwidth is N2R
• The cost and power consumption is prohibitive

3
CIOQ Models
Arbiter
Arbiter
R
R
R
R
1
1
1
1
2R
R
R
R
R
R
R
2R
R
R
R
N
R
N
N
N
BW 2NR
BW NR
N memories
N memories
Departing Packets
Arriving Packets
Departing Packets
Arriving Packets
Input Queued Switch
Combined Input-Output Queued Switch

• CIOQ switches are better but still not practical
• They can emulate OQ switches
• They need a bandwidth of only 2NR
• They have high computational complexity
• The model does not capture many different
  architectures

4
The Single Buffered Router Model

• Single Buffered Routers buffer packets only once
• The interconnects may be
• physically separate or merged
• one of the interconnects may be optional
• The memory can be
• centralized or distributed
• one or many
• reserved or shared amongst all ports

5
Why a New Model for Routers?

• SB Routers comprise a broader class of routers
• They replace the CIOQ model
• They also include other interesting router
  architectures such as
• shared memory routers, parallel packet switches
  etc.
• With this model we can compare these routers to
  an ideal router and answer
• Does a router give me quality of service?
• Can a router guarantee me 100 throughput

6
How to Compare Routers?
OQ Switch
R
R
R
Memory
R
1
1
Yes? Emulate
NR
R
R
R
R
N
N
BW N2R
Arriving Packets
Departing Packets
?
Interconnect
Any SB Switch
R
R
R
R
1
1
No
R
R
R
R
N
N
Departing Packets
Interconnect
Arriving Packets
Interconnect
Memory
7
A Modified Pigeon Hole Principle

• Consider the following
• Only one pigeon can enter or leave a hole in a
  given time
• A pigeon decides when it wants to leave
• A pigeonhole may contain many pigeons over time
• How many pigeon holes do we need so that
  departing pigeons are guaranteed to be able to
  leave, and arriving pigeons are guaranteed a
  pigeon hole?

8
The Constraint Set Technique

• A technique to analyze single buffered routers
• Determine each packets departure time
• Define the constraints on the system for both
  inputs and outputs (if applicable)
• buffer, fabrics, speedup, etc.
• Apply the Pigeon Hole Principle
• Constraint Sets can be used to analyze
• Parallel Shared Memory Switches
• Distributed Shared Memory Switches (bus-based or
  crossbar-based)
• Input Queued Switches
• Parallel Packet Switches
• .. and we expect in general any Single
  Buffered router

9
Examples of Constraints

• Physical Constraints
• These are limitations imposed by the hardware
• Memory (E.g. Parallel Packet Switch) Cant
  access a memory more than a certain number of
  times in a time period
• Bus (E.g Centralized Shared Memory) Cant use
  the same bus simultaneously for more than a
  certain number of packets
• Crossbar (E.g Distributed Shared Memory) Each
  input and output may be busy only once in a
  scheduling period
• Logical Constraints
• These are requirements imposed on the switch
• Time (Input Queued Router) A packet must face a
  delay of no more than p time slots with respect
  to an ideal switch

10
An Example Parallel Shared Memory (PSM) Router
Interconnect Fabric Bus
Interconnect Number One/Two
Interconnect Implementation Separated/ Merged
Memory Physical Location Centralized
Memory Number One/ Many
Memory Sharing Allowed Yes
NumberMemories BW per Memory TotalBW Emulate?
k 3NR/k 3NR FIFO
k 4NR/k 4NR QoS
11
Question Can a PSM Router emulate an OQ Router?

• Let a cell arrive at input i at time t and be
  destined to depart from output port j at time
  DT
• Such a cell must not be written to memories
  which
• Are used to write the other N-1 arriving cells at
  t.
• Are used to read the departing N departing cells
  at t.
• Will be used to read the N-1 departing cells at
  DT.
• There are three constraint sets
• By the pigeonhole principle, 3N memories at rate
  R, or a memory bandwidth of 3NR is sufficient

12
Distributed Shared Memory Router
Interconnect Fabric Bus/Crossbar
Interconnect Number One/Two
Interconnect Implementation Separated/Merged
Memory Physical Location Distributed
Memory Number Many
Memory Sharing Allowed Yes
S1R
S2R
No.Mem BW per Mem. TotalBW Xbarspeed Emulate?
N 4R 4NR 4NR FIFO
N 6R 6NR 6NR QoS
BW S1NR
BW S2NR
13
Question Can a DSM Router emulate an OQ Router?

• Let a cell arrive at input i at time t and be
  destined to depart from output port j at time
  DT
• The cell can be written to any intermediate port
  x such that
• The edge (i,x) is available at time t. Since, no
  more than N-1 other cells contend to write at
  time t, at least (N-1)/s1 vertices are
  available.
• The edge (x,j) is available at time DT. Since, no
  more than N-1 other cells contend to leave at
  time t, at least (N-1)/s2 vertices are available.
• There are two constraint sets
• By pigeonhole principle, if suffices that
  (N-1)/s1 (N-1)/s2 gt N.
• Hence if s1 s2 2, i.e. ss1s24 is enough.
• A bandwidth of 4NR is sufficient

14
How Complex is the Arbiter?

• For each packet, need to check k memory addresses
  for potential conflicts
• Need to maintain the bitmap for scheduled
  departures from memories
• Scheduling is done sequentially, O(N)
• Communication from linecards is minimal

15
Summary New Results, Previous Architectures,
Comparison
Emulate (QoS)
Arbiter
Xbar BW
Total BW
BW of Mem.
Num. Mem.
Fabric
Type
Yes
None
-
N(N1)R
(N1)R
N
Bus
OQ
0
-
Simple
-
2NR
2NR
1
Bus
Shared Mem.
1
No
Max. Matching
NR
2NR
2R
N
Xbar
IQ
2
Yes for FIFO, Leaky Bucket Traffic
Simple
3NR
6NR
3R
2N
Xbar
IQ (with speedup)
3
Yes
Complex
2NR
6NR
3R
2N
Xbar
CIOQ
4
Yes
Simple
-
4NR
4NR/k
k
Bus
PSM
5
Yes
Complex
5NR
4NR
4R
N
Xbar
DSM-I
6a
Yes
Simple
8NR
4NR
4R
N
Xbar
DSM-II
6b
Yes
Simple
6NR
6NR
6R
N
Xbar
DSM-III
6c
Yes
Simple
-
3N(N1)R
3R(N1)/k
Nk
Clos
PPS -OQ
7a
PPS Shared Memory
Yes
Simple
-
6NR
6NR/k
Nk
Clos
7b
Yes (FIFO)
Simple
-
4NR
4NR/k
Nk
Clos
PPS Shared Memory
7c
16
Backups
17
DSM Router Variants(Trading Arbiter Complexity
with Memory Speed)
No.Mem. BW per Mem. TotalBW Xbarspeed Emulate? Arbiter
N 3R 3NR 4NR FIFO Complex
N 3R 3NR 6NR FIFO Simple
N 4R 4NR 4NR FIFO Simple
N 4R 4NR 5NR QoS Complex
N 4R 4NR 8NR QoS Simple
N 6R 6NR 6NR QoS Simple
18
Input Queued Router
Interconnect Fabric Crossbar
Interconnect Number One
Interconnect Implementation Merged
Memory Physical Location Distributed
Memory Number Many
Memory Sharing Allowed No
Arbiter
R
R
1
1
R
R
NumberMemories BW per Memory TotalBW Emulate?
N 3R 3NR FIFO Leaky Bucket Traffic
N 3R 3NR QoS Leaky Bucket Cons.
R
R
N
N
N memories
Departing Packets
Arriving Packets
19
Parallel Packet Switch
Interconnect Fabric Clos Network
Interconnect Number Two
Interconnect Implementation Separated
Memory Physical Location Distributed
Memory Number Many
Memory Sharing Allowed Yes
20
Comparing DSM to CIOQ Routers
CIOQ DSM
Num. Mem. 2N N
Total Mem. BW 6NR 4NR
Xbar BW 4R 5R
Buffer Size NR x RTT ltlt NR x RTT

• DSM routers are less complex than CIOQ routers
• Lower requirements on memories
• Simpler scheduling algorithm
• Slightly higher crossbar bandwidth
• Two problems
• Departure times must be determined centrally
• Scheduler is sequential