ASC CRM Design

1
ASC CRM Design
John DeHartApplied Research LaboratoryComputer
Science and Engineering Departmenthttp//www.arl.
wustl.edu/arl/projects/techX
2
Advanced Services Card
RDRAM
Power
ACE
DDR SDRAM
DDR SDRAM
QDR
QDR
QDR
QDR
Power
ASP/H-1P100
40?625(160 pins)
QDR
QDR
5
DDR SDRAM
GPIO
ASP/S2850
CRMP100
QDR
Rocket IO
10 x 2.5G
16? lt800 (64 pins)
10
RDRAM
PCI Bus Signals (for control only)
DDR SDRAM
DDR SDRAM
ASP/H-2P100
GPIO
10/100 Connector
QDR
5
RDRAM
Power
40?625(160 pins)
Power
Ethernet PHY
Power
QDR
QDR
QDR
QDR
3
Target Architecture
Address width/Data width/Max Clock/Typical BW
fpga logic
High-speed Data path
DCR
DCR Bus FPGA control, PLB error
INTC Interrupt Controller
critical
IPIF (DMA/PFIFO)
non-critical
Clock Rate, 250MHz?
DMA
DCR
Arbiter
DMA
OPB/PCI bridge
PCI ????
ASP/SW (IXP)
Memory speed, 66MHz?
JTAG controller and Reset block
4
CRM Interfaces
R c k t I/O UP
S D R A M
DDR SDRAM
10
D C R B U s
2 G P I O
R c k t I/O DN
10
SelectMap
2 G P I O
P F I F O
A S P / S
16 x 800 (64 pins)
A C E
Flash
P C I
ASP/S
A S P / H-1
40x625 (160 pins)
E N E T
Ethernet PHY
A S P / H-2
40x625 (160 pins)
U A R T
Serial Port
5
CRM Blocks
Ctrl
S D R A M
R c k t I/O UP
SDRAM
10
D C R B U s
HW S w I t c h
G P I O
R c k t I/O DN
10
SelectMap
G P I O
P F I F O
A S P / S
16 x 800 (64 pins)
A C E
Flash
P C I
ASP/S
A S P / H-1
40x625 (160 pins)
E N E T
Ethernet PHY
A S P / H-2
40x625 (160 pins)
U A R T
Serial Port
6
Details

• Clock Rate 125 MHz (8 ns period)
• Data Rates on Interfaces
• Rocket IO UP 10, each at 2.5 Gb/s
• 25 Gb/s
• 32 bit wide Tx, 32 bit wide Rx,
  (25Gb/s)/32bits 78.125 MHz
• Rocket IO DN 10, each at 2.5 Gb/s
• 25 Gb/s
• CRM to ASP/H-1 40 signals at 625 Mb/s each
• 25 Gb/s
• CRM to ASP/H-2 40 signals at 625 Mb/s each
• 25 Gb/s
• CRM to ASP/S 16 signals at 800 Mb/s each
• 12.8 Gb/s
• SPI-4
• 64 bit interface to Xilinx FPGA side
• chip to chip interface 16 bits wide, 800 Mb/s
• PFIFO 100 Mb/s
• Total of all these rates makes HW Switch look
  daunting.
• Total 112.8 Gb/s

7
CRM Blocks One HW Switch
Ctrl
R c k t I/O UP
10
D C R B U s
25 Gb/s
HW S w I t c h
R c k t I/O DN
25 Gb/s
10
P F I F O
100 Mb/s
A S P / S
19.2 Gb/s
32x600 (128 pins)
A S P / H-1
40x625 (160 pins)
25 Gb/s
A S P / H-2
25 Gb/s
40x625 (160 pins)
8
CRM Blocks One HW Switch Per Interface
R c k t I/O UP
HW S w I t c h
D C R B U s
Ctrl
R c k t I/O DN
HW S w I t c h
HW S w I t c h
Ctrl
P F I F O
A S P / S
HW S w I t c h
Ctrl
A S P / H-1
HW S w I t c h
DCR Bus
Ctrl
A S P / H-2
HW S w I t c h
Ctrl
9
CRM Blocks One HW Switch Per Interface
R c k t I/O UP
Ctrl
HW S w I t c h
D C R B U s
R c k t I/O DN
HW S w I t c h
HW S w I t c h
P F I F O
A S P / S
HW S w I t c h
A S P / H-1
HW S w I t c h
A S P / H-2
HW S w I t c h
10
CRM Blocks One HW Switch Per Interface
HW S w I t c h
R c k t I/O UP
R_UP
R_DN
ASP/S
ASP/H-1
ASP/H-2
PFIFO
11
Block Ram usage estimates

• 444 Block Rams on a P100
• First estimate 216 Block Rams for output queue
  switch
• new estimate 250
• Other uses of Block Rams
• PPC related
• PFIFO 16 (up to 128)
• ENET fifo up to 16 (8 is probably more than
  enough)
• PPC boot needs only if we cant boot from flash
• PPC IP Cores Nothing else uses any
• SPI4.2 Core 14
• Route/Switch block
• Rocket IO interface resources are included in
  chip
• do not come out of the 444 block rams
• Our code that drives the Rocket IO interfaces
• ???

12
Slice usage estimates

• 44096 slices on a P100
• Output Queue switch module 15000
• Uses of Slices
• PPC related
• details on later slides
• IPIF 176-2487
• PCI 2803
• Everything else 5564
• Total 5740 - 10854
• SPI4.2 Core 4100
• Route/Switch block
• Output Queue block
• Rocket IO interface resources are included in
  chip
• do not come out of the 44096 slices
• Our code that drives the Rocket IO interfaces
• ???

13
Global Clock Buffer usage estimates

• Global clock Buffers 16
• Digital Clock Manager blocks (DCMs) 12
• Uses of clock resources
• PPC related
• PFIFO
• ENET fifo
• PPC boot needs
• PPC IP Cores
• SPI4.2 Core
• 7 Global Clock Buffers
• 3 DCMs
• Route/Switch block
• Output Queue block
• Rocket IO interface??
• Our code that drives the Rocket IO interfaces
• 4 Global Clock Buffers
• 2 DCMs

14
CoreConnect Bus Infrastructure Cores
Cores Address / Data Availability Slices Description
PLB Bus w/ Arbiter (V3.4) 32/64 EDK 401 2 masters, 4 slaves, 1 DCR interface
OPB Bus w/Arbiter (V2.0) 32/32 EDK 229 4 masters, 4 slaves
DCR Bus 10/32 EDK 4 16 slaves
OPB-PLB Bridge EDK 574 1 address range, 1 DCR interface
PLB-OPB Bridge EDK 613 1 address range, 8 PLB masters, 1 DCR interface
15
Memory Controller and GPIO Cores
Cores Availability Slices Description
PLB SDRAM Controller EDK 622 32-bit SDRAM Data Width, High Speed Pipe, Burst Cache Line Support
PLB BRAM Controller EDK 181 64-bit PLB Data Width, 32-bit PLB Address Width
PLB IPIF EDK 176-2487 DMA/SG, Wr/Rd FIFOs, Master Attachment, Interrupt, Address Decode, Status Regs
2OPB GPIO All_Inputs EDK 28 2 56
2OPB GPIO Programmable as outputs EDK 42 2 84
16
Peripheral Cores
Cores Availability Slices Description
OPB/DCR Interrupt Controller EDK 124/ 196 8/16 interrupt inputs, Include IPR, SIE, CIE, IVR
OPB 10/100 Ethernet MAC EDK Evaluation 2395 DMA, Interrupt control, 16 entry length, status FIFOs, 4K IPIF packet FIFOs
OPB UART Lite EDK 70
System ACE Interface Controller EDK 171 16-bit memory data interface, 32-bit OPB data/address
OPB-PCI Full Bridge EDK Evaluation 2803 DMA
17
Interface Blocks

• For Output queue switch design
• Input side
• Route/Switch VLI lookup
• Leaky Bucket
• Output Side
• queue per input interface

18
Action Items

• Switch design issues JST
• TCAM JohnL and DZ investigating
• Block Ram usage on PPC side
• PPC Boot FK
• IP Cores JingL
• Interface details (width and rates) JDD
• QDR Interface DZ