Planet Lab Memory Map

1
Planet Lab Memory Map
David M. ZarApplied Research LaboratoryComputer
Science and Engineering Department
2
Revision History

• 8/xx/06 (DZ)
• Created
• 8/29/06 (JDD)
• Updated with more details on counters

3
Planet Lab Memory Map

• The following SRAM is used in every design
• Buffer Descriptors in Channel 1 (8 MB)
  0x40000000
• RX PACKET COUNTERS (debug) in Channel 3 (640
  bytes)
• TX PACKET COUNTERS (debug) in Channel 3 (640
  bytes)
• Scratch Memory usage
• TO_XSCALE_RING 0x2800 0x2bff
• FROM_XSCALE_RING 0x2c00 - 0x2cff
• PS_TO_QM_RING_1 0x3000 0x33ff
• PS_TO_QM_RING_2 0x3400 - 0x37ff
• QM_TO_TX_RING_1 0x3800 0x3bff
• QM_TO_TX_RING_2 0x3c00 0x3fff
• TX1 - PACKET_TX_AVAIL_TBUF_ELE_SCR_ADDR
  SCRATCH_BASE 0x1f00

4
Substrate Decap (IPv4) Memory Map

• Initialization data
• Enet MAC Addr MR blade (48b)
• Dynamic Data
• VLAN-to-CodeOption index table
• 212 4096 VLANs
• 4b Code Option field
• 1LW for minimal cycles (16KB reqd SRAM)
• or, 4b for minimum memory (2KB reqd SRAM,
  Scratch, LM)
• Counters
• 2 LWs total for packets received sent
• 4 LWs total for Ethernet VLAN validation
• 6 LWs per VLAN for UDP/IP validation
• using power-of-2 sizes saves cycles multiply
  becomes shift
• now 32B per VLAN x 4096 VLANs 128KB SRAM reqd

5
Parse (IPv4) Memory Map

• Initialization data
• none only compile-time constants and ring data
  used
• Dynamic Data counters
• 2 LWs total for packets received and sent
• 9 LWs per VLAN for UDP/IP validation
• using power-of-2 sizes saves cycles multiply
  becomes shift
• now 64B per VLAN x 4096 VLANs 256KB SRAM reqd

6
Lookup (IPv4) Memory Map

• Initialization data
• Dynamic Data

7
HF(IPv4) Memory Map

• Initialization data (88b GP88b)
• Enet Addr MR blade (48b)
• IP Addr MR blade (32b)
• GPs (8b)
• TableGP
• Enet Addr (8b)
• Upper 40 bits are same as Enet Addr MR Blade
  above
• IP Addr (8b)
• Upper 24 bits are same as IP Addr MR Blade above
• Local Delivery
• UDP dest (16b)
• QID (20b)
• Exception
• UDP dest (16b)
• QID (20b)
• Dynamic Data (None)

8
QM (IPv4/LCI/LCE) Memory Map

• Constants
• Q 128K (total for all QM)
• QM_NUM_QUEUES
• QD_BASE_ADDR 0x000000
• QD_SIZE 0x200000
• QPARAMS_BASE_ADDR 0x200000
• QPARAMS_SIZE 0x200000
• QSCHED_BASE_ADDR 0x400000
• QSCHED_SIZE 0x140000
• PORT_RATES_BASE_ADDR 0x540000
• PORT_RATES_SIZE 0x28
• Initialization data (none)
• Dynamic Data (SRAM)
• Port Rates (5 32b)
• Per Queue
• Queue Parameters (per queue)
• Discard Threshold (32b)
• Quantum (32b)
• Qlen (32b)

9
KeyExtractor (LC) Memory Map

• Ingress
• Initialization data (none)
• Dynamic Data (none)
• Egress
• Initialization data (none)
• Dynamic Data (none)

10
Lookup (LC) Memory Map

• Ingress
• Initialization
• Dynamic Data (none)
• Egress
• Initialization (none)
• Dynamic Data (none)

11
HF (LC) Memory Map

• Ingress
• Initialization
• Enet Addr (48b)
• Dynamic Data (none)
• Egress
• Initialization (none)
• Dynamic Data (none)

12
Port Splitter (LC) Memory Map

• Ingress
• Initialization (none)
• Dynamic Data (none)
• Egress
• Initialization (none)
• Dynamic Data (none)

13
Counters

• Two sets of counters maintained for each
  successful lookup
• Pre-Queue Pkt counter and Byte counter
• Post-Queue Pkt counter and Byte counter
• Each counter is 32-bits
• Organization of Counters
• Addr of a pre-q pckt ctr BASE(4 4 Index)
  (04)
• Addr of a pre-q byte ctr BASE(4 4 Index)
  (14)
• Addr of a post-q pckt ctr BASE(4 4 Index)
  (24)
• Addr of a post-q byte ctr BASE(4 4 Index)
  (34)

IPV4_LU_COUNTERS_SRAM_INIT_BASE
Counter Index 0
Counter Index 1
. . .
Counter Index N
Pre-Queue Pkt Ctr
Pre-Queue Byte Ctr
Post-Queue Pkt Ctr
Post-Queue Byte Ctr
14
LC Ingress SRAM Channel 3
0x000000
0x000000
PreQ/PostQ Cntrs
0x00FFFF
0x010000
MI Counters
0x08FFFF
0x100000
Unallocated
0x090000
HF Init
0x090007
0x090008
Counter Error Cnts
0x090027
Unallocated
0x200000
0x090028
Unallocated
0x0FFFFF
Unallocated
0x300000
Unallocated
0x400000
0x700000
Unallocated
Unallocated
0x500000
0x7FFDFF
Rx Pkt Counters (turned off for Perf.)
0x7FFE00
Unallocated
0x600000
0x7FFE9F
0x7FFEA0
Tx Pkt Counters (turned off for Perf.)
0x700000
0x7FFF3F
0x7FFF40
Unallocated
0x7FFFFF
15
LC Ingress SRAM Channel 2
0x000000
Buffer Descriptors (0x38000 32B) (229376
32B) (7MB)
0x100000
0x200000
0x300000
0x400000
0x500000
0x600000
Queue Desc Array (65536 16B 1MB)
0x700000
16
LC Ingress SRAM Channel 1
0x000000
Q Params (65536 16B)
0x100000
QM1 Sched (13109 44B 0x8CD1C)
QM2 Sched (13109 44B 0x8CD1C)
0x18CD1C

• QM1 Freelist
• (13109 4B 0xCCD4)

0x219A38

• QM2 Freelist
• (13109 4B 0xCCD4)

0x22670C
QM1 QM2 Port Rates (10 4B)
0x2333E0
Unallocated
0x233418
0x600000
Unallocated
0x7FFFFF
17
LC Egress SRAM Channel 3
0x000000
0x000000
PreQ/PostQ Cntrs
0x00FFFF
0x010000
MI Counters
0x08FFFF
0x100000
Unallocated
0x090000
HF Init
0x09003F
0x090040
Counter Error Cnts
0x09006F
Unallocated
0x200000
0x090070
Unallocated
0x0FFFFF
Unallocated
0x300000
Unallocated
0x400000
0x700000
Unallocated
Unallocated
0x500000
0x7FFDFF
Rx Pkt Counters (turned off for Perf.)
0x7FFE00
Unallocated
0x600000
0x7FFE9F
0x7FFEA0
Tx Pkt Counters (turned off for Perf.)
0x700000
0x7FFF3F
0x7FFF40
Unallocated
0x7FFFFF
18
LC Egress SRAM Channel 2
0x000000
Buffer Descriptors (0x38000 32B) (229376
32B) (7MB)
0x100000
0x200000
0x300000
0x400000
0x500000
0x600000
Queue Desc Array (65536 16B 1MB)
0x700000
19
LC Egress SRAM Channel 1
0x000000
Q Params (65536 16B)
0x100000
QM1 Sched (13109 44B 0x8CD1C)
QM2 Sched (13109 44B 0x8CD1C)
0x18CD1C

• QM1 Freelist
• (13109 4B 0xCCD4)

0x219A38

• QM2 Freelist
• (13109 4B 0xCCD4)

0x22670C
QM1 QM2 Port Rates (10 4B)
0x2333E0
Unallocated
0x233418
0x600000
Unallocated
0x7FFFFF
20
NPE/MR SRAM Channel 3
0x000000
0x000000
SD Init
0x000007
0x000008
HF Init
0x000047
0x100000
Unallocated
0x000048
Unallocated
0x000FFF
0x001000
Rx Pkt Counters (turned off for Perf.)
Unallocated
0x200000
0x00109F
0x0010A0
Tx Pkt Counters (turned off for Perf.)
Unallocated
0x300000
0x00113f
0x001140
SD Counters
0x001157
Unallocated
0x400000
0x001158
SD VLAN Code Opt Tbl
0x009157
0x009158
PreQ and PostQ Ctrs
Unallocated
0x500000
0x019157
0x019158
Error Ctrs
0x019177
Unallocated
0x019178
0x600000
Slice Specific Mem
0x099177
0x099178
IPv4/I3 specific data
Unallocated
0x700000
0x099A30
Unallocated
0x0FFFFF
21
NPE/MR SRAM Channel 2
0x000000
Buffer Descriptors (0x38000 32B) (229376
32B) (7MB)
0x100000
0x200000
0x300000
0x400000
0x500000
0x600000
Queue Desc Array (65536 16B 1MB)
0x700000
22
NPE/MR SRAM Channel 1
0x000000
Q Params (65536 16B)
0x100000
QM1 Sched (13109 44B 0x8CD1C)
QM2 Sched (13109 44B 0x8CD1C)
0x18CD1C

• QM1 Freelist
• (13109 4B 0xCCD4)

0x219A38

• QM2 Freelist
• (13109 4B 0xCCD4)

0x22670C
QM1 QM2 Port Rates (10 4B)
0x2333E0
Unallocated
0x233418
0x600000
Unallocated
0x7FFFFF
23
Scratch Memory (16 KB)
0x0000

PACKET_TX_AVAIL_TBUF_ELE_SCR_ADDR

TO_SCALE_RING
FROM_XSCALE_RING
COUNTER_RING
PS_TO_QM_RING_1
PS_TO_QM_RING_2
QM_TO_TX_RING_1
QM_TO_TX_RING_2
0x1F00
0x1F04

0x2400
0x2800
0x2C00
0x3000
0x3400
0x3800
0x3c00
0x3FFC
24
dl_system

• define BUF_SRAM_BASE SRAM_CHANNEL_1_BASE_ADDRES
  S
• define NUM_IN_PORTS 10 number of input ports
• define NUM_OUT_PORTS 10 number of output
  ports
• define_eval PACKET_COUNTERS_SRAM_BASE
  SRAM_CHANNEL_3_BASE_ADDRESS
• define RX_PACKET_COUNTERS_SRAM_SIZE 16
  NUM_IN_PORTS
• define_eval PACKET_TX_COUNTER_BASE
  PACKET_COUNTERS_SRAM_BASE
• RX_PACKET_COUNTERS_SRAM_SIZ
  E
• define PACKET_TX_COUNTER_SIZE 16 NUM_OUT_PORTS
  
• define PS_TO_QM_RING_1 7
• define PS_TO_QM_RING_1_SIZE 256
• define PS_TO_QM_RING_1_BASE 0x3000
• define PS_TO_QM_RING_2 8
• define PS_TO_QM_RING_2_SIZE 256
• define PS_TO_QM_RING_2_BASE
  PS_TO_QM_RING_1_BASE 4 PS_TO_QM_RING_1_SIZE
• define QM_TO_TX_RING_1 9

Um, shouldnt this be 4 for the four bytes per
queue entry? And Should the rings be 256 or 512?
25
Extra Slides

• The rest of the slides are old or here for extra
  info.

26
SRAM Channel 1 (8 MB) 0x40000000
0x000000
Queue Parameters - Each is 4, 32-bit words (16 bytes) - So room for 64K
Queue Scheduling Structure (2) - Each is 11, 32-bit words (44 bytes) - Allocated 13109
Queue Free List (2) - Each is 1, 32-bit words (4 bytes) - Allocated 13109 (same as of QSS)
Port Rates - Each is 1, 32-bit words (4 bytes) - Ten ports

0x100000
0x219A38
0x2333E0
0x233408
. . .
0x7FFFFC
27
SRAM Channel 2 (8 MB) 0x80000000
0x000000
BUF_SRAM_BASE - Buffer Descriptors - Each is 8 32-bit words (32 bytes) - So room for 224K
Queue Descriptors - Each is 4 32-bit words (16 bytes) - So room for 64K
0x6FFFFF
0x7FFFFC
28
IPv4 SRAM Channel 3 (8 MB) 0xC0000000
0x000000
Packet/Byte Counters (4 words 16K) - PRE_Q_PKT_CNT - PRE_Q_BYTE_CNT - POST_Q_PKT_CNT - POST_Q_BYTE_CNT
MI Counters (512K) - PRE_Q_PKT_CNT - PRE_Q_BYTE_CNT - POST_Q_PKT_CNT - POST_Q_BYTE_CNT
Initialization Data

0x010000
0x090000
0x190000
0x7FFFFC
29
IPv4 SRAM Channel 3 (8 MB) 0xC0000000
0x000000
Per Block Initialization Memory RX Base
0x000000, Size 0 SD Base 0x000000, Size
8 PR Base 0x000008, Size 0 LK Base
0x000008, Size 0 HF Base 0x000008, Size
8 PS Base 0x000010, Size 0 QM Base
0x000010, Size 0 TX Base 0x000010, Size
0
0x000010
0x001000
30
IPv4 SRAM Channel 3 (8 MB) 0xC0000000
0x001000
Per Block Dynamic Memory RX Base 0x001000,
Size 0x280 TX Base 0x001280, Size
0x280 SD Base 0x001500, Size 0x24018 PR
Base 0x025018, Size 0x40008 IPv4
POST_Lookup Counters (PreQ, PostQ) Base
0x025018, Size 0x0010000
0x074520
0x100000
31
IPv4 SRAM Channel 3 (8 MB) 0xC0000000
0x100000

• BUF_SRAM_BASE
• - Buffer Descriptors
• - Each is 8 32-bit words (32 bytes)
• - (0x800000 0x100000)/32B 229376
• - So room for 229376 Buffer Descriptors

0x7FFFFC
32
LC SRAM Channel 3 (8 MB) 0xC0000000
0x000000
PACKET_COUNTERS_SRAM_BASE - 16 words per port, 16 ports (1024 bytes, total) - Used in RX for debugging
PACKET_TX_COUNTER_BASE - 16 words per port, 16 ports (1024 bytes, total)
Per Lookup Result Counters (pre-Q and post-Q) - 16 MR, 256 indices, 4 32-bit counters per index 64KB (See next slide for ctr details)
Per MI Counters - 64K 2 4 512KB

0x000400
0x000800
0x010800
0x090800
0x100000
0x7FFFFC