ATCA For Real Time Systems

1
ATCA For Real Time Systems

• Advanced Telecom Computing Architecture (ATCA)
• Its Performance and Application for Real Time
  Systems
• Alexandra Oltean 1, 2 Brian Martin 2
• 1 Politehnica University of Bucharest
• 2 European Organisation for Nuclear Reseach,
  Geneva Switzerland

2
The PICMG Story

• PICMG has been pushing its CompactPCI bus
• Main market is Telecomms
• CompactPCI only has small market share lt1
• For Telecom apps Compact PCI suffers
• Boards too small and underpowered
• Too close together, insufficient cooling
• Too slow
• Time for a rethink!!
• Board size and power simple mechanical issues
• But what about the architecture and the speed??

3
The need for speedBus Based vs Point to Point

• The end of the bus has long been forecast.
• Shortcomings well known
• Shared resource Doesnt scale
• Proprietary interface chips vendor specific
• BUT High Speed Serial Links were optical
• And expensive
• Big improvement in high speed over Copper
• gt3Gb/s today, 10Gb/s in demonstration

4
Pre-Emphasis
0 1 1 0 0
Transmitted Bit Stream
RX after HF loss
Compensate for loss With Pre-emphasis At
Transmitter
RX after HF loss
5
Which way to go?

• Several serial technologies to choose from
• PCIExpress, 1 or 10Gb Ethernet, Infiniband..
• more to come in the future
• Common denominator
• 100 Ohm differential transmission lines
• PICMG Strategy
• Build in enough of them to please everyone
• Be Careful its still not invented here

6
Mechanics Form Factor
7.25U
230mm
To Backplane Connectivity
Power
Transitional Module
7
Mechanics Backplane Connectors
8
Compare ATCA and Bus systems
9
ATCA Chassis, Fans, 48V supply
10
Backplane Interconnects1

• I2C shelf management
• (25 of the spec)
• Base Interface
• Dual star 1Gb/s
• Synch clocks
• two for SDH/Sonet (8Khz, 19.44Mhz)
• 1 user defined
• Update Bus
• 10 pairs between neighbours
• User Defined

11
Data Fabric Topologies
FULL MESH
DUAL STAR
N
H
H
N
N
N
N
N
N
N
N
One Channel 8 differential pairs
12
Whats wrong with it?

• Market risk
• Global signals theres only 1 User clock
• What to do with GPS, Resets, Synch signals.
• Use Update bus
• Or the rear Transitional Module
• Some users want more power
• 200 w / slot 12.8Kw for rack of 4 chassis
• Possibly manageable
• 250 w / slot 16.8Kw for rack of 4 chassis
• Possibly unmanageable
• Front panel
• 40 RJ45s ?

13
Bus Based Read Out Element

• Multiple input streams to Buffer Memories
• Interface glue to PCI bus
• Data and Control flow
• CPU master does all the work
• Samples dispatched for processing
• Accepted data sent to next stage
• Rejected data cleared
• Intensive in Messages, DMAs, I/Os
• I/O to next Level 1 or 2Gb/s
• Housed in PC boxes
• Balanced but no headroom and no way to expand

14
Switch Based Readout Element

• Multiple input data streams
• Intelligent Buffer Memories
• Interface glue to 10Gb/s Links
• Links switched on-board
• 2 10Gb/s to Hub boards
• Hub(s) multiplex output data
• 10Gb/s link(s) to next level
• Control Flow 1Gb/s Base Fabric
• Node processing N switched CPUs

Node Board
Node Board
Pn
Pn
Switch
Switch
10Gb/s Data Fabric
1 Gb/s Base Fabric
10 G Switch
1G Switch
Hub Board
Ph
15
Early Design Experience

• 10Gb/s Switch project (ESTA)
• Design our own backplane
• Saw limits of simulation
• Cant simulate a complete backplane
• ATCA comprehensive simulation backup
• Prototype Backplane Ours or industrial
• What If Proto chips marginal board??
• Concern of low level errors the time to fix
  them

16
Typical Measurement Rig
Only Possible to simulate 1 or 2 traces Only
possible to measure a few channels No fully
loaded tests ever done.
Tester Requirement Run all channels with same
TX / RX and Speed as final 10 Gb/s switch
17
Backplane Tester

• Existing testers are DC or low frequency at best
• Use working silicon for 3.125Gb/s per pair
• Marvell Alaska Serdes
• Controllable amplitude
• Pre-emphasis
• Jitter Patterns
• PRBS sequences
• Data error checking
• Node boards drive one channel to each Hub
• Microcontroller
• Simulate and Measure Results

Mem
Ethernet
u Controller
To Control PC
SerDes
Parallel I/O Reset Clk RD MDO MDI
SerDes
SerDes
Hub / Node Jumpers
Fan Out TX RX
SerDes
Backplane Connectors
18
Backplane tester Hub Board

• Same board design for all 14 slots
• Partial mounting for Node boards
• Realistic on-board routing
• Testing for Dual Star Fabric

19
Fully Installed Backplane Tester
12 Node Boards In Slots 3 to 14
Master Hub Board In Slot 1
Redundant Hub Board In Slot 2
20
Simulated and Measured Results
TX Signal 900 mV Pre-Emphasis 0

33 Measured Eye RX 575 mV
740 mV
21
Where are the limits

• Difficult to create error conditions
• Alaska SerDes
• Fix the Maximum Amplitude
• Vary the pre-emphasis
• Start with 1100mV
• Assert 300 pre-emphasis
• Reduce Eye to 220mV
• Then we start to see errors

22
Conclusions

• ATCA offers rich connectivity
• With standard FR4 at 3.125Gb/s
• Clear error free operation under full load
• Extreme degradation before errors visible
• Excellent potential for Real Time Systems