Title: Clock Distribution for IceCube
1Clock Distribution for IceCube
http//www.phys-astro.sonoma.edu/people/students/b
aker/SouthPoleFoucault.html
- Gerald. Przybylski
- Lawrence Berkeley National Laboratory,
- Design Review, September 16, 2005
2History
- string 18 implementation o Rubidium module
slaved to GPS Clocko fan-out port for each
domcom card. - string 21 implementation OCXO GPS clock,
Symmetricom ET-6000 Passive fan-out to each DOM
Hub (DSB Card) Sub-nanosecond skew and jitter
demonstrated Simple and Reliable, but not
scalable
3MCU Requirements
- Straightforward conceptually very simple
- 5ns absolute accuracy (skew and jitter), within
the IceCube counting house Across all DOM
Hubs (at DOR cards) Fixed and stable offset
from Universal Time, Coordinated (UTC) Based
on Scattering Length in the ice - Distribute 10 MHz, 1Hz and Time Value String
- Free from Metastable states/events no glitches
- Measurable and Verifiable
- Single driver per output-port no shared drivers
- Robustness requirements in ERD x Mainly
dealing with satellite drop-out and loss of
availability x Also dealing with tracking
multiple GPS clocks - Phase accuracy 0.4ns at fan-out, 0.7ns at
DSB, and 1.0 ns at DOR
4IceCube IceTop AMANDA
NOT COVERED IN DETAIL IN THIS TALK
- IceTop same as IceCube
- IceCube to AMANDA Zeuthen/Wuppertal
Install 05/06 Holger et. al. Fiberoptic
transmitter driven by GPS clockFiberoptic
receiver drives TWR (GPS4TWR)Autonomous from
the MCU Clock Fan-Out subsystem 10 MHz BNC,
1Hz BNC, IRIG-B BNC10 ns precision with respect
to IceCube time - AMANDA to IceCube Same playersTrigger system
signals over Fiberoptic link to IceCube counting
houseDepends on DOM Main Board(s) on a String
81
Justification We realized for Amanda that
any artifial jitter below 15 ns in the MC has no
influence on reco accuracy or background
rejection.We then said 10 ns just for safety.
Then others came and said that for nearly
vertical tracks close to a string scattering may
be negligible and a 5 nsec request makes sense.
-- Christian
5Our GPS clock
- ET-6000
- Time to first fix
- Outputs operational
- Timing accuracy better than 2 µS frequency
accuracy better than 1E-8 - Full system accuracy (100nS) within one hour.
- 10 MHz output, 1Hz output, Time burst output
- GPS from US Naval Observatory clock 2x10-15
Accuracy
6MCU Brick-Walls
- ET-6000 Specifications
- 1Hz output is positive (rising) edge on
time, within 100 nanoseconds relative to
either UTC or GPS with six or more satellite
averaging with 95 confidence. ( 150ns peak) - 40ns RMS accuracy (jitter not specified)
- Cannot Vote multiple clocks Neighboring clocks
dont track!x Tracking Algorithms in GPS clock
PLLsx Variations in path, and multipathx
Constellations and satellite switch-over
7Unlock Behavior
- Power Up Sync to Satellites within an hour
Elapsed Time Format until Tracking- Hic-ups
while searching every 2 hours - Potential Loss of lockx Clock Firmware (a-la
TrueTime 2000 problem)x Power Outagex
Misadventure (Murphy)x Wind/Weather damage to
Antenna (speculative)x National Security outage
(speculative)
8Flywheeling/Freewheeling
- TCXO Clock continues for hours
- Optional OCXO continues 1 week Aging
5 x 10-10 per day, 5 x 10-8 per year
Phase Noise -115 dbc/Hz _at_ 10Hz
-94 dBc/Hz _at_1Hz - Optional Rubidium Oscillator clock 100ns per
day slew WRT Aging over 20 years, Phase Noise -90
dBc/Hz _at_10Hz -80 dBc/Hz _at_ 1 Hz Ru
Good short term stability, best hang timeOCXO
best short term stability, good hang time - Based on Symmetricom ET6000 series product
specifications/experiences11
92004-2005 Implementation
- One GPS clock, an ET6000-OCXO
- Simple Passive fan-out (resistive splitters)
- All 9 DOM Hubs driven All clock BNCs used!
- 0.35ns Jitter and Skew measured in situ at NPX
- DOR Firmware Improvements fixed NPX GPS
glitches - Not scalable to 90 Hubs
102005-2006 Implementation
- 2U chassis with 24 port fan-out (2 cards)o
Modulates 1Hz signalo RJ-45 distribution cables
carry 10 MHz , modulated 1Hz, TVSo All
balanced signaling - Passed MOAT sps-ichub04, sps-ichub05,
domhubjr, domhub51 - Good Noise Immunity
- Stepping stone scalable to needs of IceCube
11Design Goals/Drivers
- 90 ports plus Spares
- Meet accuracy, jitter and skew requirements
- Convenience Single distribution cable per DOM
Hub - Measurability/Verifiability Easy to confirm
phase across all ports - Reliability/Robustness Quality components.
No electrolytics. - Noise Immunity Balanced signals to DSB cards
- Minimize hard connections between racks
Magnetic coupling - Modularity/Extensibility/Maintainability
- Hot-Swappable Port Cards
- Independent Port Drivers for each signal
- Low power
- Off the Shelf Components
- No heroic solutions
- Simplicity! (no programmable logic in clock
distribution)
12Signaling Details
- Balanced 10 MHz 500mV P-P through ethernet
magnetics- High common-mode immunity- Suppress
EMI emission RFI pick-up- Avoid Ground
Bounce pick-up in the counting house- Commodity
components compact, inexpensive - Modulate the 1Hz signal (180deg Phase
Modulation) required to pass through ethernet
magnetics - RS-422/RS-485 differential serial12/-7V CM range
10 MHz
1Hz/1PPS
Registered 1Hz
U47-4
Modulated
1Hz
13Rev 0 Fan-Out Card
- 12 Port
- 0.6ns port to port skew, worst pair
- Symmetry matters!
- Passes MOAT
- Revised DOR Firmware now supports Modulated 1Hz
- Inputs directly from GPS clock in Stand-Alone
configuration - Passes Fluorescent Lamp noise immunity test
- Status LEDs Test Header
142006 Implementation and Beyond
- VME form factor conditioner card (1) in
2006 - VME backplane fanout mezzanine card (1) in
2006 - VME form factor 12 channel fan-out cards (9)
proto now - VME form factor Monitor card (1) 2006 goal
(Least well defined)
DOM Hub
Coax Cable (2)
GPS Clock
DSB
RJ-45 Cable (90)
Serial Cable
15Port Card Features, Rev 1
- 2-wideVME form factor 12 RJ-45 port
- Common 10 MHz from Backplane
- Common modulated 1Hz from Backplane
- Common Serial from Backplane
- LVDS inputs, instead of direct GPS clock signals
- One Point Signal conditioning
- 2 minor schematic corrections
- Skew tweaks
- Additional Monitoring points P2 and Header
(e.g. demodulated 1Hz)
16Backplane Fan-Out
- Active Piggy-back card mated to Back-of-Crate
style VME backplane over Socket 2 - Independently drive each EVEN socket
- Match phase at each driven socket
- 1-to-10 LVDS-to-LVDS fan-out chips
- Match skew to each driven socket
Piggy-back Card
Industry Standard Back of Crate style Backplane
17Conditioner card
- Driven by GPS clock 10 MHz, 1Hz, and Serial
- Modulate 1Hz signal
- Control skew by design
- Ensure symmetry of port signals
- Drive LVDS backplane fan-out
- Status indicators/LEDs
- Prototyped on the Fan-Out card
18Monitor Card
- Occupies an Odd Slot in VME card cage (e.g. 1)
- Implement GPS clock Monitoring specified by PDR
Document - Could contain FPGA, SOPC, or SBC in a DIMM form
factor - Could report via ethernet
- Scope TBD
19Watch List/Wish list
- Lock status from GPS clocks
- Tracking status from GPS clock
- Power status from GPS clock
- Parse time strings for error conditions
- Parse GPS Clock console port output Rich set
of Status bits/words Satellite constellation - Monitor phase offset between multiple GPS clocks
20Verification
- Reference signals on header10 pin header on
DSB10 pin header on Fan-Out card4 pin header on
DOR card (inside DOM Hub)
21The Bottom Line
- Fan-Out card comfortably meets the 5ns
Requirement - Our GPS clock is a good choice, for the money
- No speed bumps this year
- On track for the final assaultSingle unit,
flexible, modular designBuilds on previous
successesAvoids heroics and death marches.
Fin
22Verifications in situ
- IceCube AMANDA
- Opt 1 Measure Fiber Round-Trip time (2 fibers)
- Opt 2 Use Portable Atomic Clock - packaged
PRS-10- battery power - Steps 1. Measure/calibrate WRT IceCube
clock 2. Transport clock to MAPO 3. Measure
fiber distribution signals in MAPO against Atomic
Clock 4. Transport clock back to IceCube
Clock 5. Check CalibrationRepeat 1 to 5
until satisfied - Measure/Verify within 1ns should be achievable.
4