Optical Data Links in CMS ECAL

1
Optical Data Links in CMS ECAL

• CMS ECAL architecture and needs
• Optical Data Links in HEP Experiments
• ECAL Data Link system description
• Data Link components
• Data Link system
• Opto-Hybrid assembly, Quality Assurance
• Schedule

2
The CMS Detector

3
CMS ECAL front-end architecture

ECAL has 77,000 lead tungstate crystals arranged
in supercrystals of 25. Front-end electronics of
each supercrystal send the data off-detector via
the optical links. The architecture and the
needed data rate require 600 Mb/s with
modularity for Data 1 link / supercrystal for
Trigger 1 link / supercrystal (barrel)
5 links/supercrystal (endcap) Total data
trigger 9000 links
4
Optical Links and HEP

Generic architecture of an optical link
fiber
data in
data out
Tx
Rx
Ser
Deser
MUX
DEMUX
Optical Layer
e/o, transport, o/e
First Overlay
encoding/decoding, usually performed in same chip
as serialization/deserialization
Framing
Multiplexing multiple conversations in each
link. Not usually relevant for HEP since systems
are designed to run each link continuously at
maximum data rate.
5
Optical Layer considerations - 1

Tx In HEP, usually either Edge-Emitting Laser
Diode or VCSEL. Rx Typically photodiode or
photodiode array (typically of 12), followed by
digital amplifying ASIC.
Tx considerations Edge-Emitting linear
response (relevant if you design an analog
link) VCSEL said to be more rad-resistant
due to smaller active area, but practical
experience is that fractional change of
behavior per unit irradiation is typically
similar. VCSEL is cheaper to produce
and test on wafer. Was once thought that they
would entirely replace Edge-Emitting.
But benefit is diluted since packaging dominates
the cost of both. In any case device
chosen must be consistent with the system speed
and environmental specifications (often
unique to HEP more on this later).
Edge-Emittingdiagram
VCSEL diagram
test on wafer
Rx considerations Speed, wavelength,
saturation and sensitivity limits must be
consistent with system specifications.
6
Optical Layer considerations - 2

Optical fiber considerations Two
principle kinds Single Mode (SM) and Multi Mode
(MM). Differences in impurities in the
silica and differences in core and
cladding diameters give different spectral
transmission characteristics. SM
lower attenuation, is the favored choice for
telecom (longhaul) distances (10 km and
greater). Uses smaller core to allow
only one mode to propagate, making connector
alignment more critical (and expensive).
MM favored for datacom (LAN) distances (5
to 10 km). In principle cheaper than
SM, but fiber itself may be more
expensive since market is smaller (datacoms did
not take off). HEP distances are short
(100m), so SM/MM choice is not critical.
Usually free to select the cheapest solution
satisfying environmental
specifications. Qualification is essential
Early ECAL qualification of MM fiber of various
suppliers Good fiber loses lt50
transmission after 1013 p/cm2. Bad fiber
turns black.
7
First Overlay considerations

• Serializer/deserializer considerations
• Encoding Two principle protocols are CIMT
  (G-Link, Agilent proprietary) and 8-bit/10-bit.
  Both share the following
  characteristics
• introduce an overhead of two bits per 8 bits
  of data
• are usually employed continuously
  transmitting, either data or idle pattern
• aid in maintaining synchronization of
  transmitter and receiver
• allow some error checking
• stabilize 0 and 1 levels by keeping number of
  bits of each the same
• Principle protocol differences G-Link is
  considered more robust in terms of maintaining
  and recovering synchronization. But
  it is proprietary. At the moment the fastest
  CIMT deserializer runs up to 1.4 Gb/s. In
  general there is no guarantee that a
  company employing proprietary technology will
  continue developing new products with
  it, nor any guarantee that it will not
  discontinue existing ones.
• Speed and bits serialized Typical scheme
  at LHC experiments is one (two) 16-bit words
  serialized at LHC clock rate of 40 MHz
  implying optical link speed of 800 (1600) Mb/s of
  which 80 is data and 20 is overhead.

8
General considerations

HEP experiments typically have many
specifications which are unusual in industry.
The closer the transmitter is to the center of
the detector, the more likely this is to be true.
Every need has its consequence(s)
radiation resistance, low power (heat), temperature cycle, non-magnetic materials extensive qualification of several products (variation of behavior between different products can be large)
limited space custom laser package, with consequences for hermeticity and mounting techniques
long (3-10 yrs) development time possibility that a product will be discontinued
100 reliability of 10k links over 10 years sleepless nights
The typically O(1M) procurement for a HEP
experiment is relatively small for any company
large enough to collaborate in the task.
Development teams in laser transmitters are
smaller than one might think, typically 10-30
people for a large company. gt The experiment
will incur significant development and
qualification effort and costs.
In designing an optical link for a HEP
experiment Typically start with a
shopping list of needs. From that define
specifications and select/develop technology.
End up with a laundry list of issues.
9
Optical Data Link System Diagram

• GOL Opto-Hybrid (GOH) ECAL designs,
  prototypes, qualifies, defines manufacturing
  specifications, procures the manufacture,
  tests samples during production.
• 12-channel NGK Rx Off-the-shelf component
  ECAL qualifies, procures, tests samples during
  production.
• Fiber, connectors and adaptors ECAL uses
  solutions already developed and procured for CMS
  Tracker.

10
Components Transmitter - 1

The transmitter of the data link is the GOH.
Its principle components

• The Gigabit Optical Link (GOL) serializer and
  laser driver ASIC

• Designed by CERN Microelectronics group.
• Implemented in 0.25µm CMOS technology
  employing radiation-tolerant layout practices.
• Speeds of 0.8 and 1.6 Gb/s. ECAL will run
  the GOH at 0.8 Gb/s.
• Two protocols (G-Link and 8b/10b). ECAL will
  use 8b/10b for data and G-Link for trigger.

• The laser diode, made by ST Microelectronics,
  based on a Mitsubishi die

• Custom-made for CMS Tracker (linear response),
  suitable also for digital use in ECAL (rise
  time consistent with 800 Mb/s operation).
• Die wafer lots are radiation-qualified (gammas
  and neutrons) before assembly into laser diodes.
• Pigtail fiber is attached and calibration data
  are taken by ST.

• Some specifications of the GOH
• Receives LV, clock, control and 16-bit
  parallel digital input at 40 Mb/s from FE board
  via connector.
• Transmits serialized optical output at 1310
  nm, 800 Mb/s, either protocol, via pigtail fiber.
• Output signal power -6dBm, depending on bias
  levels chosen. (0 dBm 1 mW)

11
Components Transmitter - 2

Status of the GOH

• The design is complete, prototypes exist.

• Driven by GOH evaluation board (a modified
  GOL eval board), gives a clean eye diagram
  at 800 Mb/s.

• Functions well within the full data link (more
  on this later).

12
Components Fiber and Connectors

• Fiber and connectors are adopted from the CMS
  Tracker system
• All specifications consistent with use in ECAL
  as well (e.g. temperature, rad-hardness,
  attenuation, safety).
• ECAL benefits from commercial work,
  qualification and acceptance test setups
  already a part of Tracker program.
• ECALs contribution is the in-kind to staffing
  of acceptance tests.

MFS adapter (Diamond)
sMU-MFS fanout (Ericsson, Sumitomo, Diamond)
MU-sMU adapter (Sumitomo)
MFS-MPO multi-ribbon cable (Ericsson, Diamond)
13
Components Receiver

• 12-Channel digital NGK Rx
• COTS device ECAL tasks are to qualify and
  procure.
• Specifications are consistent for use within
  the full data link, e.g.
• wavelength 1310 nm
• speed up to 1.25 Gb/s
• saturation -5 dBm
• sensitivity -18 dBm
• Functions well within the link (see following
  slides).

14
Data Link System Tests - 1

• Early system tests involved
• GOH driven by GOH evaluation board
• Rx on Rx evaluation board
• Deser (G-Link or 8b/10b) on Deser eval board
• counted Deser word errors (frame errors)

Noise data generated on neighboring channels
Channel under study

• in this case, to study effect of crosstalk in
  the Rx
• Conclusions
• Link functions well, BER lt 10-13 should be
  obtainable
• Rx is well within sensitivity spec (-18 dBm)
• Crosstalk costs about 2 dB of the optical
  power budget

15
Data Link System Tests - 2

• First test of the Data Link with the prototype
  FPGA version of the Front-End board (mid-Feb 03)

• Four triggers generate four bursts of data.
• Each trigger involves the absence of one
  Control clock cycle.
• Valid data seen by deserializer, except
  occasionally soon after the missing clock
  cycle(s). It is expected that this will be
  resolved when QPLL (CERN MIC) is integrated into
  Front-end and Off-detector boards.
• The good news is that FE Data Link works!

16
Optical Power Budget

Launched from GOH -6 dBm
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------
Laser diode efficiency 1 dB
Fiber and connector interfaces 3 dB
Rx crosstalk 2 dB
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------
Effective signal power at Rx -12 dBm

Rx sensitivity limit -18 dBm
Rx sensitivity spec is -19 dBm typical, -18 dBm
minimum. We have so far observed better than
20 dBm.
gt At least 6 dB of margin in the power budget
17
GOH Assembly

• GOH assembly involves a number of steps

• Manufacturing Specifications document for the
  GOH is complete. We plan to submit the RFQ as
  soon as possible to have the maximum amount
  of time to test the first 100 (Engineering Run).

18
GOH QA and Link System Tests - 1

• Construction of first 100 GOH (Engineering Run)
  will begin when all of these are ready
• RFQ completed and manufacturer selected
• Laser diodes are available
• Tested GOLs are available

We plan thorough tests to verify functionality
and reliability.

• The tool for these tests will be our Bit Error
  Rate Test system. Based on PC, CMS GIII DAQ
  cards and extensive Altera programming.
• Principle is the same as for the frame error
  tests plot BER vs. optical power, make a
  quantitative measure of a stress on the system
  in terms of a penalty on the optical power
  budget.
• Advantages over frame errors are that it is
  bit-by-bit, independent of protocol.
• BERT system has been working since beginning of
  the year. Presently runs at 100 Mb/s BER of
  lt10-13 in 1 day.
• Present effort is to automate it to perform
  many measurements with minimal human
  intervention.

19
GOH QA and Link System Tests - 2

• The functionality tests we plan include

Temperature sensitivity GOH, Rx,
deserializer. Sensitivity to LV
fluctuations/noise. Fine-tuning of the eye
diagram via GOH compensation network
components. Integration of the Link into the
full ECAL electronics chain.

• The reliability tests planned for the GOH
  include

Vibration Mechanical shock Thermal cycling
(-20C to 40C) Pass/fail eye diagram
test Irradiation Accelerated aging Long-term
stability
Manufacturer
Data Links group

• The goal of the tests is to uncover and resolve
  any design or manufacturing reliability issues
  that may exist, and to be able to sign off on
  the full Link system design by the October ESR.

20
Schedule
Qualification is crucialbut is the only
compressible task.
Schedule concerns May 03 If laser diodes or
tested GOL are late, time for testing of Eng Run
is reduced. Sep
03 If GOH design modification is necessary,
little time to test it before the Pilot Run.
21
Summary

• Development of an optical link for a HEP
  experiment is a fascinating project, invariably
  involves a lot of co-development with industry.
• ECAL Data Link system design and project have
  made a lot of progress in a short time. Starting
  with good components helps.