Title: Trigger
1Trigger Data Acquisitionfor LHC
UpgradesInternational Workshop on Future Hadron
CollidersFermilabOctober 16-18, 2003
- Andrew J. Lankford
- University of California, Irvine
2Acknowledgements
- My presentation draws heavily upon
- Work performed by contributors to the preliminary
studies reported in - Physics Potential and Experimental Challenges of
the LHC Luminosity Upgrade (hep-ph/020487). - A recent presentation by Nick Ellis at Erice.
3SLHC Implications of Higher Luminosity
- Higher Luminosity gt
- Increased detector occupancy
- Increased trigger rates
- Increased radiation effects
- Assumed SLHC luminosity 1035 cm-2s-1
4SLHC Implications of Higher Luminosity - 2
- Higher Luminosity gt Increased detector occupancy
- Assuming 12.5 ns bunch spacing,
- 125 interactions/crossing
- Occupancy of tracks increases 5-fold (10-fold for
some sub-detectors) - Radiation-induced hits increase 10-fold.
- Pile-up noise increases 2.2-fold (3-fold for some
sub-detectors) - Pile-up degrades performance of trigger
algorithms - Reduction in efficiency of e/gamma isolation cuts
- Increased muon candidate rates due to
radiation-induced accidentals - Larger event size to read out
- Increased by factor between 5 and 10
- Demands reduced trigger rate or increased data
bandwidth
5SLHC Implications of Higher Luminosity - 3
- Higher Luminosity gt Increased trigger rates
- Arising from
- Increased interaction rates
- Occupancy/pile-up induced trigger degradation
- Less rejection at fixed efficiency
- Need more selective triggers for same trigger
rate - Increased thresholds
- More exclusive selection (less inclusive trigger)
- Fortunately, more selective triggers are okay
- (see later)
- Need even more data acquisition bandwidth if
higher trigger rate.
6SLHC Implications of Higher Luminosity - 4
- Higher Luminosity gt Increased radiation effects
- Directly affects on-detector trigger logic
- Mechanisms
- Permanent damage
- Single-event-upset effects
- (Note that radiation effects upon detectors and
their electronics must be handled by trigger and
by data acquisition.)
7Enabling Technologies
- Enabling technologies for readout and trigger
- Integrated circuits
- Custom ICs, FPGAs, memories, etc.
- Commodity computing
- processors, networking, memory, storage,
fiberoptics - These technologies enabled current generation of
expts. - Including the nearly-deadtimeless, integrated
trigger and data acquisition systems that are now
standard. - These technologies will also be the technologies
that enable successful upgrades future
experiments. - RD programs to develop these technologies for
HEP applications were crucial to current
generation of expts. - A vital RD program will be crucial to SLHC
upgrades.
8Data Transfer Data Processing
- Trigger Data Acquisition systems provide
- Data transfer
- Data processing
- Limitations or costs of these functions define
limitations and costs of Trigger/DAQ systems. - e.g. First Level Triggers (FLT)
- Data input to FLT limits density of FLT
electronics. - Causes extensive interconnections betw. modules
and complex backplanes. - Needed to connect steps in FLT selection.
- Needed to seamlessly find tracks and clusters in
different sections of detector (to avoid loss of
efficiency
9SLHC Trigger Menu
- Need for 3 types of triggers foreseen
- Discovery physics
- Very high-pT (thresholds as high as hundreds of
GeV) - Completion of LHC physics program
- e.g. precise measurements of Higgs sector
- Lepton/photon/jet thresholds as low as for LHC
- Final states known gt exclusive selection
possible - Control / Calibration triggers
- e.g. Ws, Zs, top
- Low thresholds needed, but can be pre-scaled
- None of the above pose rate problems.
10Inclusive Triggers samples rates
LHC LHC SLHC SLHC
Selection Threshold Rate Threshold Rate
(GeV) (kHz) (GeV) (kHz)
inclusive single muon 20 4 30 25
inclusive, isolated e/gamma 30 22 55 20
muon pair 6 1 20 few
isolated e/gamma pair 20 5 30 5
inclusive jet 290 0.2 35 1
jet missing ET 100100 0.5 15080 1-2
inclusive ET 150 lt1
multi-jet triggers various 0.4 various low
Note that inclusive e/? trigger dominates rate.
(Added degradation from pile-up not included
above)
11FLT Importance of FLT upgrades
- Upgrades to First Level Trigger (FLT) are of
central importance. - They can reduce new demands on
- Front-end electronics (FEE)
- Retaining 100 kHz FLT rate avoids changes to FEE
systems that do not require upgrade for other
reasons. - Data Acquisition (DAQ)
- By reducing new demands for data bandwidth.
- High Level Triggers (HLT)
- By reducing new demands for algorithms and
processing - FLT upgrades deserve early consideration.
12FLT New Demands on Processing
- New demands on FLT processing posed by
- Increased event pile-up occupancy
- New sub-detectors or readout
- 12.5 ns crossing period
- Some (at least) new processing will be needed.
13FLT Impact of 12.5 ns crossings
- LHC FLTs
- pipelined processors
- driven at 40 MHz LHC crossing rate
- identify crossing of interest
- Should SLHC FLTs run at 80 MHz Xing rate?
- Can FLTs remain at 40 MHz ?
- Can FLTs work at both 40 80 MHz ?
14FLT Can SHLC FLTs operate at 40 MHz ?
- Concept of readout time frames
- FLT identifies pair of crossings (or more) for
read out - e.g. PEP II / BABAR
- PEP II runs at 250 MHz (4 ns).
- BABAR system clock 60 MHz (16 ns).
- L1T runs at multiples of 16 ns.
- DAQ reads out time frames as large as few
microsec, as appropriate to subdetector
technology. - May not be possible for many FEE systems
- Increases data volume on FEE links through DAQ
- Yes, SLHC FLTs could operate at 40 MHz,
- But, 40MHz FLTs will generally suffer more from
pile-up.
15FLT Should SLHC FLTs operate at 80 MHz ?
- Advantages
- Reduced pile-up effects
- more effective algorithms
- less data volume (for detectors that identify
Xing during r/o) - Ability to identify crossing that caused trigger
- Allows more processing steps within fixed latency
- 80 MHz electronics feasible (Portions already at
80MHz.) - Disadvantages
- Requires FLT upgrades
- Increased data bandwidth into within FLT
- FEE may not be able to deliver data to FLT at
80MHz - Study (cost/benefit) needed.
16FLT Can SLHC FLTs operate at both 40 80 MHz ?
- Portions of FLT could operate at 40 MHz while
other portions operate at 80 MHz. - Note Identification of 12.5 ns crossing can be
derived from 40 MHz samples for calorimeters. - Time resolution of calorimeter pulses is much
better than 25 ns. - Timing of pulses is derived by digital filtering
of multiple samples. - Such a scheme already used in ATLAS CSCs w/
sampling at 20 MHz - Some such hybrid likely to be a good solution.
17High-Level Triggers Data Acquisition
- Commercial computing networking technologies
will provide the advances necessary for High
Level Triggers (HLTs) and Data Acquisition (DAQ)
systems to perform at SHLC rates. - e.g. Moores Law will provide x10 improvement in
priceperformance ratio wrt LHC start-up 5 years
after start-up. - e.g. Appetite for high-bandwidth graphical
computing applications will drive networking
capabilities up and costs down. - RD is necessary to develop technology advances
for HEP applications. - Data transfer
- Data links
- HLT/DAQ networks
- Data Sources / Readout Buffers
- Data processing
- New HLT algorithms to maintain HLT selectivity
with increased pile-up - Complexity handling
18Data Links
- New challenges at SLHC
- for data links from Front-end Electronics to
Trigger DAQ - Higher bandwidths due to higher occupancies (
FLT rates?) - Radiation effects at transmitting end
- for some subdetectors, e.g. systems optimized for
LHC - Also to increase input capabilities of FLTs
- Limitations on data input often limit FLT
capability. - RD needed
- Applications of commercial developments
- Possible custom developments
19HLT/DAQ Networks
- Networks connect HLT processors to data sources.
CMS/Cittolin
- Every processor connects to every source.
- Data moves from sources to processors in parallel
(parallel event building) to handle high
trigger rates.
20HLT/DAQ Networks
- Commercial networking equipment provides the
infrastructure for interconnection (switches) and
data transfer (links). - At present, the cost of this equipment determines
how much data experiments can afford to move to
HLT processors. - Thus, cost can limit trigger rate capability.
- ATLAS has adopted an RoI-based Level 2 trigger
in order to reduce overall data bandwidth
requirements. - CMS has developed a scaleable event building
architecture. - SLHC network bandwidth at least 5-10 times LHC
b/w. - Even if SLHC FLTs can provide same rate as at LHC
- Network bandwidth requirements grow with
occupancy.
21HLT/DAQ Networks
Complete HLT/DAQ systems require large networks.
Individual network switches not yet of size
required for full interconnectivity.
22HLT/DAQ Networks RD
- RD should track evolution of network technology
- Seek switches with requisite number of ports to
avoid multiple switches extra ports - Technical challenge to manufacture of switches
with many high-speed ports is bandwidth
capability of switch fabric (backplane) that
interconnects ports. - Switches from different vendors behave
differently within HEP systems - Due to internal differences (e.g. buffer sizes)
- RD will sometimes require very large-scale
testbeds, which could be provided by large farms
foreseen for LHC computing and Grid projects. - If we desire to use commodity technologies, then
anticipate use of 10 Gbit Ethernet, as well as
Gigabit Ethernet in SLHC systems.
23Data Sources / Readout Buffers
- This is electronics that buffers detector data at
the input of HLT/DAQ systems and that sources
data into HLT/DAQ networks. - They tend to operate at highest rates
- (relative to other HLT/DAQ components).
- Cannot benefit from event parallelism,
- as exploited by other components
- Each data source must function at full FLT rate.
- Increased occupancy (and increased FLT rate)
increase data source internal bandwidth
requirements. - These elements likely to need upgrade for SLHC
rates. - Compare performance with SLHC requirements
24Data Sources / Readout Buffers RD
- Upgrade directions
- Internally, these elements tend to employ busses
for data transfer. - Reasonable for O(80Mword/sec) on circuit board
- Challenging for higher bandwidths between groups
of modules (e.g. on backplanes) - High-speed serial connections may provide better
data transfer in upgrades. - Technology trends in this direction
- FEE inputs to these elements already on serial
links. - Serial output links to HLT/DAQ network with
bandwidth comparable to bandwidth of input links
will remove bottleneck. - I.e., push network technology closer to FEE, to
exploit - High-speed serial links
- Parallelism afforded by networking
25HLT Algorithms
- New HLT algorithms needed
- To maintain selectivity in face of increased
occupancy and pile-up - e.g. Electron triggers
- Dominate FLT output rate, unless thresholds
raised high - FLT rate 22 kHz _at_ 30 GEV threshold at LHC
- FLT selectivity degraded because pile-up blurs
isolation - HLT algorithms must recovery selectivity despite
degraded isolation. - How can this be accomplished? Will refined
tracking information need to brought to bear
early in selection sequence? - e.g. Muon triggers
- Degraded by occupancy
- Degraded by loss of momentum-resolution at higher
threshold - HLT algorithms must recover selectivity.
- HLT algorithms must accommodate upgraded muon
detectors. - Moores Law will provide data processing power
for new algorithms.
26Complexity Handling
- HLT/DAQ systems are extremely complex.
- Large numbers (thousands) of processors
- Even larger numbers of software processes tasks
- Highly distributed, heterogeneous system
- Real-time demands not present in offline
systems - Complex control (e.g. startup shutdown)
procedures - Remote access required for monitoring
troubleshooting - Very high reliability required
- Robustness, redundancy, fault tolerance
- Including robustness of complex selection
algorithms - Note Technology evolution may mean that SLHC
HLT/DAQ systems are no larger than for LHC. In
this case, SLHC complexity stays same as LHC
27Complexity Handling RD
- RD can develop solutions to manage complexity.
- RD can track development of tools for
complexity handling in very large commercial
and other applications. - E.g. web tools from e-commerce for high-level
controls and user interfaces - Similar remote access, security, database issues
28Summary
- Higher Luminosity gt
- Increased event pile-up detector occupancy
- Increased trigger rates data volume
- Increased radiation effects
- Enabling technologies
- Integrated circuits custom, FPGAs, memories,
etc. - Commodity computing networking
- Challenges arise in data transfer in data
processing - Processing challenges felt mainly in First Level
Triggers - Data transfer challenges felt in both FLTs and
HLT/DAQ - Trigger rates manageable by
- Increasing thresholds on inclusive triggers
- Using exclusive triggers where low thresholds
needed - Technical solutions exist, but RD is required.
29RD Summary
- First Level Triggers
- Operation with 80 MHz crossing rate
- Coping with increased occupancy (muons) pile-up
(calorimeter) - Adapting to new sub-detectors or readout
- Radiation-tolerant on-detector FLT electronics
- Input data links from Front-end Electronics
- High Level Triggers Data Acquisition
- Coping with increased data bandwidth
- Input data links, HLT/DAQ networks, data sources
/ readout buffers - Coping with increased occupancy pile-up ( new
detectors or r/o) - New HLT algorithms
- Complexity handling