SLAC Detector R

1
Electronics Data-Acquisition Development

• Gunther Haller
• Particle Physics and Astrophysics Research
  Engineering Group Leader
• DOE Detector RD Review
• July 8-10, 2009

2
DAQ Electronics

• This presentation gives an overview of selected
  Detector RD efforts at SLAC in the area of
  electronics and DAQ
• See write-up for additional efforts, some are in
  back-up slides
• Front-end electronics and readout
• Sensor readout, mixed-signal ASICs
• Low-noise, many channels, direct connections
• Analog, digital, processing all on chip (system
  on chip)
• KPiX ASIC (3 slides)
• High-occupancy ASIC in back-up slide
• Readout systems for front-end electronics
• Pixel sensor readout (1 slide)
• Multi-gigabit data transmission integrated inside
  detector system
• Transmission RD and concentrator ASIC (backup)
• Radioactive free front-end electronics for
  underground experiments (2 slides)

ASIC Application Specific Integrated Circuit
3
DAQ Electronics cont

• High-reliability systems (inaccessible ground
  electronics, space based electronics)
• Applicable to e.g. JDEM and also most future
  ground-based projects (LSST, SID, etc) (1 slide)
• Focal-plane detector readout electronics (1
  slide)
• High-rate, high volume data-acquisition system
  from detector to offline
• SLAC RCE and CIM modules (5 slides)
• Realtime DAQ software development
• Electronics system architectures
• RD on electrical/mechanical/analog/digital/power/
  cooling interaction and consequences on
  performance
• Important at early stage to avoid problems later
  on (SLAC examples are SLD, BaBar, GLAST/FERMI,
  LCLS experiments, EXO-200, SiD)
• Mechanical, thermal, electrical integration and
  packaging of detector electronics (in write-up)
• Several potential RD projects wont be able to
  be pursued, would need additional funds, see
  write-up for descriptions
• Fast Timing ASIC, Polychrome Imager ASIC, 3D ASIC
  Assemblies, Graphical Processing Units

4
SLAC PPA REG Selected Projects

• Past Current
• BaBar
• PEPII Detector at SLAC
• GLAST/FERMI
• Space-based detector, in orbit
• Atlas
• Help in DAQ for Muon detector system
• EXO-200
• Detector readout system, low-noise analog, DAQ
• LCLS X-Ray Experiments
• Detector/ASIC electrical/mechanical integration,
  DAQ

• Future
• JDEM
• Space-based detector, DAQ
• Linear Collider
• Detector, RD
• LHC Atlas Upgrade
• Tracker DAQ
• LSST
• Ground-based telescope, detector/readout, DAQ
• EXO
• Detector readout system, low-noise analog, DAQ
• Various RD efforts

5
KPiX ASIC

• The intent is to obtain a system-on-chip design,
  which
• increases integration in order to eliminate a
  costly and capacitive cable plant for the
  detector signals
• minimizes the occupied volume and extraneous
  materials
• enables higher density/more functionality inside
  detector
• new technologies making future projects more
  affordable
• changes cost/channel way to estimate cost
• In addition reliability is increased by
  substantially reducing the number of required
  connections
• Only 4 signals are required to configure the
  1,000 channel chip, autonomously control the
  acquisition, and read-out of data
• Applications are e.g. in several SiD sub-systems

6
KPiX ASIC Description and Functionality

• 3232 array 1,024 channels
• Designed to be
• bump-bonded to the Si sensor, or
• bumped to a hybrid for large area detectors
  (RPCs, GEMs, etc)

• For each channel of the system-on-chip
• 4 samples per train with individual timestamps
• auto-triggering
• internal per-channel 13-bit ADC
• automatic range switching for large charge
  depositions
• bias current servo for DC coupled sensors
• power down during inter-train gaps (20 uw avg
  power)
• built-in calibration
• nearest neighbor trigger ability
• high-gain feedback capacitor for tracker
  application
• dual polarity for GEM and RPC applications
• external trigger for test beam
• Digital IP core with serial data IO (only 4
  signals)
• 0.25µm TSMC

Blockdiagram of a single channel
KPiX64
Noise
7
KPiX ASIC Past Future Effort

• Past effort
• Designed, fabricated, and tested 64-channel KPiX
• Developed hardware and software for
  control/readout/data analysis
• Support detector sub-system tests
• Designed and fabricated 256-channel version
• Future effort
• Modify test hardware/software
• Evaluate 256-channel version
• Design, fabricate, evaluate 1,024-channel version
• Collaborate in detector specific packaging
• Support effort in detector specific testing

256-channel KPiX
Example Calorimeter 12 KPiXs on cable
Example Tracker
8
Pixel-Detector Readout Electronics

• Development of pixel (e.g. NIR or HyViSi)
  detector readout electronics for large
  temperature range applications (room-temperature
  down to 100K)
• Examples space-based (JDEM) or ground-based
  (LSST) telescopes
• High reliability design, space-flight
  qualification path
• Past Effort
• Design electronics, assemble, test
• Development of micro-code executing in custom
  ASIC
• Future Effort
• Functionality and performance tests at
  room-temperature and cold
• Iteration on design if required

Custom ASIC Board
Dewar with detector and ASIC
9
Radioactive free Optical FE Module

• SLAC has designed, implemented and is currently
  testing electronics and DAQ for EXO-200
• See slide in back-up, is also example of system
  design effort at SLAC
• Electronics is external to chamber
• Problem with electronics inside chamber for
  underground experiments due to radio-active
  contamination
• RD towards radio-active free integrated
  front-end module inside the detector would
  provide substantial benefit

Bulk silicon and some CMOS technologies have been
proven to be radioactive free (ex. TSMC 0.25um)
free negligible radioactivity levels 10-12
levels of U, Th, and K40

• Advantages
• Minimized interconnection length between
  sensors and front-end electronics
• Minimized input capacitance and thus electronics
  noise
• Reduced space required for data interconnections
• Minimized impact on the chamber architecture
• No need for AC coupling

Optical Components VCSEL - PIN
Photovoltaic cell
V-grooves for Fiber alignment
ASIC
Optic Fiber
Silicon PCB
10
Radioactive Free Electronics

• Silicon PCB specifications
• Passive Components
• V-Grooves for Fiber Alignment
• Integrated Photovoltaic cell

Integrated capacitors 1fC/um2
Si module
V grooves for fiber alignment

• Research and Development of a radioactivity
  free optical front end module at SLAC for
  underground experiments
• Experience at SLAC regarding radio-active issues
  from EXO-200 effort
• Previously fabricated Si boards with integrated C
  and R at our Nanofabrication laboratory
• Need to evaluate VCSEL, PIN, Photovoltaic cell
  for suitability radioactivity
• Plan to design and fabricated prototype of
  elements of front-end module
• Corresponding Cryo-ASIC block diagram in back-up
  slides (would need additional funds)

11
Space-Based Electronics

• SLAC past experience
• Electronic project management of GLAST/FERMI
  Large Area Telescope Instrument
• In space gt 1 year, very successful
• Most of electronics, all DAQ designed/implemented/
  tested at SLAC
• Effort towards future JDEM mission next slide

All-sky survey showing 16 new gamma-ray only
pulsars and 8 msec pulsars
12
Focal-Plane Readout Electronics
Focal Plane (s) with gt 100,000,000 pixels

• SLAC effort towards JDEM
• Design of overall electronics system architecture
• Design of several modules for focal-plane
  control, readout processing
• Future effort
• Will transition into JDEM project
• Applications
• JDEM or other focal-plane detector control and
  readout systems, ground or space-based

FPE Pwr, Cntrl Readout
Instrument Control Unit
Spacecraft
To/from other sub-systems/focal-planes
FPE
ICU
Sensor
Custom ASIC Module
Sensor readout and processing electronics
13
High-Rate, High-Volume Data Acquisition
In-detector front-end electronics sensors/ASICs
In-detector concentrators, fiber optics interface
Off-detector DAQ controls/data-processing
Fiber

• DAQ RD has identified need for 3 types of
  building-blocks for DAQ section
• Computational elements
• must be low-cost (, footprint, power)
• must support a variety of computational models
• must have both flexible and performant I/O
• Mechanism to connect together these elements
• must be low-cost
• must provide low-latency/high-bandwidth I/O
• must be based on a commodity (industry) protocol
• must support a variety of interconnect topologies
• hierarchical, peer-to-peer, fan-In fan-Out
• Packaging solution for both element
  interconnect
• must provide high availability
• must allow scaling
• must support different physical I/O interfaces
• preferably based on a commercial standard

• The Reconfigurable Cluster Element (RCE) based
  on
• System-On-Chip technology (SOC)
• Virtex-4 5

• The Cluster Interconnect Module (CIM)
• Based on 10-GE Ethernet switching

• ATCA
• Advanced Telecommunication Computing Architecture
• Crate based, serial backplane

14
Detector Control and DAQ Chain
ATCA Crate
RCE
CIM
Fiber
FPGA board
Detector/ASIC

• DAQ ATCA (Advanced Telecom Computing
  Architecture)
• Based on 10-Gigabit Ethernet backplane serial
  communication fabric
• 2 custom development boards
• Reconfigurable Cluster Element (RCE)
• Interface to detector
• Up to 16 x 6.5 Gbit/sec links to detector modules
• Adoptable to different detector interfaces
  protocols via Rear Transition Module
• Cluster Interconnect Module (CIM)
• Managed 48-port 10-G Ethernet switching
• Interconnection of RCE modules
• Interconnection to event-builder, L3, other crates

ATCA crate with SLAC DAQ Boards

• Scalable
• One ATCA crate can hold up to 14 RCEs 2 CIMs
• 480 Gbit/sec switch capacity
• Crates can be interconnected
• Architecture with building blocks
• Adoptable to different experiments
• Via software, firmware, and Rear Transition
  Modules

15
SLAC PPA DAQ Modules Reconfigurable Cluster
Element

• Reconfigurable Cluster Element module with 2 each
  of following
• Virtex-4 FPGA
• 2 PowerPC processors IP cores
• 512 Mbyte RLDRAM
• 8 Gbytes/sec cpu-data memory interface
• 10-G Ethernet event, 1-G control data interfaces
• RTEMS operating system
• Up to 512 Gbyte of FLASH memory
• 1 TByte/board
• RCE Rear Transmission Module
• Provides for tailoring to specific detector,
  fiber or electrical
• Firmware and Software

Reconfigurable Cluster Element Module
RCE Rear Transition Module
16
SLAC PPA DAQ Modules Cluster Interconnect Module

• Cluster Interconnect Module
• 10-GE switch ATCA low-latency network card
• Interconnection of up to 14 ATCA 10-G IOs plus
  external IO
• 2 x 24-port 10-G Ethernet Fulcrum switch ASICs
• Managed via Virtex-4 FPGA
• Firmware
• Supports a variety of interconnect topologies
  (hierarchical, peer-to-peer, fan-in fan-out)
• Software
• CIM Rear Transmission Module
• Provides for tailoring to specific application
• 12 channels of XFP 10-GE fiber to
• interface crate to external systems
  (event-builder, L3, etc)
• interconnect crates

CIM Rear Transition Module
Cluster Interconnect Module
17
SLAC PPA DAQ Effort

• Past effort
• Designed implemented prototype of RCE and CIM
  modules
• Ported RTEMS open-source real-time operating
  system to module
• Development of firmware and real-time software
• Future effort
• Modifications of existing module Higher clock
  speed, more gates, more memory and more DSP
  tiles, Virtex-5 cross-bar to tap into the native
  memory system, true zero-copy transfers, layer-3
  interface, real-time software
• Stay at cutting edge of technology
• Enables future experiments, substantial
  performance and cost benefits
• Some applications
• Peta-Cache, LCLS detector control/readout/DAQ,
  LSST DAQ, LSST camera simulator, ATLAS upgrade of
  TDAQ, and SiD
• Main thrust
• Combined hardware software infrastructure to
  help exploring the high speed I/O and vast
  processing power of the modent large FPGAs that
  makes the RCE concept attractive
• Building blocks, can be customized at firmware
  software level and hardware interface RTM level
• SLACs ability to step into the ATLAS muon CSC
  DAQ with very short notice is a prime example of
  the value of core expertise built up over many
  projects at the frontier of DAQ technology

18
Selected Milestones for some of the RD Efforts
19
Summary

• SLAC has an excellent track record in engineering
  all aspects of major PPA experiments, from BaBar
  and GLAST/FERMI in the recent past, to ongoing
  efforts with LSST, JDEM, EXO, ATLAS, ATLAS
  upgrade, and Linear Collider Detector
• Strong engineering base for experiment
  development
• Need to support core group to maintain and
  advance knowledge
• Generic RD important, is source of innovation,
  advances the field, and enables future
  experiments (potential ATLAS DAQ upgrade only one
  example)
• Substantial competency in several areas
• Detector, ASIC, readout system development
• High-rate, high volume data-acquisition systems
• In addition strong systems approach to developing
  electronics from sensors, through triggers and
  processing, ground-based and space-qualified
  hardware and real-time software
• Described several accomplishments in the area of
  Detector RD, and plans for future use of funds

20
Backup
21
Research Electronics Core Competencies

• Unique Systems Design Capabilities
• Design of complete electronics system
  architectures
• From detector to offline
• System-engineering
• Front-End Electronics/DAQ
• Analog, digital mixed-signal ASIC design
• Low-noise electronics system design and
  performance evaluation
• Integration of detectors with electronics,
  mechanical electrical
• High-speed communication links
• High-speed, high volume data-acquisition
  processing systems
• Electronics for spaceflight
• DAQ Hardware and Software
• Real-time software design
• Low-latency-high-density storage systems

22
ASIC for High-Occupancy Applications

• Requirements
• High occupancy (100)
• Large input signals (40pC)
• High input rate (3.25 MHz)
• Dual-gain front-end electronics charge
  amplifier, pulse shaper and T/H circuit
• ADC, one per channel
• Digital memory, 2820 (10 bits parity) words per
  channel
• Analog addition of 32 channel outputs for fast
  feedback low-latency ADC
• Radiation hard
• Potential Applications
• BeamCal Readout ASIC for SiD

ASIC Simplified Block Diagram (only one channel
shown)
23
ASIC for High-Occupancy Applications cont

• Past effort
• Designed, fabricated, and tested novel ADC to be
  used in ASIC
• Evaluated performance
• Designed, simulated single-channel version
• Future effort
• Finish design and fabrication of hardware and
  development of software for control/readout/data
  analysis
• Layout and fabricate single-channel version
• Evaluate single-channel version
• Test bench and test beam
• Radiation testing
• Design 32-channel version

24
RD for Data Transmission from Detector
Front-Ends

• gt 4m coaxial-type cable with low attenuation,
  high radiation hardness, low amount of material,
  low temperature operation (-40 C)
• E.g. ATLAS pixel upgrade
• Past effort
• Evaluated transmission options
• Future effort
• perform tests, iterate on design
• Possible applications
• Any future detector systems
• sLHC ATLAS Upgrade
• Also effort on Multi-GBit/s transmitter and
  receiver ASIC that
• Multiplexes, enhances the signal, encodes the
  data (e.g. 8B/10B)

Transmission ASIC
25
Research and Development of a radioactivity
free optical front end module for underground
experiments
In underground experiments sensitivity is limited
by background noise
Adequate shielding is a must
No radioactive materials can be used inside the
chamber
Front End Instrumentation (generally
radioactive)cannot be mounted close to the
sensors
Typical approach use of long cables
Impacts on the chamber architecture
IncreaseCosts
FE inputcapacitance
IncreaseNoise
Goal find a way to put electronics inside the
chamber
26
Cryo ASIC for Silicon PCB

• Block diagram of ASIC for Silicon Module
• Needs to operate/perform at cryo temperatures
• Would need additional baseline detector RD funds

Block Diagram of Cryo ASIC
27
Star-Guider with custom ASIC Image Readout

• 2kx2k pixel detector and associated readout
  system
• Measurement and analysis of detector performance
• E.g. JDEM or LSST
• Past effort
• Development of code and readout system
• First performance tests
• Future effort
• Development of guiding algorithms
• Coding
• Functionality and performance tests

Pixel Detector
Custom ASIC
Laser
Pixel Image
28
TPC Readout System Example of System
Architecture, Design

• Effort
• Design, implementation of architecture and
  electronics for EXO-200
• Example of system design, but also RD effort
  towards under-ground experiment electronics

29
Detector Mechanical/Electrical Vacuum Assembly

• RD in packaging of detector ASICs
• Illustrates integration of electronics,
  grounding, cooling, mechanical
• Need to have expertise in all those areas
• Past effort
• Designed several detector and ASIC packaging
  systems
• Future effort
• Provide engineering for future packaging systems

Pixel Detectors Bump-bonded to custom ASICs and
bonded to SLAC custom rigid-flex module