Architecture of the

1

Architecture of the ATLAS High Level Trigger
Event Selection Software
Monika Grothe, CERN EP, on behalf of the ATLAS
Trigger/DAQ High Level Trigger group

• Outline
• General strategy of the HLT event selection
• The components of the HLT event selection
  software
• Using offline code in the HLT online context

2
The stage
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
3
ATLAS data flow diagram
Level 1 trigger
Region of Interest RoI
Level 2 trigger
High Level Trigger
Event Filter
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
4
ATLAS trigger overview

• Level1 hardware trigger, 40 MHz ? 75 kHz, 2.5
  ms maximum hardware latency
• Looks for regions of potentially interesting
  activity, with high pT objects
• Region of Interest (RoI) muon, electromagnetic,
  tau/hadronic, jet clusters
• Looks in calorimeters, muon spectrometer
• Does not combine information of more than one
  detector

• Level2 software trigger, 75 kHz ? 1 kHz, 10 ms
  average latency
• Data are held in readout buffers (ROB) during
  Level2 processing
• Selection software run by Processing Application
  on one node of Level2 farm
• (PCs, Linux)
• Input (seed) is Level1 RoI (its type, position,
  pt threshold passed) so that typically
• only few of full event information in the
  ROBs need to be transferred to Level2
• Feature extraction in RoI region by specialized
  algorithms that are optimized for
• speed and cover all sub-detectors
  sequentially
• For events accepted by Level2 EventBuilder
  builds full event

• EventFilter software trigger, 1 kHz ? 100 Hz,
  1 sec average latency
• Full event from EventBuilder passed to
  EventFilter farm (PCs, Linux)
• Independent Processing Applications run
  selection algorithms on farm nodes
• Selection software consists of offline-type
  algorithms that have access
• to latest calibration and alignment data

Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
5
How the HLT works
Signature ?

• Level1 finds 2 isolated
• em clusters with each pTgt20 GeV
• possible signature for Z ? ee-
• Method
• Validate step-by-step
• Check intermediate signatures
• Reject at earliest possible moment
• Managed by
• HLT Steering package

Iso lation
Iso lation
STEP 4
Signature ?
pTgt 30GeV
pTgt 30GeV
STEP 3
Signature ?
t i m e
track finding
track finding
STEP 2
Signature ?
Alg
Alg
Cluster shape
Cluster shape
Alg
STEP 1
Alg
Alg
Alg
Alg
Alg
Alg
Level1 seed ?
Alg
Alg
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
6
Main components of HLT selection software
HLT DataFlow Software
Event Filter
HLTSSW
HLT Selection Software
Processing
HLT Core Software
Application
Level2
Steering
HLT Algorithms
Processing
Application
HLT Algorithms
Data Manager
ROBData Collector
Event DataModel
Event selection how ? Steering
what ? - HLT ALgorithms Data access
how ? - DataManager what ? -
EventDataModel
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
7
Event selection - how ? HLT steering

• For each step there exists a
• TriggerMenu
• List of signatures for which
• an event can be accepted,
• e.g.
• SequenceTable
• Specifies which algorithms are
• to be run given the input objects
• to the step, e.g. run on each
• input object algo

StepController
StepHandler
ecand
ecand

StepSequencer
StepDecision
STEP 1
ecand
track finding
STEP i
Different from the typical find-all-electrons/trac
ks/ in event Localized reconstruction, guided
by seed Steering calls relevant algorithm once
per seed, in same event
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
8
Event selection - how ? HLT steering (2)
Steering hands over seed to algorithm via
TriggerElements Algorithm returns result of
processing as another TriggerElement

• TriggerElement
• has a label
• doesnt hold any data content
• is related to DataObjects and other
  TriggerElements in a navigable way
• Signatures in a TriggerMenu consist of logical
  AND/OR of TriggerElements

Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
9
Event selection - how ? HLT steering (3)
Steering realm Hypotheses
HLT algorithm reconstruction realm Supporting
information
etc.
etc.
etc.
uses
TriggerElement e
DataObject track
DataObject cluster
uses
1 ..
seeded by
TriggerElement ecand
uses
DataObject CaloCells
uses
DataObject CaloCluster
1 ..
seeded by
uses
TriggerElement EM20i
DataObject Level1 EMRoI
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
10
Data Access What ? Raw Event Data Model

• DataFlow system ships raw event data to HLT in
  bytestream format,
• for Level2 fragments on demand, for
  EventFilter complete event
• Raw data objects (RDO) object representation
  of raw event data (bytestream)
• Algorithms access raw event data by means of
  RDOs and transient event store
• Importance of raw Event Data Model (EDM)
• Different granularity of data as provided by
  DataFlow system ( min. granularity ROB)
• and data as deployed by algorithms (per geometric
  unit DetectorElement)

Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
11
Raw EDM (2)
Example ATLAS Muon Spectrometer
DetectorElement
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
12
Data Access How ? ByteStreamConverters
Transient EventStore
Region Selector
Algorithm
Trans. Event Store
HLT Algorithm
Region Selector
region
list DetElem IDs
list DetElem IDs
list DetElem IDs
ROB ID
raw event data
DetElems
DetElems
ByteStreamConverter maps DetectorElements onto
minimum readout granularity Geometry ? ROBs
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
13
ByteStreamConverters (2)

• HLT steering calls an HLT algorithm by passing
• to it a TriggerElement with a navigable link
  to a
• RoI DataObject that contains the RoI
  position
• The HLT algorithm typically needs to access the
  
• raw event data in the DetectorElements
• in a certain region around the RoI position
• The HLT algorithm requests from the
• RegionSelector the identifiers of all
• DetectorElements within that region
• The HLT algorithm passes the list of
• identifiers to the transient event store
• At this stage 2 cases can occur
• 1. The requested DetectorElements are already
  available and are just returned
• 2. The information first needs to be retrieved
  from the DataFlow system
• The transient event store invokes a
  ByteStreamConverter which requests
• the bytestream data from the correct ROB
• The ByteStreamConverter extracts from the
  succession of 32-bit words in
• the ROB bytestream the relevant event data and
  fills the collections of RDOs
• The thus obtained collections of RDOs are stored
  in the transient event store

Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
14
Offline code in online context

• Design aim Use same environment for development
  maintenance (offline)
• and deployment (online) of HLT selection
  algorithms
• Advantage
• Ease of developing HLT algorithms and studying
  their performance on
• simulated and real data in well-tested
  user-friendly offline environment
• Ease of code migration offline ? EventFilter ?
  Level2
• ATLAS offline framework ATHENA, based on a GAUDI
  (developed by LHCb) core
• Basic principles
• Separation of data and algorithms which produce
  and consume data
• Separation of transient and persistent
  representation of data
• Converters convert one data representation into
  another
• Consequences for HLT design e.g. Algorithms
  forced to communicate via transient
• event store, persistent data made available by
  converting into transient
• representation stored in transient event store,
  common database access tools, etc
• Importance of these offline-imposed elements to
  meet stringent online
• performance requirements

Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
15
HLT selection software offline dependencies
HLT DataFlow software
Event Filter
HLT selection software
HLTSSW
Processing
HLT Core Software
Application
1..
Level2
Steering
Monitoring
Service
HLT Algorithms
Processing
Application
Data Manager
HLT Algorithms
ROBData Collector
1..
EventData Model
MetaData
Service
ltltimportgtgt
ltltimportgtgt
ltltimportgtgt
ltltimportgtgt
Offline
Package
Offline
Athena/ Gaudi
StoreGate
Reconstruction
EventDataModel
Algorithms
Interface
Offline Reconstruction
Offline Architecture Core Software
Dependency
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
16
Complications from special Level2 environment

• Stringent requirements on framework in Level2
• Speed
• Robustness
• Multi-threading

• Example multi-threading Used to keep idle time
  of CPUs in Level2 farm to minimum
• Implications
• Control over event loop needs to be with the HLT
  DataFlow software
• All algorithms/services have to be
  configured/initialized outside event loop
• ATHENA controls event loop in the offline
  environment
• How to reconcile with common online/offline
  environment for HLT select. software ?

Athena Level2
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
17
Complications from special Level2 environment (2)

• Solution
• SteeringController as interface layer between HLT
  selection HLT DataFlow software
• 1. Uses GAUDI light, i.e. SteeringController
  uses only minimum set of GAUDI features
• 2. Uses Modification to GAUDI base libraries
  such that for every thread
• EventLoopMgr_(threadID)-gtexecuteEvent()
• AnyServiceNeededByAlgo_(threadID)
• AnyHLTAlgorithm_(threadID)
• Modified GAUDI can be used in offline environment
  as ATHENA is based
• on GAUDI (suffix collapses to blank)
• Modifications since included in GAUDI as standard
  features
• Unexpected pitfall of using off-the-shelf
  software in Level2 STL locks
• Thread-safe but thread-inefficient implementation
  of STL containers
• Use different implementation of allocator
• Due to different way DataFlow and HLT selection
  software interact in EventFilter
• ATHENA can be used as framework for running HLT
  algorithms
• Timing issue 30 of processing time spent in the
  raw EDM available at that time

Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
18
Conclusions and outlook

• For HLT Technical Design Report in summer 2003
• Architecture of the HLT selection software is
  currently in validation stage
• Integration of HLT selection software with other
  components of the HLT
• software on-going
• Setting-up of full trigger vertical slices
  (electron/g and m) in testbeds
• Detailed measurements of timing and physics
  performance on-going
• Re-use of offline code in online context
  currently experimental
• News from testbeds encouraging, though fair
  amount of code hand-tuning
• by online required
• But also very fruitful dialogue offline-online
  with concrete results

Why dont you program your trigger in assembler
? (offline to online) Improvement of CPU speed
allows ever higher level of abstraction
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
19
More details in the ATLAS talks _at_ parallel
sessions

• The StoreGate a Data Model for the Atlas
  Software Architecture
• P. Calafiura, Monday StoreGate
• The Algorithm Steering Trigger Decision
  mechanism of the Atlas High Level Trigger
• G. Commune, Tuesday HLT Steering
  package
• The Region of Interest Strategy for the ATLAS
  Second Level Trigger
• V. Boisvert, Tuesday RegionSelector
• Configuration of event-selection criteria in the
  ATLAS trigger system
• M.Elsing, Thursday Use of
  TriggerMenus
• Use of a generic identification scheme
  connecting events and detector description
• in the ATLAS experiment
  DetectorElement Collection identifiers
• R.D. Schaffer, Thursday

Further ATLAS Trigger/DAQ talks The DataFlow
system of the ATLAS Trigger DAQ, G. Lehmann,
Monday The use of Ethernet in the DataFlow of the
ATLAS Trigger DAQ, S. Stancu, Monday
Experience with multi-threaded C applications
in the ATLAS DAQ system, S. Gadomski,
Monday Supervision of the Atlas High Level
Trigger System, S. Wheeler, Tuesday Online
Monitoring software framework in the ATLAS
experiment, S. Kolos, Thursday
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
20
Level2 testbed
Purpose Integration of all components of the HLT
selection software Only minimum of components of
the DataFlow system necessary to run the HLT
selection software
Switch with 8 fast Ethernet ports (for single
processor PCs) and 2 Gigabit Ethernet ports
(for dual processor PCs)
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03