The D

1
The DØ Control System

• J. Frederick Bartlett
• For
• The DØ Controls Group

2
Outline

• Fermilab and the D0 Experiment
• The DØ Control/Monitoring System
• IOC extensions
• Process Variable Naming Conventions
• Database Management
• Significant Event System
• Detector Configuration Management
• Operator Display Support Library

3
The Fermilab Collider Complex
CDF
DØ
4
DØ Detector
Forward Muon Layer C
Superconducting Solenoid
Liquid Argon/Uranium Calorimeter
Scintillating Fibers
Central Muon Layer A
Central Muon Layer B
Silicon Tracker
Central Muon Layer C
Muon Toroid
5
DØ Run II Control System

• EPICS-based controls
• Field buses
• VME
• MIL/STD1553B
• CANBUS (Run IIB)
• Size
• 15 host-level processors
• 100 IOCs (Input/Output Controllers)
• MVME2301, MVME2304, MVME5500
• 6600 high-level devices
• 138000 process variables (records)

6
DØ Run II Control System

• 15 major detector sub-systems
• Host-Level processes written in Python
• Source management - CVS
• Operating systems
• Host processors - Linux
• IOC processors vxWorks
• Software versions
• Now - EPICS 3.14.6
• Next - EPICS 3.14.7 with patches, using SNS linux
  build tools
• Controls staff
• Core system - 3 FTEs (4 people)
• Detector-specific components - 2 FTEs
• Primarily from other institutions

7
Devices and Records
Detector Devices Records
Argon/Uranium Calorimeter 844 9219
Scintillating Fiber Tracker 445 35900
Muon Spectrometer 1572 20027
Silicon Microstrip Tracker 2235 45706
Controls 440 3775
Trigger Systems 694 21644
Other Systems 468 2693
Total 6653 138964
8
EPICS IOC Extensions
HV Generator State Machine
Customizing Elements
VME Bus Access With Retries
MIL/STD1553 Bus and CANBUS
9
Gateways to Other Systems

• Accelerator
• Gateway link to ACNET system
• Bidirectional
• Data access only (no control)
• Cryogenics and Gas
• Gateway link to DMACS system
• Read-only
• Data access only (no control)

10
ACNET Gateway
11
DØ Naming Convention

• Name elements
• Detector ltdetgt CAL
• Sub-det ltsubgt N
• Device ltdevgt VBD
• Locator ltlocgt 01
• Attribute ltattrgt STATUS
• I/O ltiogt W
• Field ltfieldgt SCAN
• Template

Frequently used in filters
Many utility tools use this field for
device-specific actions
ltdetgtltsubgt_ltdevgt_ltlocgt/ltattrgtltiogt . ltfieldgt
CALN_VBD_01/STATUSW.SCAN
12
Database Management

• Basic EPICS uses flat text files
• Template definition file
• Multiple record definitions with substitution
  parameters
• Template generator file
• Create a group of records (instance of a
  template) with values for parameters
• Record instance file
• Instances of records read by IOC to create record
  database

13
Relational Database System

• EPICS database creation from text files
• Record structure defined by DBD files
• Template structure defined by template definition
  files
• Template instances defined by template generator
  files or from Browser GUI
• IOC record instances (record instance file) are
  extracted from database
• Database support programs written in Python

14
Relational Database System
Template Definitions
Template Generators
Record Field Definitions
Instance Creation
DB Creation
Oracle Hardware Database
Record Extract
Template Extract
List of Records For IOC
Template Generators For IOC
IOC Id
15
Significant Event System

• The significant event philosophy
• Alarms are a only a sub-set of the significant
  events
• EPICS does not generate all of the significant
  events of interest
• Alarm utilities enhance reliability
• Detect impending failures and fix them before
  they fail
• Minimize the time to correct failures
• Archive all event transitions
• The archive is a history of the state transitions
  of the experiment
• Tools available to search the event archive
• What was your system doing last month?

16
Alarm/Event Distribution

• SES is a server-based, event (alarm) system
• IOCs and user processes connect to and send alarm
  transitions to the server
• Pushed by sources not pulled by the server or
  client
• Server holds the current experiment (alarm) state
• Server has one or more filters for each receiving
  client
• Makes use of name structure
• Rapid display startup of receiving clients
• User processes may also declare events via API
  (C, C, Python)
• Written in Python

17
Significant Event System
Significant Event Server
18
SES Alarm Display
Select Guidance button
Events in line selected by filter
Select minor alarm button
19
Detector Configuration Management

• EPICS Save-and-Restore is adequate for
  accelerators
• Accelerators are tuned for an operation mode and
  the configuration is saved for future restoration
• Some complex devices like HEP detectors do not
  work this way
• There are too many distinct modes to be saved
• Tuning may not be required
• Settings are often predictable from trigger
  selection
• Many detector component settings are taken from
  separate calibration measurements

20
Comics Server

• Manages the configuration of the detector
• Receives load requests from
• Run control
• Expert GUIs
• Program APIs
• Load map is a directed graph (tree)
• Tree node (intermediate)
• Establishes a layered hierarchy
• Establishes an execution order
• Action node (leaf)
• Issues EPICS CA requests

21
Comics Server

• Constructed on the server model with multiple
  clients sending commands
• Server may be activated independently for
  configuring detector component
• Coded in the Python language

22
Configuration Management Tree
Intermediate Tree Node
Root Node
Links determine the order of execution
DØ
Configuration Data
Action Node Uses EPICS Channel Access
23
Comics Expert GUI
24
Operator Display Library

• Standard, process flow (synoptic) displays do not
  adapt well to the monitoring of HEP detectors
• Like accelerators, there are many similar devices
  that make up the typical detector component
  (cells in a calorimeter)
• The components are not related in a serial
  fashion like a cryogenic plant
• Tabular (spread-sheet designs) are more natural
• Similar properties for different devices are
  easily compared
• Deviations are apparent

25
Operator Display Library

• DØ has developed a series of Python display
  classes for building tabular displays that
  collect information from EPICS process variables

26
Resource Display
Basic Frame with Menu Bar, Status Window, and
Button Bar
Tabbed Pages with CA Connect and Disconnect
Device Status (Bit-Encoded) Property
Device Window
Device Analog Property
Right-Click on Name Field