Managing CEBAF Accelerator Operations

1
Managing CEBAF Accelerator Operations
Institutional Management Review August 30/31, 2004

• Andrew Hutton

2
Outline

• CEBAF Accelerator Characteristics
• Response to Hurricane Isabel
• Accelerator Achievements in FY04
• G0 Experiment completed
• Hypernuclear Experiment completed
• HAPPEx-He and HAPPEx-II initial runs
  completed
• Operations Metrics
• Preparing for Upcoming Challenges
• Path forward new Operations Vision
• Summary

3
Brief Description of CEBAF

4
Continuous Electron Beam Accelerator Facility
5
CEBAF Capabilities

• CEBAF delivers independent beams to all three
  Halls
• Energy must be multiple of linac energy
• 1, 2, 3, 4, or 5-pass to any Hall
• All Halls can simultaneously have 5-pass beam
• Current fully independent
• Halls A C take up to 140 µA
• Hall B takes up to 50 nA (and down to 100 pA!)
• Polarization orientation of longitudinal
  polarization depends on Hall energy due to
  precession
• At least 50 of experiments want longitudinal
  polarization
• An increasing number of experiments want parity
  quality beams
• Small helicity-correlated change in current,
  position, angle, polarization

6
Dynamic Operational Requirements

• Unlike a storage ring, the operating conditions
  of CEBAF are changed frequently based on User
  needs
• In FY02, FY03, FY04 there were
• 6 9 3 linac energy
  changes
• 21 15 5 pass changes in Hall
  A
• 8 6 5 pass changes in
  Hall B
• 4 10 4 pass changes in
  Hall C
• 25 30 14 accelerator state
  changes
• On average, the accelerator state changes about
  once per operating week
• This does not include special set-ups for Moeller
  runs, energy measurements, etc.

7
Response to Hurricane Isabel

8
Hurricane Isabel

• Isabel arrived ashore as a Category 1 hurricane
  on September 18, 2003
• Removed electrical power from site for four days
  specifically from CHL so cryomodules warmed up
• Recovery took six weeks
• Aggressive preventive maintenance carried out on
  almost every component - improved reliability
  during the year
• Engineering, SRF Institute, Operations
• Accurate beam set-up provided a solid,
  reproducible base for operations
• CASA, Operations
• Launched us into extremely successful year
  operating period
• Details on Poster

9
Improving Hurricane Preparedness

• Evaluated back-up power options
• Full back-up power is expensive, requires active
  management
• Renting seems better (RFP is out)
• Major investment in switchgear and long term
  contractual obligation
• Decided to implement emergency power loop
• Provides power to critical systems
• Pumps to maintain insulation on cryomodules,
  valve actuators
• Special funds from DOE awarded June 2004
• Expect completion before next hurricane season
  (May 2005)
• Interim, temporary solution developed (extension
  cords, UPS, small generators, etc.)
• Ready to implement if needed
• Initiated aggressive tree-cutting near to offsite
  power line

10
Tree Clearing near Power Line

• Insert Photo Here

11
Accelerator Achievements in FY03/4

12
Experiment Successes FY03/FY04

• G0 required 40 µA at 31.2 MHz every 16th bucket
  filled
• Bunch charge 6.5 times more than original
  specification
• Parity quality beam imposed optics constraints
• Hall A hypernuclear experiment required
• Energy spread lt 3x10-5
• Scheduled in parallel with G0
• HAPPEx-II and HAPPEx-He required
• Tightest helicity correlated asymmetries ever
• Position asymmetries lt 2 nm
• Energy asymmetry lt 0.6 ppm

13
G0 Parity Quality Beam
Total of 744 hours (103 Coulombs) of parity
quality beam
Beam Parameter Achieved (IN-OUT)/2 Specs
Charge asymmetry -0.14 0.32 ppm 1 ppm
x position differences 3 4 nm 20 nm
y position differences 4 4 nm 20 nm
x angle differences 1 1 nrad 2 nrad
y angle differences 1.5 1 nrad 2 nrad
Energy differences 29 4 eV 75 eV
All parity quality specs have been achieved!!
14
Hypernuclear Experiment Energy Spread
Data from April 21-29
15
HAPPEX-II
Electron only Photon only
Preliminary
New superlattice photocathode Polarization
gt85 Figure of Merit improves by 30 (over
strained-layer cathode)
CASA and EGG have worked closely with HAPPEX to
meet stringent requirements on helicity-correlated
position differences. After correcting early
problems at source, the ability to meet
helicity-correlated specifications was
demonstrated.
Dx (nanometers)
slug number
16
DOE Metrics for FY03

• Metrics for FY03 were excellent
• Availability for multi-Hall Physics operation
  not as good as our Users would like, but
  performance better than DOE goal

17
DOE Metrics for October July FY04
Hall A septum
Post-hurricane maintenance extremely effective
18
Preparing for Upcoming Challenges

• EnergyParityPolarization

19
Energy Outlook for FY04/05

• Scheduled to deliver 5.75 GeV, 100 kW beams in
  September 04
• Hurricane reduced accelerating voltage by 40
  MV/turn, 200MeV from top beam energy
• Predicted RF trip rate will be high 15/hour
• Will make operation of accelerator difficult
• Required to reach goals of experimental program
• Compromise accepted by Users
• Expect RF trip rate to improve when new 12 GeV
  prototype cryomodule replaces NL11 (operational
  by July 05)
• RF trip rate at 5.75 GeV will be acceptable
  10/hour
• Refurbishment of existing cryomodules would
  provide 6 GeV operation by July 06 with
  acceptable trip rate (10/hr)

20
Parity Violation Experiments at CEBAF

• Helicity-correlated asymmetry specifications
• Achieved for G0 4 4
  nm -0.14 0.32 ppm

Experiment Physics Asymmetry Max run-average helicity correlated Position Asymmetry Max run-average helicity correlated Current Asymmetry
HAPPEX-I 13 ppm 10 nm 1.0 ppm
G0 2 to 50 ppm 20 nm 1.0 ppm
HAPPEX-He 8 ppm 3 nm 0.6 ppm
HAPPEX-II 1.3 ppm 2 nm 0.6 ppm
Lead 0.5 ppm 1 nm 0.1 ppm
Qweak 0.3 ppm 20 nm 0.1 ppm
21
Superlattice Cathode

• Polarization 87 (recent User measurement)
• Typical polarization from traditional strained
  layer material 75
• Quantum Efficiency 1
• Typical QE of traditional strained layer material
  0.2
• Analyzing power 4
• Factor 3 better than strained-layer material in
  the lab
• Smaller intensity and position asymmetries on
  beam
• Improvement not yet seen in experimental data
• Installed on Accelerator 5/17/04
• Successfully operated for experimental program
  (HAPPEx)
• Lifetime was not good attributed to bad vacuum
• NEG pumps replaced in present accelerator
  shutdown
• Will be standard for all experiments
• Matt Poelker and Maud Baylac (Injector)

22
New Laser Clean Room for Injector

• Insert Photo

23
Path Forward New Operations Vision

24
Drivers for Change

• Our accelerator operations are second to none
• Biennial Workshop on Accelerator Operations
  initiated by JLab
• Our Control System is one of the worlds best
  managed
• Karen White is regularly invited to lecture on
  managing software
• But, we believe in continuous improvement
  (really)
• Four main drivers for change
• Main Control Room (MCC) needed renovating
• Aging flooring, improve air conditioning, bad
  ergonomics, needed better integration of ODH
  alarms, fire alarms and access controls
• ORACLE database available, needed EPICS
  integration
• Full accelerator model will be available soon and
  we should plan for it
• Must prepare to commission and operate 12 GeV
• Goal use these drivers to revamp operations
  processes

25
MCC Upgrade

• Layout modified to provide
• Crew Chief oversight of operators
• Station for Program Deputy accessible to support
  staff
• Responsible for program oversight for two-week
  period
• Stations for Principle Investigators
• Direct special machine set-ups and beam studies
• Improved teaching environment for operators
• Discussion area with mirrored computer screen
• Existing tall racks replaced with desk height
  work stations
• Multiple small monitors replaced with few large
  screens
• Better visibility of access controls (personnel
  safety system)
• Integrated beam diagnostics displays
• Managed by Mike Spata and Tom Oren (Operations)

26
Old MCC

27
New MCC (three weeks later)
28
Operations Vision

• Primary focus are beams meeting User
  requirements?
• Secondary focus is each region performing
  correctly?
• Provides common structure for thinking about
  accelerator operations, database, accelerator
  model, HLA, new installations
• Hierarchy based on the accelerator layout
• Usual focus on kinds of element (magnets,
  steering, RF) - WBS
• Change to functional segmentation system
  derived from beam-based set-up
• Highest level derived from User requirements
• Halls, energies, currents, polarizations, beam
  specifications
• Increases focus on diagnostics to ensure that
  beam meets specifications

29
Highest Hierarchical Level

• Defined standard set of beam specifications for
  Users
• User may negotiate tighter specs when proposing
  experiment (TAC)
• Experiment schedule defines which experiments are
  running
• User requirements are known import requirements
  from database
• Use these requirements to configure the
  accelerator
• Derive set-points for the machine set-up
• Energy, current, polarization . . . . .
• Integrate beam specs with instrumentation to
  monitor compliance
• Energy spread, spot size, helicity-correlated
  effects . . . . .
• Highest level display shows if beam
  specifications are being met, and if not, which
  parameters are out of tolerance
• Managed by Hari Areti (Experiment Coordinator)

30
Beam Specifications

• DC Beam Properties
• There are also AC Beam Properties and
  Helicity-correlated Beam Properties

31
Example

• Experiment beam request
• Experimental requirements

32
Diagnostics

• Each beam specification is mapped to at least one
  diagnostic
• Diagnostics are of three main types
• Run-time monitors that function at all times
• BPMs, Synchrotron light monitors, OTR, beam loss
  monitors, experiment detectors, Compton
  back-scattering
• Invasive monitors that cannot take full power
• Screens, Harps
• Infrequent monitors that require special set-up
• Moeller and Mott measurements, current and energy
  calibrations
• Long term goal is to monitor all beam
  specifications to required accuracy
  non-invasively over complete range of operating
  conditions
• Diagnostics must be integrated with software
  packages and tightly coupled to User-specific
  beam specifications
• Managed by Arne Freyberger (CASA)

33
Database

• Master copy of all information will be held in a
  database
• Authoritative source
• All other instances will reference database to
  obtain current value
• Vital for maintaining control over machine
  changes
• Information will be assigned to one of two
  databases, depending on the frequency of change
• We already have a dynamic, run-time database
  EPICS
• Adding master database for static and slowly
  changing data - ORACLE
• Databases will eventually manage all accelerator
  data
• Database will be the information source for
  everyone
• Engineering support groups, operations, controls
• Managed by Theo Larrieu (Controls)

34
Impact on Control System

• Robustness requires nested checks at all levels
  of software
• Example of making tools robust
• BPM passes self-check
• Feedback system uses model to determine best
  corrector, BPM configuration based on Optics
• System measures BPM response to corrector kicks
• Compare corrector-BPM response to model
• Downstream elements monitored to ensure feedback
  system is performing desired function
• Providing all necessary hooks requires global
  re-examination of Control System at every level
• Device drivers, low-level applications (Matt
  Bickley)
• High level applications, communication protocols
  (Brian Bevins)
• Managed by Karen White (Controls)

35
Optics Model-Database Relationship

• Model obtains input from
• ORACLE
• Component layout derived from Survey group
• Component specifications from Engineering Support
  Groups
• Impacts all Support Groups
• Vehicle for configuration control
• Global settings
• Configured from User Requirements
• Off-line optics calculation by CASA
• Result goes into Oracle database
• Set points calculated for dipoles, quadrupoles,
  RF
• Model server output is available to all high
  level applications
• Eventually, all high level applications will be
  model driven
• Managed by Yves Roblin (Controls)

36
Optics Model Improvements

• Model requires accurate knowledge of magnets over
  wide energy range
• We have 2000 magnets, not all properly
  characterized
• Uncertainty due to dipole gradients from remanant
  fields
• Additional uncertainty from orbit-related
  focusing errors due to badly characterized gold
  orbit
• Diagnostics added in spreaders and recombiners
• Beam-based measurements being used to measure
  errors
• Requires special optics (weak focusing)
• Data taken over last year, dedicated period at
  end of last run
• Evaluated during the summer accelerator down
• Will be used for setting up the machine in
  September
• Managed by Mike Tiefenback (CASA) and Tommy Hiatt
  (Engineering)

37
Implementation Status

• MCC refurbishment complete (MCC visit during
  Tour)
• Planning, implementation and result are fantastic
  success
• Requirements Document for Control System being
  written
• Executive Summary complete
• Ensures coherency of Vision across Division
• Some aspects already implemented
• Model under active development
• Guiding principles of the Vision will be
  integrated into new and upgraded software for
  years to come
• Expect positive impact on operations within six
  months
• Changes the way we do business for years to come
• Prepares operations for commissioning and
  operating 12 GeV

38
Summary

• FY03 operations were excellent, FY04 were
  outstanding
• G0, an incredibly difficult experiment, got more
  data than requested, beam exceeded all
  specifications
• Hypernuclear experiment received beam with
  outstanding energy spread run average 2.2 X
  10-5
• Even more impressive as experiments ran in
  parallel
• HAPPEx tight parity quality specs achieved
• Availability for Physics much improved since
  hurricane due to additional maintenance that was
  performed
• New Vision will improve Operations in coming
  months
• Motivates and energizes multiple Groups
• Prepares for commissioning and operating 12 GeV
  Upgrade