Enterprise CRM Solution

1
Observing Process Project Development
Plan GBT e2e Software Review May 3, 2005 Nicole
Radziwill (nradziwi_at_nrao.edu) Amy Shelton
(ashelton_at_nrao.edu)
2
Goals

• To streamline the observing process
• Enable automated dynamic scheduling and remote
  observing
• Make the GBT easier to use for both visiting
  observers and staff astronomers.
• But now we recognize that a focus on usability
  can distract us from meeting primary two goals of
  dynamic scheduling and remote observing
  sometimes, we may have to make the system a
  little more difficult to use for some people, to
  enable us to make the system easier to use for
  all people. This is very subjective!
• Reduce setup times significantly
• Reduce the reliance of visiting observers on
  scientific staff
• Includes the ability to
• Define observations in advance of observing as
  Scheduling Blocks,
• Execute and monitor those observations,
• Evaluate comprehensive status information during
  execution, and
• Get easy access to any documentation you need to
  do all of the above
• Improving quick look data display now being
  worked as part of the Data Handling project.

3
2004 vs 2005

• 2004
• Many disparate software applications used by
  astronomers, much duplication of code
• GBT Observe (GO), a GUI application written in
  glish
• Control Library for Engineers and Operators
  (CLEO), a Tcl/tk GUI
• Ygor, a distributed object-oriented telescope
  control system in C
• IARDS, an aips quick look real time data
  display
• DISH, the aips single dish data reduction
  environment
• Glens Spectrometer Tool Franks pulsar GUI and
  pulsar monitors
• Difficult to test functionality from end to end
  through all these systems
• Difficult to determine which application should
  host a particular function no clearly defined
  structure by which one would know where to
  develop a particular new requirements.
• Operational problems were common! (Ex. An
  individuals application using up too many
  connections to devices, causing cascading errors
  that require extensive troubleshooting and
  operational support)
• 2005
• Scientists working to transition users to
  Astronomers Integrated Desktop (Astrid), a
  single application that enables
• Creation, validation, submission and monitoring
  of Scheduling Blocks
• Interactive commands via APIs while Scheduling
  Blocks are running
• Quick-Look Display with offline playback, access
  to status and documentation
• Unified collection of applications designed with
  remote observing in mind

4
High-Level Architecture
GUIs
Application
Application Component
0..n
uses
HLAPIs
Expert User
Programmer
uses
uses
LLAPIs
Control System
Observed Data (in an EDF)
Real-Time Data (streaming)
5
Project Description
SB Executor
Quick Look Data Display
GBT OT
Configuration API
Status Screen
uses
Observing API
BUILD
Balancing API
EDIT
Console Windows
RUN
MONITOR
Easy Access to Documentation, Help
Data Capture
Integrated Desktop
Scheduling Blocks Execution Log File
Annotated Index of Observation
Observation Management Database (e2e Project
Database)
Note that APIs are very telescope-specific, yet
are used to abstract the observation into terms
not specific to the telescope
6
Development Phases

• ?Configuration 10/1/03 to 3/31/04 6 mo
  elapsed
• Configuration API docs at http//wiki.gb.nrao.edu/
  bin/view/Knowledge/ConfigurationAPI
• Configuring within a Scheduling Block
  http//wiki.gb.nrao.edu/bin/view/Software/Observin
  gToolsUsageDefineConfiguration
• ? Observing 11/15/03 to 9/17/04 6 mo elapsed
• Observing API docs at http//wiki.gb.nrao.edu/bin/
  view/Observing/ScanTypes and http//wiki.gb.nrao.e
  du/bin/view/Software/ObservingDirectives
• ? Scheduling Blocks 5/17/04 to 9/17/04 3 mo
  elapsed
• Docs that describe observing on the GBT using
  Scheduling Blocks available at
  http//wiki.gb.nrao.edu/bin/view/Software/Observin
  gTools
• Single SBs are now being executed on the
  telescope regularly staff scientists preparing
  to move visiting observers to this paradigm over
  the summer (specific date TBD by astronomers)
• Full Functionality / Remote Observing C3 to C8
  2005
• Includes resolving known issues, implementing
  security (Online/Offline modes), improving
  process for SB submission and FIFO queue
  management, enabling offline validation
• Priority of remote execution of single SBs raised
  because of operational benefits it can provide
• Science Program Level Integration 2006
• Still need to identify best way to do this!
  Impacts strategy for Scheduling Block Builder GUI
• Want to adopt a technology for this, not develop
  from scratch. Boundary must be clearly
  identified.

7
Scheduling Blocks

• APIs are used in the execution of Scheduling
  Blocks
• Has identical entities/attributes as the ALMA
  Scheduling Block, with the possible exception of
  setup/configuration
• Although these will be eventually archived in XML
  with the observers calibrated data products,
  right now we simply aim to store them in plain
  text, and have chosen a format similar to the
  user preferences file
• Uses the Configuration API and Observing API to
  execute complete observations on the telescope
  Scheduling Block Executor (implemented within
  Turtle) controls the flow of information

SB EXECUTOR
Configuration API
Balancing API
Observing API
8
Full Functionality (2005)

• SB-based observation management system released
  on 9/17/04, but because technology transfer was
  not adequately planned for, adoption by support
  staff delayed until C2 2005.
• It covered all GBT Standard Observing Modes
  except Radar, or observations with fast-moving
  near Earth objects (comets/satellites)
• Pulsar observing and some enhancements for spigot
  ease of use were scheduled and completed as
  maintenance activities for Fall 2004 this
  included additions for configuration and
  balancing.
• Required capabilities to be developed include
• Low-Bandwidth Remote Observing Solution
• High-Bandwidth Remote Observing Solution
• Support for generating ephemeris tables and
  observing fast-moving sources like comets and
  satellites
• Be able to use and define source catalogs
• A Balancing API to eliminate user dependence on
  CLEO, which requires expert level knowledge

9
Program Level Integration (2006)

• Work has focused on building SBs and executing
  them by populating a queue and letting those SBs
  execute first-in-first-out (FIFO), but we will
  still need solutions for
• Queue Management, which integrates constraints on
  the SBs (such as opacity) and puts the management
  of observation execution more in the hands of the
  operators
• Building the logical relationships between SBs to
  create ObsUnitSets that fulfill scientific intent
• Kicking off pipeline processing (this should be
  done at the Program level since the products of
  multiple SBs may be required to yield data
  products e.g. maps)
• We may already have the tools to implement both
  remote observing solutions, however, our
  constraints are specifications and science staff
  time

10
Development Plan Presented in August 2004
1 CONFIGURATION API
2 OBSERVING API
QUEUE MGMT
TEST
3 SBs
BALANCING API
SOURCE CATALOGS
CONSTRAINTS
INTEG W/PIPELINE
EPHEMERIS
COMPLETED BY END 2004
PHASE 4 TO BE SCHEDULED (2005)
PHASE 5 TO BE SCHEDULED (2006)
11
Development Plan Status as of May 2005
1 CONFIGURATION API
Single SB Observing
Science Program Mgmt
KNOWN ISSUES
2 OBSERVING API
QUEUE MGMT
BALANCING API
TEST
3 SBs
SOURCE CATALOGS
CONSTRAINTS
INTEG W/PIPELINE
EPHEMERIS
COMPLETED BY END 2004
PHASE 4 IN PROGRESS/PLANNED (2005)
PHASE 5 TO BE SCHEDULED (2006)
12
Relationship to ALMA and EVLA

• All development has been done with strong
  reliance on
• Design documentation from ALMA PDR (March 2003)
• ALMA SB Definitions (January 2004)
• NRAO e2e Archive Model (May 2004)
• NRAO e2e Observing Model (May 2004)
• NRAO e2e Project Model (May 2004)
• Basic functionality of ALMA OT has been
  duplicated
• Approximately one weeks worth of effort less
  effort than customizing ALMA OT to use GBT APIs
• We didnt want things like perspectives and
  were concerned that we are not ready to support
  ACS in -Lite or -Lighter forms
• We could adopt the ALMA OT and Queue Management
  system in the future if this is desired in any
  case, staff astronomers and observers will be
  able to observe according to a common look and
  feel with ALMA upon completion of the testing
  period (end 2004)

13
Risks

• No Project Scientist for Data Display no written
  specifications or singular point of
  responsibility yet for what should be contained
  in spectral line data display
• Although we do have the capability now to display
  waterfall diagrams, which has been mentioned as a
  desirable enhancement
• Once specified, development could be done in a
  reasonably short amount of time if approved by
  the GB Project Planning Committee
• Data display work has been driven by PTCS needs
  so far
• Adoption of SB-based system by staff astronomers.
  Writing down observation plans a priori in the
  form of SBs may require a shift in behavior for
  some staff.
• Adoption of OT (which does not look like former
  GO application!)
• Staff scientists desire greater ease of use (e.g.
  variable recognition, case-insensitivity) this
  is a high cost/benefit and programming efforts
  may be better spent elsewhere
• Staff scientists not fully plugged into ALMA work
  or e2e directions.

14
Still Need

• A whole lot more balancing

15

• We are currently operating in two modes
• Scheduling Block based observing recommended for
  all observing modes except radar moving sources
  (planets, comets, etc)