EVLA Advisory Committee Meeting

1
EVLA Computing

• Software Overview

2
Contents

• Organization and staffing
• Funding Profile
• Requirements testing, status and plans
• Selected subsystems current status
• Future plans
• Major risks

3
Organization and Staffing

• EVLA computing division
• Head 0.8 FTE
• MC 9 FTEs
• 8 FTEs on EVLA (4.3 EVLA, 3.7 Contributed Effort)
• 1 FTE on VLA/VLBA operations
• In original EVLA budget
• E2e 6 FTEs
• 4.3 FTEs on EVLA (3.2 EVLA, 1.1 Contributed
  Effort)
• Soon to decrease by one to 3.3 (2.2 EVLA, 1.1
  Contributed effort)
• 1.7 other
• Not in original EVLA budget
• In March, 2006 MC and E2e groups combined

4
Organization and Staffing (continued)

• Systems Engineer for Software (0.5 FTE)
• Reports to EVLA project manager
• Project Scientist for Software (0.5 FTE)
• Reports to EVLA project scientist
• Science Software Group
• Managed together with ALMA through Interferometry
  Software Division
• 8 FTEs, 2.65 FTEs EVLA

5
Organization and Staffing (continued)

• Staffing developments in 2005/2006
• Butler replaces Clark as EVLA Project Engineer
  for Software
• Frail steps down as EVLA Project Scientist for
  Software
• Ye (e2e/Proposal Tool) moves to ALMA, position
  filled by Loveland
• Morgan (MC/Correlator Backend) leaves NRAO,
  position filled by Pokorny
• Two new e2e positions created out of EVA
  contingency
• Harland High Level Architecture
• Truitt Observation Preparation Tool
• Waters (e2e/HLA) to move to Science Software in
  May 2006 will not be replaced for now due to
  uncertainty in funding

6
Funding profile (1) MC
7
Funding Profile (2)E2e
8
Science Domain/e2e requirements

• Scientific requirements for all subsystems
  developed in 2003 2004, reprioritized in 2005
• Each requirements is assigned priority (1, 2, 3)
  
• 1 essential, 2 important, 3 desirable
• and timescale (A E)
• A prototype WIDAR, B Limited Production WIDAR,
  C development of more user-friendly tools, D
  WIDAR installed, major modes available, E all
  antennas converted, full science operations
• Requirements were reorganized according to
  timescale
• For each priority 1 requirement, time to
  complete was estimated
• Plan to deliver all priority 1 requirements by
  2010 developed

9
Required e2e staffing
10
E2e staffing required vs. available

• FTE-years required 25.8
• FTE-years budgeted 12.7 (EVLA) 4.9
  (contributed effort) 17.6
• Shortfall 8.2 FTE-years, or 2 FTEs through
  remainder of project
• Assumes contributed effort funding continues at
  current level
• New NRAO e2e operations project has the potential
  to alleviate this shortfall
• Further de-scope would require reprioritizing
  priority 1 requirements
• Barriers to non-radio astronomers remain
• Limit products archived affects ease of access

11
Management/target tracking

• MC weekly general EVLA coordination meeting
• With Electronics, Engineering Services, etc
• E2e weekly Science Domain meeting
• High-level Architecture
• Subsystems
• Proposal preparation/submission
• Observation preparation
• Scheduling
• Archive
• Pipeline
• Post-processing bi-weekly meetings
• NAUG NRAO AIPS Users Group
• NAWG NRAO Algorithms Working Group
• Monthly Division Meeting during which a topic of
  interest is presented
• Weekly/bi-weekly code reviews/walk-throughs

12
Subsystems High level architecture

• Builds on Overall Design presented in 2004, after
  which effort stalled because of lack of manpower
• Activity resumed as soon as two new e2e positions
  were filled
• Largely concurrent with Observation Preparation
• Project data model
• Scheduling block definition
• MC design document developed in parallel

13
Subsystems MC

• 3 EVLA antennas in array
• MIB module programming
• interim executor
• operator interface
• Control/Monitor processor archives EVLA monitor
  data, will soon control VLA antennas
• monitor data archive
• WIDAR
• Correlator board drivers complete and ready for
  testing
• Station and baseline board test screen GUIs
  developed
• Correlator backend ready to support prototype
  testing

14
Subsystems Proposal Preparation

• Goal one proposal tool for all NRAO instruments
• Developed in parallel with NRAO-wide user
  database
• May 10, 2005 1st release, June GBT deadline
• September 10, 2005 2nd release, October GBT
  deadline, VLA tests
• January 10, 2006 3rd release, February GBT and
  VLA deadline
• Optional for VLA 67 opts for new tool
• May 10, 2006
• Uses new code base allowing code sharing with
  other subsystems
• Will be only way to submit proposals for both GBT
  and VLA
• September 10, 2006 planned release for October
  deadline
• bug fixing, improve handling tools
• No support for other instruments
• EVLA support will be implemented much later when
  required

15
Subsystems Observation Preparation

• Started as soon as two new e2e positions were
  filled
• Shares objects with proposal preparation tool
• Timeline
• Apr06 framework with minimal functionality
• Jul06 Add VLA calibrator database access,
  initial spectral setup
• Oct06 Full calibrator setup, more observation
  setup
• Jan07 VLA mostly supported. Some validation/PB
  creation
• Apr07 Beginning prototype WIDAR support
• Jul07 VLA fully supported prototype WIDAR
  mostly supported
• Oct07 Prototype WIDAR fully supported

16
Subsystems Archive

• Monitor data archive functional and in active
  further development now a MC item
• Visibility archive
• Will be using ALMA NGAS software/hardware
• NGAS Software to be delivered by ALMA archive IPT
• NGAS Hardware has been purchased
• Plan to have rudimentary archiving in place for
  WIDAR prototype
• Agreement on a common Science Data Model between
  ALMA and EVLA in progress

17
Post-processing

• Science Software group co-managed with ALMA
• CASA/AIPS Infrastructure and most (2/3) of
  applications in common
• Main differences
• Wide field, wide-bandwidth imaging
• Calibration
• RFI
• EVLA testing
• Summer 2005 wide field images w-projection
• Spring 2006 User Interface working group (with
  ALMA)
• Fall 2006 UI, basic calibration, full beam,
  full Stokes imaging
• Q2 2007 RFI, Auto-flagging

18
Collaboration with ALMA

• Subsystems
• Proposal preparation EVLA has working version,
  agreement on common project model with ALMA in
  progress
• Observation Preparation. More difficult because
  of more instrument dependencies also requires
  agreement on common project model
• Visibility data archive sharing will require
  agreement on common science data model, which we
  are actively working on
• Pipeline Not among priority 1 items for EVLA
  we expect heavy borrowing from ALMA
• Post-processing both ALMA and EVLA have adopted
  CASA as their default data post-processing system

19
Plans for next 12 months

• New release proposal tool every four months
• Complete HLA, e2e PDR early fall 2006
• EVLA tests CASA fall 2006
• MC CDR late fall 2006
• Modcomp replacement 1st half 2007
• Observation preparation tool in time for WIDAR
  prototype
• Visibility archive in time for WIDAR prototype
• Start Observation Scheduling

20
Risks (1)

• MC
• Unpredictable hardware redesigns add extra cycles
  of development and testing. Mitigate by
  contracting work out, which does not come cheap
• Key responsibilities shared by a few expert staff
  members who are oversubscribed. Mitigate by
  training other staff where possible
• E2e
• Insufficient staffing. Mitigate by requesting
  staffing increase or further de-scope
• Reaching agreement with ALMA on data models in
  order to leverage parallel developments.
  Negotiate with much bigger partner with different
  priorities and deadlines can be a challenge

21
Risks (2)

• Post-processing
• Present acceptable user interface to user.
  Mitigate by ongoing user interface testing
• Acceptance of CASA by user community. Mitigate
  by substantial internal NRAO testing and
  documentation before external release
• Reliance on historic increases in hardware
  capability. Mitigate by looking into parallel
  processing using e.g. clusters, and increase code
  efficiency
• Algorithmic development hard to plan ahead as
  some of this is more research than software
  development. We attempt to mitigate this by
  employing students/postdocs