VISTA Data Flow System

1
VISTA Data Flow System

• for WFCAM VISTA data2nd overview for AGOC

2
Some events since 1st AGOC visit

• Mar 28 1st AGOC visit
• Apr 04 VISTA Project Board moved pipeline PDR
  coincident with camera s/w FDR (Nov)
• Apr 15/16 v1/2 VDFS Archive CDR (wfcam)
• May 05 v1/2 VDFS Archive CDR panel report
  received
• May 28 v1/2 VDFS Archive Software Architecture
  Design Document
• Jun 06 VISTA Project Board
• Jun 12/13 Workshop on Observing Strategy (wfcam)
• Jul 04 VDFS Q2 reviewed, Q3 plan set
• Jul 04 Jim Gray Alex Szalay visit archive
• Aug 01 VISTA Project Board
• Sep 17-19 VISTA Camera pre-FDR workshop
• Oct 01 VDFS Q3 reviewed, Q4 plan set
• Oct 02 VISTA Project Board

3
v1Archive CDR conclusions

• VDFSv1 Science Archive CDR was held 15/16 Apr.
  The main task of the Panel was to determine
  whether the Science Archive design put forward by
  WFAU members met eight (seven!) criteria, their
  conclusions were
• 1 The end product is acceptable Yes.
• 2 The project as conceived is feasible The Panel
  had major concerns regarding the project schedule
  and risk assessment. Both were far too
  optimistic, and timescales could easily be scaled
  up by a factor of two.
• 3 Detailed plans for building the complete system
  are in place A plan is in place, but important
  areas need to be better defined and planned. One
  important example is software architecture and
  software design procedures.

4
V1 Archive CDR conclusions 2

• 4 The budget and staffing resources are
  sufficient for the project and commensurate with
  the estimated time to completion Staffing is a
  concern especially after deployment. Too little
  staff was allocated for v1 operations.
• 5 A reasonable estimate of the risk has been
  made, and whether there are useful de-scope
  options Risk assessment needs to be planned in
  greater detail. No de-scoping options were
  presented although the Panel strongly felt that
  the hardware and software needed to produce
  satisfactory pixel data and source catalogs and
  curate them should be in place before any
  resources were allocated to the ingestion of
  external catalogs for example.
• 6 External interfaces are well controlled Yes.
• 7 The path from v1 to v2 is understood and
  feasible The path from v1 to v2 will be easier
  if sufficient attention is paid to proper
  software architecture and design procedures that
  will maintain portable code and clean interfaces
  between subsystems.

5
AGOC visit 28 Mar 2003 pt1/5

• Conclusions
• 1.     The Committee recommended the following
• a.      Further VISTA DFS Advisory Committee
  members should be appointed who were potential
  data users who were not linked to the delivery of
  the project.
• On further reflection it was felt there was no
  need for a formal AC as individual contacts
  served the advice functions needed by the team.
• The question of a future role for a Users
  Committee remains open with PPARC for the future.

6
1st AGOC visit 28 Mar 2003 pt2/5

• b.     Appropriate procedures appeared to be in
  place for the proper engagement with ESO over the
  ESO deliverable aspects of the project.
• Apr 04 VISTA Project Board moved pipeline PDR to
  be coincident with camera s/w FDR (Nov)
• Sep 11 ESO asked for DFS PDR after camera FDR

7
1st AGOC visit 28 Mar 2003 pt3/5

• c.     A regular external oversight activity was
  required. At present this function fell to the
  VISTA Board. The VISTA Board appeared to be
  focusing most of its efforts on other aspects of
  the VISTA project, to the extent that the DFS
  project was not getting the level of scrutiny and
  oversight that it needed. PPARC should institute
  a more effective oversight structure for the DFS
  project.
• PPARC wish to continue current oversight by VISTA
  Board supplemented 6 monthly this panel, at least
  until next tranche of e-science money.

8
1st AGOC visit 28 Mar 2003 pt4/5

• d.      The unfunded future effort issue would
  need to be resolved through appropriate bids to
  the current PPARC e-science call.
• May 31 VEGA proposal submitted
• Jul 14/15 PPRP passes 2.3m VDFS part of VEGA
• Sep 30 GSC letter re VDFS/VEGA
• Oct 14 telecon Oct 28 discussion meeting
• Jan 2004 GSC

9
1st AGOC visit 28 Mar 2003 pt5/5

• e.     Project management procedures appeared to
  be emerging but these had only been instituted
  recently. A specific set of milestones and
  interim deliverables was still required, as was a
  process for measuring progress against them.
  Future oversight would need to ensure that these
  procedures became fully operational and
  effective.

10
Changes to External Milestones (top level)

• WFCAM DFS
• 04 Jan Commissioning - no data camera
  hardware problems
• 04 Apr Operations - no data camera hardware
  problems
• 05 Jan v2 pipeline archive
• 05 Apr 1st year science out but wont be a
  year of it by then!
• VISTA DFS
• 03 July Preliminary Design Review moved by
  Board, then by ESO -- new date being negotiated
• 03 Nov Final Design Review corresponding
  slippage
• 05 Dec Instrument in Chile
• 06 May Instrument and onsite DFS Commissioning
• 06 Aug Instrument Acceptance gt Operations

11
Top Level Milestones to end 2004
12
Top Level Milestones 2005-8
13
Working of Management Process

• Project Wide Mgt team Local Mgt? added Hambly
  Hodgkin to monthly review team
• Deliverables detailed?
• Work Packages ( their managers) ?
• Sub tasks defined ( implementers)
• Long range plan?
• Risk analysis ? ? but not yet reviewed
• Quarterly detailed plan deliverables?
• Report against in future
• Review past and make next
• Monthly Mgt team (tele) progress meetings ?
• Weekly/fortnightly (local) team progress
  meetings. ? Minutes actions distributed. ?

14
Deliverables - metrics
Key
100 complete 50 almost finished
10 started 0 not started
15
Changes to Management Structure
GSC
VISTA Project Scientist now based in Cambs
VISTA Board
6 monthly reports??
Two Monthly reports
VDFS Advisory Committee
PI Management Team
Weekly reports
Fortnightly reports
CASU manager
WFAU Manager
report
CASU WPMs
WFAU WPMs
manage
advice
16
VDFS Advisory Committee

• Membership (provisional)
• Project Leader (chair)
• VISTA Project Office s/w WPM, UKIRT Director
• UKIDSS VISTA Project Scientists
• Astrogrid Project Scientist
• Archive/pipeline implementer (from Leicester?)
• CASU/WFAU co-Is and managers in attendance
• Purpose
• Provide advice to Project from customers and
  external experts