Goals and plans for Grid Data Warehouse

1
Goals and plans forGrid Data Warehouse

• Guy Rixon
• AstroGrid Consortium meeting
• 3-4 November 2003

2
AstroGrid goals

• Data warehouse
• Concentrate catalogues in one DB
• Therefore solve distributed-join problem
• Allow users to add tables to DW.
• Exploit UK e-Science Grid.
• OGSI at arms length
• Exploit OGSI.
• Dont rely on OGSI in critical places
• Use OGSI/OGSA in leaf node services.
• See separate grid-tech talk at this meeting.

3
GDW goals

• Use pure grid sites and resources
• Dont require astro software at grid sites.
• Use OGSA-DAI as remote interface to DB.
• Use job-manager services?
• Translate VObs ? grid in A/G web-service.
• Support basic DW operations
• Query preloaded catalogues
• Add user catalogues
• Allow query results to stay in DB for later
  queries

4
Design

• See wiki web, keyword GdwDesign.
• See http//astrogrid.ast.cam.ac.uk/javadoc/
• but following detailed material hadnt made it
  to the web at the time of presentation model
  inside IDE. (Will probably be on the web when
  this presentation is archived.)

5
Iteration 4 planning (1)

• Pre-loaded data-sets
• 2MASS catalogues
• XMM1
• USNOB1
• FIRST catalogue
• INT-WFC catalogues
• FIRST is first because its small.

6
Iteration 4 planning (2)

• Do use cases SynchronousQuery and
  AsynchronousQuery for one catalogue by middle I4.
• Do above for all data-sets by end I4.
• Try to do either LoadEndUserDataSet or
  AsynchronousQueryIntoTable by end I4.

7
Iteration 4 planning (3)

• Develop at Cambridge.
• Deploy copy to Leicester.
• If possible, deploy copy of back-end to UK
  e-Science grid, possibly at NeSC/EPCC.