Experiences of the Reading eScience Centre

1
Experiences of the Readinge-Science Centre

• Jon Blower
• ReSC Technical Director
• jdb_at_mail.nerc-essc.ac.uk

2
Overview

• What is the ReSC?
• What have we done lately?
• What can we share with others?

3
The ReSC

• Formed in November 2003 as Centre of Excellence
  for environmental e-Science
• Hosted by NERC Environmental Systems Science
  Centre
• Focus on research projects, not training or
  outreach
• Hence complementary to NIEeS
• Act as local e-Science focus in Reading
• Involve industry and government agencies
• Met Office, ECMWF, Environment Agency, BMT, IBM,
  Barrodale, Lost Wax, Vita Nuova
• Build software for the community
• Support others who wish to start e-Science
  projects

4
Who we are
Keith Haines and Rachel Harrison (co-directors)

• Many Associates including
• Chunlei Liu (ESSC)
• Ian Bland (Comp Sci, networking)
• Steve Gough (IT Services)
• Kevin Hodges (ESSC)
• Kecheng Liu (Comp Sci, Informatics)
• Two e-Science PhD students

Jon Blower Technical Director
Adit Santokhee Data Manager
Dan Bretherton Program Manager
5
ReSC projects
6
GADS Web Service
External applications

• GADS (Grid Access Data Service) is a Web Service
  for accessing environmental data
• gridded data only (numerical model output,
  satellite data)
• serves 3TB of atmospheric and oceanographic data
• Hides details of data store from users
• Users can download data in a number of popular
  formats (NetCDF, GRIB, HDF)
• Data originate from UK Met Office, Hadley Centre,
  European Centre, National Oceanography Centre
• Being re-engineered in the DEWS project to comply
  with international standards

WS interface
GADS library
DATA
META DATA
7
GODIVA web portal

• The GODIVA Web portal provides a graphical
  interface to data at ESSC
• Uses GADS to query and extract data sets
• Users can make simple visualisations
• pictures and movies
• 30 access per day from all over the world

http//www.nerc-essc.ac.uk/godiva
8
GADS application Search and Rescue

• British Maritime Technology produce software
  (SARIS) to help the Coastguard with Search and
  Rescue
• Predicts drift patterns of people and objects
  that have fallen overboard
• This significantly cuts the time to rescue
• Have worked with BMT to produce prototype that
  uses live Met Office data from GADS to improve
  its predictions
• Uses forecasts of surface winds and surface
  currents
• Can also be applied to oil spills

9
All very well but

• GADS is not compliant with any standards
• but it is simple
• GADS does not deal well with very large data
  volumes
• Needs to handle data extractions that take
  significant time
• GADS is not very secure
• GODIVA web site provides some security
• GODIVA web portal is powerful but we would like
  it to be more interactive

10
OGC Web Services

• (roughly in decreasing order of maturity)
• Used by NERC DataGrid
• Services can be composed to create distributed
  geospatial applications

11
Delivering Environmental Web Services (DEWS)

• 2.2M project, 50-funded by Department of Trade
  and Industry
• Goal is to provide access to UK Met Office data
  securely over the Internet using Web Services
• Develop applications and markets for these data
• Marine Search and Rescue
• Health Predict occurrences of Chronic
  Obstructive Pulmonary Disease (COPD, a type of
  lung disease)
• GADS will be re-engineered as an OGC Web Coverage
  Server
• In line with many other projects on geospatial
  data access
• Need to modify WCS spec in some areas large data
  sets, NetCDF data format
• Will use NERC DataGrid security model
• Collaboration between Met Office, British
  Atmospheric Data Centre, ReSC, Lost Wax and IBM

12
What have we learned so far?

• Exposing data through standard interfaces has big
  advantages
• Interoperability
• Software re-use
• But the standards dont always do exactly what we
  want
• We would advise projects to look at the OGC
  standards
• Bear in mind that some are in state of flux
• Strongly recommend use of NetCDF as data format
• CF-compliant

13
The next generation?
14
Google Maps

• Web-based widget for viewing map data
• or any images in fact
• Like Streetmap, Multimap etc but much slicker
• draggable map
• fast response time
• Can mark locations

15
Google Earth

• Mapping for the masses
• According to Nature
• Desktop application (Windows and Mac) for
  displaying geographical data
• Satellite images
• Earthquake locations
• Live data!
• All on a 3-D spinning globe
• Can view data at all scales
• Very easy to incorporate new data
• easy as writing a simple Web page

16
Example of a KML file
17
How it renders
18
More examples of Google Earth data

• Post-Katrina satellite images
• Sea ice cover and ice velocity
• Locations of ARGO floats
• Bird flu outbreaks

19
Google Maps vs Google Earth
20
GODIVA 2

• (currently under development)
• Near-instantaneous previews of data
• Draggable Google Map for easy navigation
• Adjustable scale
• links to Google Earth
• Now we really are exploring data!
• An AJAX application
• (all donkey work is still done by GADS)

http//lovejoy.nerc-essc.ac.uk8080/Godiva2
21
ESSC Data serving architecture
SARIS
Other apps
Google Maps
Google Earth
SOAP messaging
HTTP GET
Web Service interface
Google Maps interface
Google Earth interface
GADS library
Tomcat Application Server
META DATA
DATA
22
VEIGA Very Easy Interface to Grid Applications

• (formerly known as Styx Grid Services)
• Aim is to make running applications on a Grid
  just as easy as running local applications
• Use exactly the same command in both cases
• e.g. myprog i input.dat o output.dat
• Input/output files are automatically
  up/downloaded as required
• Create workflows with simple shell scripts
• Stream data directly between distributed apps
• Run multiple jobs on a Condor pool
• e.g. myprog i inputdir/ -o outputdir/
• Upload directory of input files instead of a
  single file
• Runs myprog over each file in the input directory
• Submit jobs to Globus
• via Condor-G or direct
• Similar in principle to GridSAM and Geodise but
  with different interface

http//www.resc.rdg.ac.uk/jstyx/sgs
23
Bringing it all together...
24
Grid toolkit for environmental science

• Over next 3 years ReSC will create a toolkit to
  make it easier for environmental scientists to
  use e-Science/Grid technology
• Could include
• Enhancing Godiva2 to allow simple data processing
  and visualization to be done (e.g. simple stats
  functions)
• Use of geospatial databases for fast access to
  data
• Provide Google Earth access to OPeNDAP datasets
• VEIGA
• Elements of Geodise (to allow Grid use from
  Matlab for example)
• Wed like feedback from you!
• See me or Dan Bretherton

25
To sum up...
26
Conclusions

• Please consider sharing your data via OGC
  services if possible
• Talk to us or NERC DataGrid for further info and
  help
• Sharing data via Google Maps/Earth is quick and
  easy
• Security not yet a solved problem here
• GM and GE can get images from OGC Web Map Servers
• Godiva2 website is live and wed like people to
  try it out!
• Still some bugs ? but it mostly works!
• Please try out VEIGA
• Again, not a finished product yet
• Please tell us what youd like to see in a Grid
  toolkit

27
Geospatial databases

• A lot of the above relies on fast access to data
  in a multi-user environment
• This is the sort of thing that databases do well
• But most databases dont deal well with
  geospatial data
• Some exceptions, e.g. PostGIS
• Gridded data is still a problem for most systems
• We have been evaluating software from Barrodale
  Computing Services
• Very advanced geospatial database that supports
  gridded data
• Versions for PostgreSQL, Informix, Oracle
• Demos exist at www.barrodale.com
• Results are very promising
• Faster than our system especially for small data
  extractions
• Caches recently-used data for extra speed
• But this is commercial software
• We have an evaluation version, in return for
  feeding back requirements