The Earth System Grid: A Visualisation Solution

1
The Earth System GridA Visualisation Solution

• Gary Strand

2
Introduction
3
Acknowledgments

• ESG Development Team
• Veronika Nefedova (ANL)
• Ann Chervenak (ISI/USC)
• Carl Kesselman (ISI/USC)
• David Bernholdt (ORNL)
• Kasidit Chanchio (ORNL)
• Line Pouchard (ORNL)
• Alex Sim (LBNL)
• Arie Shoshani (LBNL)
• Bob Drach (LLNL)
• Dave Brown (NCAR)
• Gary Strand (NCAR)
• Jose Garcia (NCAR)
• Luca Cinquini (NCAR)
• Peter Fox (NCAR)

• PIs
• Ian Foster (ANL)
• Don Middleton (NCAR)
• Dean Williams (LLNL)

4
Current Practices

• Scientist (or others) wants a visualisation
• Visualisation person gets appropriate data after
  verifying with data manager as to the name,
  location, total size, etc.
• Data moved to local machine that has
  visualisation tools
• Visualization created on local machine
• Hopefully, someone remembers to archive the
  visualisation

5
Simple Vis Example
6
Problems in the process

• What if the data cannot be found (e.g. we have
  1.2 million files, 73 TB of data), or the data
  manager is unavailable?
• What if there isnt enough disk space or
  sufficient other resources?
• What if a better visualisation tool is located
  elsewhere?
• What if the visualisation should be shared?
• What if the visualisation is lost?
• ESG is part of the answers to these questions

7
What is ESG?
ANL Computational grids, grid-based
applications
LBNL Climate storage facility
LLNL Model diagnostics inter-comparison
USC/ISI Computational grids, grid-based
applications
ORNL Climate storage computational resources
LANL Next generation coupled models computing
NCAR Climate change predication and scenarios
8
ESG Architecture
LBNL
HPSS High Performance Storage System
disk
ANL
CAS Community Authorization Services
NCAR
SRM Storage Resource Management
gridFTP Striped server
gridFTP server
openDAPg server
Tomcat servlet engine

MyProxy server
LLNL
MCS client
MyProxy client
disk
CAS client
RLS client
SRM Storage Resource Management
GRAM gatekeeper
gridFTP server
gridFTP server
ORNL
gridFTP
SRM Storage Resource Management
ISI
gridFTP
gridFTP server
SRM Storage Resource Management
MCS Metadata Cataloguing Services
SOAP
HPSS High Performance Storage System
RLS Replica Location Services
RMI
MSS Mass Storage System
disk
disk
9
Solutions

• What happens when data cannot be found, or the
  data manager is unavailable?
• Metadata catalogue service (MCS)
• Replica location service (RLS)

10
MCS and RLS and Metadata Services
11
Solutions (contd.)

• What if there isnt enough disk space or
  sufficient other resources?
• Hierarchical Resource Manager (HRM)

12
HRM
13
Solutions (contd.)

• What What if a better visualization tool is
  located elsewhere?
• Distributed visualization

14
CDAT Example of an ESG Script Access

• The next-generation language, Python, is used to
  access the Earth System Grid (ESG) at LLNL

15
CDAT Example of an ESG GUI Client Access
16
Solutions (contd.)

• What if the visualization should be shared?
• Access Grid plus Visualisation Tool

17
Collaborative Environments
Science Portals AccessGrid University of
Michigan (Knoop, Hardin) Vegetation Ecosystem
Mapping Program (VEMAP) NCAR/SCD
VETS/KEG Argonne National Labs
18
Conclusions

• Visualisation can require as many services and
  resources as the initial computation
• Many sites do not offer sufficient resources for
  the visualisations earth sciences require
• ESG provides, and will provide, the tools that
  enable visualisation on a grander scale

19
Conclusions (contd.)

• ESG tools enable better data access, better data
  knowledge, and the processes of collaboration for
  the needs of investigating, visualising, and
  learning