Hydrologic Information System Services Architecture

1
Hydrologic Information System Services
Architecture

• Collaborative project
• UTAustin (D. R. Maidment) SDSC (C. Baru, I.
  Zaslavsky) Utah State U (D. Tarboton) Drexel
  U (M. Piasecki) Duke U. (J. Goodall)
• CUAHSI Office (R. Hooper)

2
The Grid is becoming the backbone for
collaborative science and data sharing
CI is about RE-USING data and research resources
!!
3
CI Vision for Hydrologic Science

• Leverage ongoing cyberinfrastructure projects
• The Geosciences Network (GEON)
• Share data between Earth Disciplines
• Secure access to Grid resources, single sign-on
  authentication/ authorization, distributed data
  management, data publication, search, information
  integration, knowledge management, scientific
  workflows, archiving
• Integrate with common COTS (commercial off-the
  shelf) software
• Excel, ArcGIS, Matlab
• and Fortran mostly on Windows
• Interesting survey of CUAHSI partners by David
  Tarboton!

4
HIS User Assessment (Chapter 4 in Status Report)
Which of the four HIS goals is most important to
you?
Data Access
Observatory support
Science
Education
5
Tuning to unique features of hydrology

• Hydrologic observations
• Reliance on federally-organized data collection
  (NWIS, STORET, Ameriflux, etc.) with huge and
  complex nomenclatures
• ? simplifying access to federal repositories
• ? relatively lower emphasis on data ownership
• Handling time in both UTC and local
• Various spatial offsets
• Multiple data types time series, fields, spatial
  data
• Integrative discipline
• Interoperation with atmospheric, ocean, soils,
  geomorphology, social datasets and services
• Community
• Organized by natural boundaries
• ? natural object hierarchy
• ? networks of relatively autonomous self-managed
  data nodes
• Partnership with public sector water management

6
Hydrologic Information System Service Oriented
Architecture
Downloads
Uploads
HTML -XML
Data access through web services
WaterOneFlow Web Services
WSDL - SOAP
Data storage through web services
7
Main Components

• Web services for accessing hydrologic
  repositories
• Hydrologic Observations Data Model
• Hydrologic Data Access System Time
  SeriesViewer
• Collection of CUAHSI nodes

8
Database Sizes
Records
Stations
Time range
USGS
250 million
1.5 million
100 years
EPA
200 million
800,000
100 years
NWS
100 years
?
19,000
(From Jon Goodall, Duke U.)
9
Language for Data Representation
Time of Measurement
Unique Identifier for a Observation Station
Latitude, Longitude
USGS
dec_lat_va, dec_long_va
dv_dt
site_no
EPA
Station Latitude, Station Longitude
Activity Start
Station ID
NWS
LATITUDE, LONGITUDE
YEAR,MO,DA,TIME
COOPID
Lots of semantic differences in parameter names,
methods, etc.
10
CUAHSI Search Wizard
Output (Drexel Univ.)
NWIS Parameter Codes
Inside the NWIS Module
Ontology Viz Demo
Search Wizard Demo
11
WaterOneFlow Web Services
Output string standardized to a common set of
objects (XML schema)
12
CUAHSI Web Services
http//www.cuahsi.org/his/webservices.html
NCEP North American Forecast Model 12 Km grid for
continental US
New services http//water.sdsc.edu/wateroneflow/
13
CUAHSI Point HydrologicObservations Data Model
(D. Tarboton, USU)

• A relational database stored in Access,
  PostgreSQL, MS SQL Server, .
• Stores observation data made at points
• Consistent format for storage of observations
  from many different sources and of many different
  types.

Streamflow
Groundwater levels
Precipitation Climate
Soil moisture data
Flux tower data
Water Quality
Community design requirements (22 reviewers)
14
Schema
15
Uses and tools for HODM

• HODM is central to HIS infrastructure, but lacks
  tools
• Testing HODM with two types of data federal
  repositories, and external databases (Panola).
  Personal and enterprise versions.
• Mapping wizard loading Excel observation data
  to HODM database
• Can save mapping files for subsequent runs of
  similarly formatted spreadsheets
• Local data analysis can be done charts and
  stats
• HDAS as an interface to HODM datasets - but
  shall not be the only one - so exposing HODM
  as Web services

16
Hydrologic Data Access System
http//river.sdsc.edu/hdas/
17
Hydrologic Data Access System
18
Cross-platform design
GEON Data Node (Linux)
Central CUAHSI HIS Node (Windows)
HODM
Web
HDAS
Web
Geon Software Stack
Services
Service
Proxy
IIS Web Server
Apache Tomcat
ASP
.
Net
Web
ArcGIS
SQL Server
Service
Technologies
proxies
Data
Data
Remote CUAHSI HIS Nodes (Windows)
19

• Resource registration
• Shapefiles
• TIFF images, GMT rasters
• Web Services, WMS services
• Relational databases, ASCII
• PDFs, URLs
• CUAHSI data
• NetCDF
• Coming Geodatabases and ODM

20
HIS Scalability

• Adding
• data types and datasets processing models and
  services servers users and roles
• - shall not create unmanageable bottlenecks that
  require system re-engineering
• Designing for scalability
• Distilling a generic set of web service
  signatures resolving semantic and structural
  heterogeneities
• Using HODM as a common generic format for time
  series data, for ease of coding and uniform
  search interfaces
• HDAS GUI design to abstract specifics of
  disparate repositories
• Leveraging common CI components developed in GEON
  and other projects
• Have good design docs, to allow others develop
  and deploy systems
• Also Need to work with agencies to remove the
  web services bottleneck some progress with USGS
  and NOAA!!

21
Plans

• HIS 1.0, Oct 30, 06
• Version 1 of WaterOneFlow web services
• version 1 of ODM
• Additional catalogs in HODM
• HDAS deployable
• HIS workbook
• HIS design document
• Web services and catalogs for fields
• Further GEON integration registering ODMs to
  GEON catalog, and completing a Windows-based node
• Planning for 5 more years!