Development of a Community Hydrologic Information System

1
Development of a Community Hydrologic Information
System
David Maidment (PI), David Tarboton, Dan Ames,
Jon Goodall, Richard Hooper, and Ilya Zaslavsky

• http//his.cuahsi.org/

2
The way that data is organized can enhance or
inhibit the analysis that can be done
I have your information right here
Picture from http//initsspace.com/
3
Data Models


Continuous surface
4
What are the basic attributes to be associated
with hydrologic data values and how can these
best be organized?
5
What is HIS

• The CUAHSI Hydrologic Information System (HIS)
  provides web services, tools, standards and
  procedures that enhance access to more and better
  data for hydrologic analysis.
• http//his.cuahsi.org

Precipitation
Water quantity
Water quality
Meteorology
6
Consortium of Universities for the Advancement of
Hydrologic Science, Inc.

• 110 US University members
• 6 affiliate members
• 12 International affiliate members
• (as of March 2009)

An organization representing more than one
hundred United States universities, receives
support from the National Science Foundation to
develop infrastructure and services for the
advancement of hydrologic science and education
in the U.S.
http//www.cuahsi.org/
7
HIS Team and Collaborators

• University of Texas at Austin David Maidment
  (PI), Tim Whiteaker, Ernest To, Bryan Enslein,
  Kate Marney
• San Diego Supercomputer Center Ilya Zaslavsky,
  David Valentine, Tom Whitenack
• Utah State University David Tarboton, Jeff
  Horsburgh, Kim Schreuders, Justin Berger
• Drexel University Michael Piasecki
• University of South Carolina Jon Goodall, Tony
  Castronova
• Idaho State University Dan Ames, Jirí Kadlec,
  Ted Dunsford, Teva Veluppillai
• CUAHSI Program Office Rick Hooper, David
  Kirschtel, Conrad Matiuk, Yoori Choi
• WATERS Network Testbed Data Managers
• HIS Standing Committee
• USGS Bob Hirsch, David Briar, Scott McFarlane
• NCDC Rich Baldwin

8
Data Searching What we used to have to do
Searching each data source separately
Michael Piasecki Drexel University
9
What HIS enables
Searching all data sources collectively
GetValues
GetValues
GetValues
GetValues
generic request
GetValues
GetValues
Michael Piasecki Drexel University
GetValues
GetValues
10
Services-Oriented Architecture for Water Data
HIS Central
Data Discovery and Integration platform
Metadata Services
Metadata Search
Service and data theme metadata
Service registration
Data carts
Catalog harvesting
Data Services
Data Publication platform
Data Synthesis and Research platform
Water Data Services
Spatial Data Services
HydroDesktop
HIS Server
11
CUAHSI Water Data Services System
Discovery
Access
Analysis
Query, Visualize, and Edit data using ODM Tools
Hydroseek
GIS Matlab Splus R IDL
Hydro Desktop HydroExcel
Java C VB
ODM Database
Service Registry
Hydrotagger
WaterML
Streaming Data Loader
GetSites GetSiteInfo GetVariableInfo GetValues
Excel, text
HIS Central
Harvester
Water Metadata Catalog
USGS NWIS
WaterOneFlow Web Service
ODM Data Loader
EPA STORET
NCDC
Others
12
WaterML and WaterOneFlow
WaterML is an XML language for communicating
water data WaterOneFlow is a set of web services
based on WaterML

• Set of query functions

• Returns data in WaterML

Slide from David Valentine
13
GetValues
GetVariables
qualifier
ISO Time
value
14
Direct analysis from your favorite analysis
environment. e.g. Matlab
create NWIS Class and an instance of the
class createClassFromWsdl('http//river.sdsc.edu/w
ateroneflow/NWIS/DailyValues.asmx?WSDL') WS
NWISDailyValues GetValues to get the
data siteid'NWIS02087500' bdate'2002-09-30T00
0000' edate'2006-10-16T000000' variable'NWI
S00060' valuesxmlGetValues(WS,siteid,variable,b
date,edate,'')
15
What are the basic attributes to be associated
with each single data value and how can these
best be organized?

• Observations Data Model (ODM)
• A relational database at the single observation
  level (atomic model)
• Stores observation data made at points
• Metadata for unambiguous interpretation
• Traceable heritage from raw measurements to
  usable information
• Standard format for data sharing
• Cross dimension retrieval and analysis

16
CUAHSI Observations Data Model
http//his.cuahsi.org/odmdatabases.html
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment
and I. Zaslavsky, (2008), A Relational Model for
Environmental and Water Resources Data, Water
Resour. Res., 44 W05406, doi10.1029/2007WR006392
.
17
Stage and Streamflow Example
18
Lets explore the systemhttp//his.cuahsi.org
19
Hydroseekhttp//www.hydroseek.org
Bora Beran, Drexel
Supports search by location and type of data
across multiple observation networks including
NWIS and Storet
20
HydroExcel
http//his.cuahsi.org/hydroexcel.html
21
HIS Server

• Internet Map Server built using ArcGIS
• Web browser client
• Combine spatial data and observational data
• Self contained fully documented with local
  control of data
• Launch data visualization tools

e.g. http//icewater.usu.edu/map and
http//littlebearriver.usu.edu/
22
Loading data into ODM
ODM Data Loader

• Interactive ODM Data Loader
• Loads data from spreadsheets and comma separated
  tables in simple format
• Streaming Data Loader (SDL)
• Loads data from datalogger files on a prescribed
  schedule
• Interactive configuration
• SQL Server Integration Services (SSIS)
• Microsoft application accompanying SQL Server
  useful for programming complex loading or data
  management functions

SDL
SSIS
23
Reducing Semantic Heterogeneity

• ODM Controlled Vocabulary System
• ODM CV central database
• Online submission and editing of CV terms
• Web services for broadcasting CVs

ODM VariableNameCV
Term

Sunshine duration
Temperature
Turbidity

Variable Name
Investigator 1 Temperature, water
Investigator 2 Water Temperature
Investigator 3 Temperature
Investigator 4 Temp.
From Jeff Horsburgh
24
Dynamic controlled vocabulary moderation system
ODM Data Manager
ODM Website
ODM Tools
ODM Controlled Vocabulary Moderator
XML
Master ODM Controlled Vocabulary
Local ODM Database
ODM Controlled Vocabulary Web Services
Local Server
http//his.cuahsi.org/mastercvreg.html
From Jeff Horsburgh
25
HydroDesktopHarvesting and analyzing data from
web services
http//hydrodesktop.codeplex.com
26
HydroDesktop Building blocks
Actual Data Dababase
MapWindow GIS Components
Metadata Cache Database
HydroDesktop Main Application
Plug-in Interface
Search plug-in
Graph View plug-in
Metadata plug-in
27
Advancement of water science is critically
dependent on integration of water information
Databases Structured data sets to facilitate
data integrity and effective sharing and
analysis. - Standards - Metadata - Unambiguous
interpretation Analysis Tools to provide
windows into the database to support
visualization, queries, analysis, and data driven
discovery. Models Numerical implementations of
hydrologic theory to integrate process
understanding, test hypotheses and provide
hydrologic forecasts.
Models
ODM
Web Services
Databases
Analysis
28
Conclusions

• Data Storage in an Observations Data Model (ODM)
• Data Delivery through internet-based Water Data
  Services using a consistent data language, called
  WaterML
• Data Discovery through federated thematic keyword
  search services and a National Water Metadata
  Catalog
• Data Access and Analysis through a Client
  Application (HydroDesktop)

The combination of these capabilities creates a
common window on water observations data for the
United States unlike any that has existed before.
29
Reports

• 2008 status report summarizes the conceptual
  framework, methodology, and application tools for
  HIS
• Shows how to develop and publish a CUAHSI Water
  Data Service
• 2009 status report details the expansions and
  updates

http//his.cuahsi.org/documents/HISOverview2009.pd
f
30
http//his.cuahsi.org