Web Services for Water Observations in Texas

1
Web Services for Water Observations in Texas

• By David Maidment
• Center for Research in Water Resources
• University of Texas at Austin
• Presented at Water Data Roundtable
• University of California at Davis
• May 14, 2009

2
Web Services for Water Observations Data in Texas

• CUAHSI Hydrologic Information System
• CUAHSI Water Data Services
• Texas Water Data Services

3
State and Regional Hydrologic Information Systems

• CUAHSI Hydrologic Information System
• CUAHSI Water Data Services
• Texas Water Data Services

4
Consortium of Universities for the Advancement of
Hydrologic Science, Inc (CUAHSI)
118 Universities in North America (and 3 in
Europe)
NSF supports building a Hydrologic Information
System (HIS)
5
Services-Oriented Architecture Interactions
Centralized service registry and metadata catalog
HIS Central
WaterML
WaterML
WaterML
HIS Server
HIS Desktop
Serves observations data
Gets data from services
Under development
6
HIS Desktop Harvesting data from web services
Observations
Models
GIS
Climate
Remote Sensing
Goal for 2009
7
Water Observations Data
Rainfall
Water quantity
Soil water
Groundwater
Meteorology
Water quality
8
Data are Collected by Many Organizations
9
Data are Published in Many Formats
10
Services-Oriented Architecture
A services-oriented architecture is a concept
that applies to large, distributed information
systems that have many owners, are complex and
heterogeneous, and have considerable legacies
from the way their various components have
developed in the past (Josuttis, 2007).
11
HTML as a Web Language
Text and Pictures in Web Browser
12
State and Regional Hydrologic Information Systems

• CUAHSI Hydrologic Information System
• CUAHSI Water Data Services
• Texas Water Data Services

13
Point Water Observations Time Series
A point location in space
A series of values in time
14
WaterML as a Web Language
Discharge of the San Marcos River at Luling, TX
June 28 - July 18, 2002
USGS Streamflow data in WaterML
WaterML is constructed as a Web Services
Definition Language using WWW standards
15
Point Observations Information Model
Information is transmitted through the internet
in WaterML as web services
Utah State Univ
Data Source
Network
Little Bear River
GetSites
Sites
Little Bear River at Mendon Rd
GetSiteInfo
GetVariableInfo
Variables
Dissolved Oxygen
GetValues
Values
9.78 mg/L, 1 October 2007, 5PM
Value, Time, Metadata

• A data source operates an observation network
• A network is a set of observation sites
• A site is a point location where one or more
  variables are measured
• A variable is a property describing the flow or
  quality of water
• A value is an observation of a variable at a
  particular time
• A metadata quantity provides additional
  information about the value

16
Data Values indexed by What-where-when
Time, T
t
When
A data value
vi (s,t)
Where
s
Space, S
Vi
What
Variables, V
17
Data Values Table
Time, T
t
vi (s,t)
s
Space, S
Vi
Variables, V
18
Data Series Metadata description
There are C measurements of Variable Vi at Site
Sj from time t1 to time t2
19
Series Catalog
Sj
Vi
t1
t2
C
20
CUAHSI Observations Data Model
Values
Series
http//his.cuahsi.org/odmdatabases.html
21
The Simplified ODM Structure
22
HIS Servers at Universities in the WATERS Network
HIS Central at San Diego Supercomputer Center
23
Synthesis and communication of the nations water
data http//his.cuahsi.org
Government Water Data
Academic Water Data
National Water Metadata Catalog
Hydroseek
WaterML
24
CUAHSI Water Data Services
35 services 15,000 variables 1.75 million
sites 8.33 million series 342 million data
25
How CUAHSI works with federal water agencies

1. Establish an agreement with the agency, identify
   agency points of contact
2. Identify the scope of the service
3. Translate the semantics of the service to WaterML
4. Invite agency personnel to join OGC/WMO Hydrology
   Domain Working Group
5. Develop a first draft of the web service
6. Harvest an observations metadata catalog for
   agency data
7. Perform unit testing, over a series of validation
   cases
8. Develop a procedure for catalog updates
9. Document and register the water data service at
   HISCentral
10. Review and test the service together with the
    agency, for possible approval as operational

26
State and Regional Hydrologic Information Systems

• CUAHSI Hydrologic Information System
• CUAHSI Water Data Services
• Texas Water Data Services

27
Texas Hydrologic Information System
Sponsored by the Texas Water Development Board
and using CUAHSI technology for state and local
data sources (using state funding)
28
Texas Water Data Services
10 services 7,010 variables 15,870 sites 645,566
series 23,272,357records
29
Synthesis of Data Sources
TCEQ TRACS
TWDB Water Quality
TWDB Evaporation
TPWD Water Quality
Sabine TIFP
San Antonio TIFP
30
Publishing an ODM Water Data Service
TPWD Coastal Fisheries Raw Data
TWDB Coastline Raw Data
TIFP Lower Sabine
Ingest Data From Different Sources
Data Upload
Transform Data into Uniform Format
WaterML
Load Newly Formatted Data into ODM Tables in MS
SQL/Server
Observations Data Model (ODM)
TPWD ODM
TWDB ODM
TIFP ODM
Wrap ODM with WaterML Web Services for Online
Publication
31
Publishing a Hybrid Water Data Service
TCOON Metadata are Transferred from XML to the ODM
WaterML
TCOON METADATA ODM
Web Services can both Query the ODM for Metadata
and use a Web Scraper for Data Values
TCOON Water Data Service
Get Values from
Metadata From ODM Database in Austin
TCOON Web Site in Corpus Christi
GetSites GetSiteInfo GetVariableInfo
Calling the WSDL Returns Metadata and Data Values
as if from the same Database
32
Web Services in Space and Time

• Water Markup Language (WaterML) is a schema for
  encoding water observations time series data and
  metadata
• Geographic Markup Language (GML) encodes spatial
  data about sets of geographic features
• Sensor Web Enablement (SWE) specifies encoding
  and management of observations made at geographic
  features
• Combine these specifications so that you can
  transmit water data in space and time

33
CRWR Web Services Library
34
WFS and WaterML
Observations Metadata in Space in GML as a Web
Feature Service
Observations Data in Time in WaterML
35
WaterML Data Services
36
Metadata Web Feature Services
37
A Theme Layer
Synthesis over all data sources of observations
of a particular variable e.g. Salinity
38
Texas Salinity
7900 series 347,000 data
7900 series TPWD 3400 TCEQ 3350 TWDB 150
39
Copano and Aransas Bay Salinity
Number of Data 0 50 50 150 150 400 400
1000 1000 3000
Copano Bay
Aransas Bay
40
Austin Travis Lakes Streamflow
Years of Data 0 10 10 20 20 40 40 60 60
110
41
Texas Water Temperature
22,700 series 966,000 data
42
Austin Travis Lakes Water Temperature
Number of Data 0 50 50 150 150 400 400
1000 1000 5000
43
Texas Natural Resources Information System Data
Viewer
44
Texas Natural Resources Information System Data
Viewer Observation Sites
45
Time Series
46
GIS and Water Data Services

• GIS Hydro Preconference Seminar at the ESRI User
  Conference in San Diego
• Sunday July 11, 830-5PM at San Diego Convention
  Center
• See http//www.esri.com for signup
• Describes latest developments in web services for
  GIS data and their relation with water data web
  services

47
Conclusions

• We have built a successful and functioning
  services-oriented architecture for water
  observations data in the United States
• WaterML is critical as the common water data
  language
• Texas HIS is built as an independent system at
  the state level consistent with what CUAHSI is
  doing nationally
• Themes are an interesting way of thinking about
  the data