Title: From STORET to WQX New Jersey
1From STORET to WQXNew Jerseys Solution for
Integrating Ambient Water Quality, Facility and
GIS Information from Federal, State, County
Volunteer Organizations
- Harry Chen
- New Jersey Department of Environmental Protection
- November 28, 2007
- Austin, TX
2Project Purpose
- Integrate water quality data
- Streamline and simplify data transfer
- Provide better, more timely access to water
quality data - Aid decision-making
- Improve resource efficiency
3Goals
- Develop an up to date, searchable inventory of
ambient water quality monitoring in New Jersey - Integrate ambient water quality monitoring data
into NJDEPs enterprise environmental data
management system to enable more integrated
environmental decisions - Make NJ ambient water quality monitoring data
available to the public - Report ambient water quality monitoring data to
USEPA
4Project Partners
- NJDEP
- Bureau of Freshwater and Biological Monitoring
- Bureau of Marine Water Monitoring
- Bureau of Water Quality Standards and Assessment
- Division of Watershed Management
- Division of Science, Research, and Technology
- USGS
- Department of Health and Senior Services (DHSS)
- County Governments
- Volunteer Monitoring Organizations
- NJ Water Monitoring Coordinating Council
(Inventory)
5ExistingAmbient Water Quality Data Producers
6AMNET Field/Hab dB
Reformat Field/Habitat dB
Field Form
Existing NJDEP Data Entry Mechanisms
Text file
Keypunch Field Msr Habitat Sample Info
MS Access Link
Note Proofreading and correction steps occur for
all data, but are not shown due to space
limitations.
AMNET Taxa dB
Bench Sheet
Reformat Taxa dB
Keypunch Taxa counts
Text file
Text file
Make Final dB
Extensive Manual Reformat Of Text File
FIBI dB
Field Form
Keypunch Field Msr Habitat Taxa counts Sample Info
Text file
Change Result Status To Final
ODBC Connect
DHSS Chem Results
Import e-file Chem Results
Migrate
NJ STORET
FTP Oracle Export
Delete Migration
Lakes dB
Reformat Lakes dB
QC Labs Chem Results
Text file
Import e-file Chem Results
EPA
Text file
Field Staff
Data Staff
Field Form
Keypunch Field Msr Habitat Sample Info
ODBC Connect
Corrections Text file
DHSS Chem Results
Keypunch Chem Results
Proofreading dB
Others
Text file
Others
QC Labs Chem Results
Keypunch Chem Results
Other Projects dBs
Corrections
Proofreading Reports
Manual Proofreading
Field Form
Sample Tracking dB
Keypunch Field Msr Sample Info
7Water Quality Data Management Model
8Core project components
- WQX data flow, consisting of the following
- Data-in E2 WQX module with data validation
- Data transmission to and from COMPASS
- Data-out NJ Water Monitoring Inventory / Data
Portal - Outbound data transmission to EPA CDX
9High-Level Data Flow
EPA CDX
E2
COMPASS
ENDS
10Data System Integration
- E2
- COMPASS
- ENDS
- Data will be transmitted through each component
using NJs Exchange Network NODE.
11What is E2?
- The Electronic Environmental Reporting system
(E2) is a secured XML/Internet based system.
12What is COMPASS?
- Comprehensive Organizational Monitoring Program
and Assessment Support System (COMPASS) is an
enterprise-wide methodology and information
management solution.
13What is ENDS?
- Exchange Network Data Services are web-services
based applications used to exchange data between
EN-compliant Nodes.
14FutureAmbient Water Quality Data Producers
EPA USGS Web Services
4
15Data Flow for Submission of All Publicly
Available DataFrom Multiple OrganizationsEx.
305(b) and 303(d) Assessments
Sampling Data
16Data Flow When An Organization UsesMultiple
Groups to Generate Results(Ex. Field
Measurements by One Group and Lab Analysis by a
Second Group)
17Data Flow From a Single Organization(Ex. Field
and Lab Work Done by Same Group)
18Data Submission Options
19CIMP CSV-file Import Mapping Procedures
20Data Field (column) Mapping
21Data Normalization
22Mapping to Domain values
23Save Mapping configurations
24Water Quality Data Transmission to CDX
25GIS Integration
Monitoring Inventory
Data Reports
Link
Web-based Maps
Link
Link
26GIS Proposals
- EN Browser hosted by ECOS, a national solution
- NJ-GeoWeb, a NJ solution
27EN Browser
- Web services data discovery tool
- Allows users to discover and execute Data
Requests on Exchange Network Nodes - Offers users the ability to remove (filter)
unwanted sections of the XML data set returned
from a Data Service - Users have the option of choosing the format that
they would like to retrieve the data in - Can integrate with enterprise data reporting tools
28NJ-GeoWeb
- A New Vision for Seeking Spatial Intelligence.
Integrating customized GIS and analysis tools
with enterprise databases for improved
decision-making on the Internet.
29What is NJ-GeoWeb
- DEPs next generation i-MapNJ on the Interent.
Replaces ArcIntelligence on the intranet. - Uses profiles (tabs) that default data layers
and reports (ie. Child Care, Whats in My
Neighborhood, Smart Growth, Business Viewer) - Allows Internet users to view, interact with,
analyze and submit points, lines and polygons - Links to departmental databases (DataMiner,
eNJEMS, COMPASS..). - Uses enterprise data from all Agencies (and
serves from the Web) - Links to external services on the web.
- Working with NJ OIT make NJ-GeoWeb a state
agency resource.
30NJ-GeoWeb
Permitting Compliance Enforcement
Clean Plentiful Water
Water/Sewer Infrastructure Water Allocation Storm
Water Runoff Monitoring
Whats in My Backyard
Surveillance Inspections ePermits
Regulated Sites Incident Mgmt Flood
Mapping Endangered Species
Data Server
Sustainable Growth
SDE
Data
Taking care of Business
Align Economic and Env. Goals Env. Sensitive
Areas Green Technologies
Economic Incentives Employment Info Revitalize
and Redevelop
Live Feeds from State Government
L.awrence Thornton, NJDEP
31WQDE Project Challenges
- Pulling data from USGS (web services in
development) - Pulling data from EPA (web services in
development) - Coordinating with a large number of project
partners - Requires compromise
- Re-engineering current reporting procedures,
while designing new WQX reporting procedures - How to teach an old dog new tricks?
32Status of NJDEP Data Exchanges
- Water Quality Data Exchange
- Completed Business Process Analysis with 11
project partners - Identified Needs and Analyzed Options for
implementing data exchange - Completed Requirement Specification
- Design currently in progress
- Volunteer Monitoring
- Application in production
- Currently designing reports
33Project Phases and Tasks
- Business Process Analysis
- Functional Requirements Specification and System
Design - System Development, Data Preparation and
Implementation - Acceptance Testing
- User Training, Documentation and Outreach
34Next Steps
- Complete System Design
- Programming and Development
- User Acceptance Testing
- Documentation and Training
- Production Deployment
- Send data to EPA
- Target Date October 2008
35Thanks!
Contact Info NJDEP- Office of Info Resources
Management Name Harry Chen Phone
(609)-633-1261 E-mail Harry.Chen_at_dep.state.nj.us