Near Real Time Ocean Observations Online

1
Near Real TimeOcean Observations Online

• the Escape of SEACOOS(Southeastern Atlantic
  Coastal Ocean Observing System) Data Management
  and Visualization Secrets

2
The (tiny) big picture
3
(No Transcript)
4
The nuts bolts

• Data is aggregated and stored
• Data is normalized
• Data is visualized
• Data is disseminated

5
Data aggregation storage

• Format flavors
• netCDF coverage may vary format may not
  vary
• PNG coverage may not vary color scale may not
  vary

• Technology
• Perl

6
Data aggregation storage

• Relational database
• One table per in-situ variable type
• E.g. a wind_table, sst_table
• Contains the real data
• One table per raster variable
• E.g. a sst_raster_table
• Contains pointers to PNG files boundaries
• One table per model variable per hour
• E.g. a 12_00_00_current_forecast_table

PostgreSQL PostGIS

7
Normalization

• Reporting time varies
• In-situ data
• E.g. daily, hourly, half-hourly, every 10 minutes
• Remotely-sensed data
• E.g. twice daily
• Reporting area varies
• Remotely-senseddata passes

PostgreSQL PostGIS

8
Visualization

• Main engines
• PHP, PHP-MapScript
• Perl
• MapServer, of course
• Icing on the cake
• Dynamic shapefile creation for colorbars
• AnIS and GIFSicle for animations
• ImageMagick for image manipulation and raster
  queries
• Mouseovers
• Time-series graphs usingGDGraph

9
Icing on the cake dynamic shapefiles
10
Icing on the cake animations

• AnIS and GIFsicle
• Produce severalimages and glue them together

11
Icing on the cake raster queries

• Let MapServer take care of projection issues
• How to get an SST, for example, from a raster PNG
• Dump the current extent view of just the raster
  to PNG
• Get (x,y) clicked pixel RGB of the dumped image
  with PerlImageMagick
• Lookup pixel RGB in database to find out SST
• Return SST in the normal query data train

12
Icing on the cake mouseovers

• Use MapServer 4.x searchmap
• Produce two images for an HTML page
• One is static with the querymap
• The other is an input image

• Use javascript to swap the two images based on
  user activity
• The querymap is produced by making a simple
  LWPsimple request for a querymap, and then it
  is merged with the HTML page

13
Icing on the cake time series graphs

• Pass a query result set to GDGraph for a
  time-series graph

14
Dissemination

• Maps and charts, of course
• OPeNDAP (DODS) data access
• OGC-friendly WMS, WFS
• SEACOOS pushed a lot of data to the IOOS
  interoperability project
• Down the pipeline
• Ad-hoc CSV and/or netCDF data extraction

15
Magic 8 ball

• Everyone wants speed
• Near real time implies constantly updated
  databases
• Recently introduced round-robin-ing of database
  requests
• As machine A is populated with new data, send all
  DB requests to machine B
• When machine A is done populating, have it resume
  DB handling, and go update machine B
• Want to round-robin MapServer requests, too, as
  well as pull application server to its own box

16
A friend in the business

• Development site http//nautilus.baruch.sc.edu/r
  s
• Production site (new interface to be released
  soon)http//seacoos.org
• Very active listserv dealing with mainly
  remote-sensing issues remotesensing_at_caro-coops.o
  rg
• mailto majordomo_at_caro-coops.org
• message text subscribe remotesensing
• Who am I?
• Charlton Purvis, University of South Carolina,
  SEACOOS
• cpurvis_at_sc.edu
• Happy to help and share.