Configuring NVODS Datasets in LAS

1
Configuring NVODS Datasets in LAS

• Joseph McLean
• TMAP/PMEL/NOAA and JISAO/UW
• OAR WebShop 2002

2
Part 1 -Getting a Dataset "into" LAS-

• Best Case - COARDS Compliant NetCDF File
• The metadata ingestion process.
• The relational database and user interface.
  
• Ready to access and process.
• The LAS Data Configuration XML File
• operations/institution information
• data set attributes
• variable attributes
• back-end properties
• lon/lat/depth/time axis definitions

3
-Data Configuration XML File-

• lt?xml version'1.0' ?gt lt!DOCTYPE spec SYSTEM
  "spec.dtd" lt!ENTITY StdOperations SYSTEM
  "operations.xml"gtgt
• ltlasdatagt
• ltinstitution name"Pacific Marine
  Environmental Lab
• url"http//www.pmel.noaa.gov"/gt
• ltoperations url"http//foo.server.com
  /las-bin/LASserver.pl"gt StdOperations
  lt/operationsgtlt!-- Define datasets/variables
  --gt
• ltdatasetsgt
• ltcoads_climatology_cdf name"COADS
  Climatology"
• url"http//ferret.pmel.noaa.gov/cgi-b
  in/dods/nph-dods/data/PMEL/coads_climatology.ncl"
  
• doc"doc/coads_climatology.html"gt
• ltvariablesgt
• ltairt name"Air Temperature"
  units"DEG C"gt
• ltlink match"/lasdata/grids/coads_c
  limatology_cdf_grid"/gt
• lt!-- Define properties --gt
• ltpropertiesgt
• ltferretgt
• ltpalettegtlight_centeredlt/palettegt
  
• ltinit_scriptgtCOADS_clim_initlt/ini
  t_scriptgt
• ltferretgt
• ltpropertiesgt

LAS attributes

Data set attributes
variable attributes
back-end properties
Axis definitions
4
Part 1 -Getting a Dataset "into" LAS-

• Configuring the XML File
• when needed
• custom palettes or contour levels
• variable name conflicts
• virtual variables
• incomplete/incorrect metadata
• Worst Case - No Metadata
• frequently missing or inadequate
• supply metadata in XML file by hand
• need adequate documentation
• explore the dataset
• custom initialization scripts

5
Part 1 -Getting a Dataset "into" LAS-

• Back-end Scripts
• multiple back-ends
• "NCL","CDAT",GrADS, Matlab,IDL
• Ferret is free and distributed with LAS.
• NVODS LAS(via Ferret) uses the NetCDF API
• to access DODS datasets.
• Custom User Interfaces
• high resolution maps
• data organization

6
Part 2 -Examples of Dataset Problems-

• Missing Attributes
• bad dataflag/missing_value attribute
• Usually this is straightforward.
• Dimensional Metadata
• COARDS Coordinate Variables
• Fixing this can be time consuming.
• need documentation.
• custom scripts
• Time axes
• some time units not supported
• "modulo" attribute
• Wrong Metadata
• Freeform file format descriptor
• Ancillary DAS Files

7
-DODS .das File with Missing Attributes-

• Attributes
• time
• level
• mean_freq
• NC_GLOBAL
• String dtg "2002010700
• String model_name "WAM
• String dset_name "fcst_ops
• String geom_name "adriatic
• Float32 southern_latitude 40.
• Float32 northern_latitude 46.
• Float32 western_longitude 12.
• Float32 eastern_longitude 20.
• Float32 delta_x 0.08333332837
• Float32 delta_y 0.08333332837

8
Part 3 -NVODS/DODS Dataset Statistics

• How does Metadata affect Performance?
• A summary of Tests on a sample of the DODS
  Dataset List
• Dataset Ready to Use as accessed - 30
• Dataset Usable after Ferret repairs - 35
• Unable to use Dataset - 35
• Please Note that not all of the problems with
  opening and using datasets were related with
  unavailable use metadata. Some of the reasons for
  not being able to use a Dataset included password
  needs, bad links, and incompatible file types.
  With the use of back-end scripts to enhance and
  correct use metadata, we were able to
  successfully access 65 of the DODS datasets
  tested.

9
- Conclusion -

• How can the Ancillary Information Service Help?
• The NVODS LAS is Not Unique
• Fix the Dataset, Not the Application
• Allow a User to Determine the Fix