Clients

1
Clients
2
Accessing Data with DAP

• Web Browser
• Already discussed
• Graphical clients
• ncBrowse, ODC, Ferret, GrADS
• Command-line clients
• getdap (UNIX, win32), loaddap (Matlab, IDL), nco
  (UNIX, win32)
• Custom clients
• C, C, Java, Python
• netCDF

3
Using a Graphical Client

• Example The OPeNDAP Data Connector
• Combines data location with retrieval and display
• Shows the built URL, including constraint
  expression
• Can be transferred to another application

4
Start the ODC
5
The ODC opens to the search pane
Five different panes
Choices within a pane
6
Use the dataset list to find the TPAC
climatologies
Choose the Antarctic Cooperative Research Centre
TPAC/CISRO Climatologies
then hit To Retrieve to move the selection to
the next pane
7
The Retrieve pane
Double click levitus_annual_97.nc To see the
contents of the file in The area on the right
8
The ODC shows the URL as it builds it.
Click the checkbox for SALT and O2. For both, set
the range of z_index to 0 to 0. Make sure to
hit tab/return in The boxes.
then hit Output to to move to the View pane
9
(No Transcript)
10
There are a number of ways to view The data. Here
the plotter has been Chosen (the default).
Hit Plot to to generate a plot using
the Default settings.
11
When the plot is made, the interface Switches to
the Preview tab
Switch back to the Variables tab to Plot O2
12
Choose O2 from the menu, then hit Plot to.
13
Now that the data have been read and Cached, you
can switch back and Forth between variables
quickly without Any additional data transfers
When ready, go back to the Retrieve Pane.
14
Choose TEMP
Set the constraint
then plot
15
(No Transcript)
16
ODC Summary

• The ODC provides a way to search for, access and
  plot data
• Acts as a URL builder the URLs can be pasted
  into other applications
• We didnt need to know anything about DAP, its
  Request or Response objects or how a URL is used
  to request data
• The data set list often contains stale entries
• Also supports using the GCMD for data location -
  more on this when we cover searching

17
Using a Command-line Client

• Matlab - demonstration
• NCO - a powerful tool developed and maintained by
  another group

18
Matlab

• Demonstration of custom-built Graphical
  interfaces for Matlab
• Matlab scripting is used to build the interfaces
  and provide some dataset-specific processing
• A Matlab command extension is used to read the
  data (written in C/C).
• Two things are required in addition to Matlab
  The DAP command extension (loaddap) and the
  graphical interface software.

19
Running the Matlab Demonstration

• Start Matlab
• Download the command extension
• Download the interface software
• In Matlab change directory to the
  ml-ocean-testbed directory.
• Type OCEAN_TOOLBOX
• The interface will start

20
The Ocean Toolbox
21
Open a dataset
I choose the Pathfinder dataset
22
Fill in the information
SST Quality fields
Load data into the Matlab workspace
23
Get the data
Load data into the Matlab workspace
24
Plot/Display the data
25
Using the loaddap command extension directly

• Start Matlab
• Add the directory with the extension to the
  Matlab command path
• Verify the command extension is working
• Feed it a URL
• Plot the data

26
Pass a URL, constrain the response To the u and
v vectors only
Plot those vectors See Figure 1
27
Matlab Summary

• Command line client is the tool used to move the
  data
• Easily used in Matlab scripts to hide the details
  and make custom interfaces
• To the the command extension directory, user must
  know
• Data location (URL)
• Internal structure of the data set (syntactic
  metadata - DDS/DDX)
• How to write a constraint expression

28
NetCDF Operators (NCO)

• Unix command line client
• Unlike the previous two clients, NCO uses the
  netCDF client library to read from a DAP server
• A client library is a collection of functions
  which hide the mechanics of (most of) the
  interaction with a server so the client can go
  about its business
• The NCO client is, in fact, just the NCO package
  linked to our (OPeNDAPs) version of the netCDF
  library (aka. the netCDF client library)

29
Build the NCO Software

• Change directory to /root/src/nco-3.9.2
• root_at_slax cd /root/src/nco-3.9.2
• Run configure to build the Makefile, then build
  and install the software
• root_at_slax ./configure
• root_at_slax make
• root_at_slax make install

30
Use NCO to Convert the FNOC1 vectors into a speed

• NCAP NCO Arithmetic Processor
• ncap2 -s speedsqrt(u2,v2) http//localhost80
  80/opendap/data/nc/fnoc1.nc spd.nc
• The URL is the input file and wndspd.nc is the
  output
• Use ncdump to look at the result file
• ncdump -h spd.nc
• ncdump -v speed spd.nc

31
View the Result ncBrowse

• We can use ncBrowse to look at the local neetCDFZ
  we just built
• ncBrowse can also look at the DAP server directly
• Built using the DAP-enabled Java netCDF library
  (a client library where access to DAP servers
  hides behind the netCDF API)

32
Start ncBrowse
Double click on speed - the new data we made
with the previous NCO example
33
Fix up the latitude and longitude axes, the
Graph Variable.
34
We have to be somewhat savvy about the units -
check back and look at the attributes
35
Custom clients

• What options exist to build clients
• C using libdap
• C using Ocapi
• C,Fortran using the netcdf client library
• Python using PyDAP
• Java using Java-OPeNDAP
• Matlab IDL using the respective versions of
  loaddap

36
Clients Summary

• Custom clients offer an opportunity to develop
  for a specific audience or a particular
  problem/project.
• Example ComMIT Tsunami inundation model client
  developed by NOAA/PMEL and BOM
• General purpose clients like loaddap can read any
  kind of data while clients built using the netCDF
  client library are limited to the semantics of
  netCDF
• Example Record access is slow because each
  access is separate network request

37
Clients Summary, cont.

• ODC A client built specifically to provide a
  browse capability for any data source
• Uses Java-OPeNDAP
• Loaddap a client built to read any data into an
  analysis application
• Can be used as a building block for more
  sophisticated applications
• Use libdap (C, Matlab) or Ocapi (C, IDL)
• netCDF client library A client-building tool
• convert legacy code
• provide a simple way to write new applications
• C, C, Fortran

38
Finding Data

• Ways to find data
• The OPeNDAP Data Set List
• GCMD
• TPAC
• Google
• THREDDS
• We maintain a page with links to dataset
  searching sites
• http//www.opendap.org/data/index.html

39
Common Features

• All of these data location features except Google
  depend on active community involvement in
  building catalogs of data
• The solutions can be described as static
  documents or crawlers
• Google and TPAC are crawlers
• Crawlers can discover datasets without human
  intervention
• They can make mistakes that seem silly
• The The Dataset List, GCMD and THREDDS are static
  documents or collections of static documents
• Static lists can be tailored by hand
• They can go out of date quickly

40
Differentiating Features

• Google TPAC
• Google is just crawling HTML. If a server is not
  linked to a HTML page, it wont be found.
• TPAC is preset with server locations and picks up
  changes at those sites

41
Differentiating Features, cont.

• The Static Lists
• The Dataset List has a very low metadata
  requirement
• Not maintained as actively as either GCMD or
  THREDDS catalogs
• GCMD
• The GCMD has a fairly high entry level threshold
• Professional staff maintain the GCMD as their
  sole job
• THREDDS
• THREDDS catalogs are, or can be, located at the
  data - locality distributes maintenance
• Quality varies from site to site

42
Finding Data Summary

• Locating data seems like it would be the place to
  start building a system, but its far more varied
  than the one-size-fits-all approach most tried in
  the 1990s
• Crawlers and hierarchical lists show the most
  promise but maintained centralized lists are also
  useful