GODAE Quality Control Pilot Project

1
GODAE Quality Control Pilot Project

• Status Report August 2007
• IGST Meeting
• Department Fisheries Ocean
• St. Johns, New Foundland, Canada
• Outline
• New Data Sources
• Output File and Reference Data
• Plans and Issues
• Argo FSI Float QC Outcomes

2
GODAE Quality Control Pilot Project

• New Data Sources
• UKMO
• long list form of UKMO QC output now available in
  real-time
• UKMO back filled long list data to 2003 thanks
  to Matt Martin
• long list provides additional QC outcomes and
  both in situ and potential temperatures (short
  list had only potential temperatures)
• convention is to use in situ temperature in QC
  databases
• in situ calculated from potential when salinity
  reported
• not always the case in the UKMO short list files
• BMRC
• pushing QC data from BoM system to GODAE server
• currently have data for 2005 and 2006 (nothing
  from 2007 yet)
• working on BMRC data decoder

3
GODAE Quality Control Pilot Project

• New Output File Format and Reference Data
• Call sign database conversion to NetCDF format
  complete
• based on Coriolis/Argo multiple profile NetCDF
  file structure
• easily expanded with new QC variables or new QC
  data providers
• sorted by observation time and data center for
  ease of use
• Navy GDEM climate and climate variability
  profiles added as reference profiles in call sign
  database
• temperature and salinity climatology
• global, monthly, 0.25 degree resolution
• 78 vertical levels (47 in upper 1000 m of water
  column)
• climate mean and standard deviations interpolated
  to observed levels for month in which observation
  taken

4
GODAE Quality Control Pilot Project

• New MEDS QC Variable in Call Sign Database
• decoded MEDS hex bit codes on QC tests performed
• Meds QC Tests (binary string bit position from
  the RIGHT) Bit - Test
• 0 - Platform Identification 1 - Impossible
  Date/Time 2 - Impossible Location
• 3 - Position on Land 4 - Impossible
  Speed 5 - Impossible Sounding
• 6 - Global Impossible Parameter Values
• 7 - Regional Impossible Parameter Values
• 8 - Increasing Depth 9 - Profile Envelop
  10 - Constant Profile
• 11 - Freezing Point 12 Spike
  13 - Top and Bottom Spike
• 14 Gradient 15 - Density
  Inversion 16 - Levitus Seasonal Clim
• 17 - Emery and Dewar Clim 18 - Asheville Clim
  19 - Levitus Monthly Clim
• 20 Waterfall 21 - Cruise Track
  22 Profiles
• 23 Bottom 24 - Temperature
  Inversion
• experience indicates only two tests flagged as
  typically performed
• Levitus seasonal climate check and temperature
  inversion

5
GODAE Quality Control Pilot Project

• Plans
• initialize call sign databases with 2003-2006
  production run
• NetCDF call sign databases will be hosted on US
  GODAE server
• target date for completion Oct 2007
• Issues
• US GODAE server hardware upgrade has never taken
  place
• IBM X-346 machines plus 30 TB storage purchased
  years ago by ONR
• never installed due to security issues
  completion date still unknown
• GODAE QC application cannot run on existing
  server hardware
• cannot do real-time updates of call sign
  databases as planned
• in process of purchasing additional disk space
  for personal Linux workstation to support GODAE
  QC function
• using my Linux workstation (at best) only monthly
  updates of NetCDF call sign databases on GODAE
  server possible

6
GODAE Quality Control Pilot Project

• FSI Argo Floats Pressure Offset Errors
• discovered in February, 2007 that Argo profiles
  from SOLO floats with FSI CTD (Argo Program WHOI)
  have incorrect pressures
• profile data may be offset upward by one or more
  pressure levels, resulting in a significant cold
  bias for the data
• a data denial list was posted on the Argo web
  site (287 WMO IDs)
• data no longer distributed on GTS
• Argo science team asked if bad FSI float data had
  been accepted into operational assimilation
  systems
• FSI float QC outcomes from the NRL/FNMOC
  real-time QC system were evaluated from 2004
  through 2006
• instrumentation error checks
• climate background field check
• cross validation (buddy) check

7
float track
error probability of entire profile (reject if p
gt 0.95)
GDEM climate anomaly
cross validation anomaly
error probability of profile level (reject if p gt
0.95)
NRL FSI Float QC Outcomes Display
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
GODAE Quality Control Pilot Project

• Summary NRL/FNMOC QC of
  FSI Argo Float Pressure Errors
• rejected 11 of the float temperature profiles 
• rejected 5 of the float temperature levels
• 185 float profiles rejected due to inadequate
  vertical sampling as a result of the vertical
  level rejections (created gaps in profiles)
• Positive feedback from Howard Freeland on call
  sign databases
• From Freeland, Howard mailtoFreelandHj_at_pac.dfo-
  mpo.gc.ca Sent Wednesday, August 01, 2007 258
  PMTo Cummings, James cummings_at_nrlmry.navy.mil
  Subject Rejected floatsI think the call sign
  files are exactly what I'd like to look at.  I've
  been doing some simple investigations about data
  being rejected by less sophisticated users than
  GODAE people and have some interesting results...

12

• END

13
Call Sign Database Structure
dac source of data (FNMOC, NAVO, MEDS, Coriolis, UKMO, BMRC)
dtg observation time
dtg_qc quality observation time
lat, lon observation position
pos_qc quality observation position
n_lvl number levels
lvl observation depth levels
sal observed salinity
sal_fcst salinity forecast
s_fcst_err salinity forecast error
sal_clm salinity climatology
s_clm_err salinity climate variability
sal_qc salinity profile quality
sal_qc_lvl salinity level quality
sal_flg salinity QC test failures
tmp observed temperature
tmp_fcst temperature forecast
t_fcst_err temp forecast error
tmp_clm temperature climate
t_clm_err temp climate variability
tmp_qc temperature profile quality
tmp_qc_lvl temperature level quality
tmp_flg temperature QC test failures
sign observation call sign
upd_dtg profile update time at center
qc_conflict flag indicating QC outcomes differ among data centers
14
NRL
MEDS
Coriolis
UKMO
BMRC
GODAE QC Application Program Interface (API)

read different QC data formats, check for valid
call sign, match profiles (lat, lon, time, call
sign), populate database structure with
observation, QC codes, QC flags, and supplemental
information, sort by observation time, update
NetCDF call sign database
52007 TAO Mooring
1900305 Argo Float
A8ES9 Ship XBT
Call Sign Databases