Operational%20%20%20Quality%20Control%20in%20Helsinki%20Testbed

1
Operational Quality Control in Helsinki
Testbed

• Mesoscale Atmospheric Network Workshop
• University of Helsinki, 13 February 2007
• Hannu Lahtela Heikki Turtiainen

2
What is Quality?

• The degree to which a system, component, or
  process meets (1) specified
  requirements, and (2) customer or users needs or
  expectations IEEE
• Data are of good quality when they satisfy
  stated and implied needs... such as required
  accuracy, resolution and representativeness.
  WMO Guide to Meteorological
  Instruments and Methods of Observation

3
Quality Management and Quality Control (QC)

• The purpose of quality management is to ensure
  that data meet requirements (for uncertainty,
  resolution, continuity, homogeneity,
  representativeness, timeliness, format, etc.) for
  the intended application, at a minimum
  practicable cost. Good data are not necessarily
  excellent, but it is essential that their quality
  is known and demonstrable.
• Quality control is the best known component of
  quality management systems, and it is the
  irreducible minimum of any system. It consists of
  examination of data at stations and at data
  centres to detect errors so that the data may be
  either corrected or deleted .
• WMO Guide to Meteorological Instruments and
  Methods of Observation
• ) Deleted must be understood here in the sense
  that erroneous data is not used for applications
  however, it should remain stored in the
  database, only flagged faulty.

4
Other Quality Management functions

• In addition to QC, Quality Management includes
• equipment specification and selection
• station siting and sensor exposure planning
• maintenance calibration procedures
• data acquisition and processing (sampling,
  averaging, filtering...)
• personnel training and education
• metadata management
• etc...

5
Levels of QC
6
Quality Flags

• Information about suspicious or certainly wrong
  data values detected in the QC process should be
  passed on together with an information label, or
  flag, in order to
• indicate the quality level
• inform which control methods and control levels
  data have passed
• inform about the error type if an error or
  suspicious value was found
• Such flagging information is useful both in
  quality control phases (technical flags) and for
  users of meteorological information (end-user
  flags).

7
HTB uses FMI end-user flagging system

• Four-digit code, one digit for each QC level
• HQC QC2 QC1 QC0
• 1000 100 10 1
• Value of the digit defines quality of the data
• 0 no check 4 calculated
• 1 OK 5 interpolated (spatial)
• 2 suspicious, small difference 8 missing
• 3 suspicious, large difference 9 deleted
• Example 1531
• 1 QC0 at the site is OK
• 30 QC1 found big difference (e.g. monthly
  limit exceeded)
• 500 QC2 interpolated the value using
  neighbour station data
• 1000 HQ accepted the interpolated value

8

Proposal for HTB QC process (by Jani Poutiainen)
- so far implemented only partially and with some
modifications.
9

Proposal for HTB QC process (by Jani Poutiainen)
- so far implemented only partially and with some
modifications.
10
Metman QC1 Quality Control

• Quality control of weather observations is based
  on real time quality control, containing the
  following quality control tests
• range test
• step tests ( 1hr and 3 hrs)
• persistence test
• spatial test
• At present the following observations are tested
• wind speed (10 min. average)
• barometric pressure
• air temperature
• The best fit quality control algorithms and
  recommendations by NORDKLIM (KLIMA report no
  8/2002) and Oklahoma Mesonet QC are superimposed
  on the Metman quality control process.

11
Metman QC Control Domains
Each weather stations must be part of quality
control domain. Each quality control domain
contains predetermined suspicious and erroneous
limits for each parameters needed in each test.
The values can be configured based on seasonal
climate extremes. Also meteorologically
non-representative and representative weather
stations should be located in different quality
control domains. Spatial test can be performed
only with stations located on same representative
quality control domain. In Helsinki Testbed
project all weather stations belong to one and
the same quality control domain.
However, there are some special sites that
should belong to a different domain. For example
air temperatures in Heimoonkruoppi differ
dramatically from the weather stations near by.
12
Metman QC1 process
QC1-process
Range test
data flow without qc-flag
data flow with qc-flags
erroneous
valid
Suspicious
Step tests
erroneous
valid
Suspicious
Persistence test
valid or erroneous
Suspicious
Spatial test (under testing)
valid, suspicious or erroneous
Technical flag code is stored in the MetMan
database. Four-digit end-user flag code is
composed, converted to FMML and posted to CDW
together with the observation data.
13
Metman QC Range test

• Range test is a test that determines if an
  observation lies between predetermined range.
• Erroneous ranges are based on sensor
  specifications and suspicious ranges can be
  configured based on seasonal climate extremes.
• Metman real time quality control process performs
  range test first.
• Range test doesn't need historical observations
  to perform.
• If range test
• succeeds, step test will be performed next
• fails, the rest of the tests won't be performed,
  and observation is flagged with erroneous flag
• gets suspisious value, spatial test will be
  performed

14
Metman QC Step tests

• Step tests use sequential observations (1-hour
  and 3-hours) to determine which data represent
  unrealistic 'jumps' during the observation time
  interval.
• Erroneous and suspicious step thresholds can be
  configured based on seasonal climate extremes.
• Metman real time QC process performs step tests
  after the range test.
• Step tests need historical observations to
  perform.
• If the tests
• succeed, persistence test will be performed next
• fail, the rest of the tests won't be performed,
  and observation is flagged with erroneous flag
• get suspicious value, spatial test will be
  performed

15
Metman QC Persistence test

• Persistence test analyses data on hourly basis to
  determine if observation underwent little or no
  variation.
• Metman real time quality control process
  persistence test after the step test.
• Persistence test needs historical observations to
  perform.
• If the test
• succeeds, observation is flagged with valid flag
• fails, observation is flagged with erroneous
  flag
• gets suspicious value, spatial test will be
  performed

16
Metman QC Spatial test

• Spatial test performs intercomparison between
  neighbour stations in the same quality control
  domain.
• Metman real time quality control processes
  spatial test only if one of the earlier tests
  returns suspicious value.
• Spatial test searches a nearby reference station
  and compares the parameter under test with that
  of the reference station. The reference station
  must
• belong to the same QC domain
• be sufficiently close
• have about the same altitude and installation
  heights
• the reference parameter must have passed range-,
  step- and persistence tests.
• The spatial test is currently under testing, not
  yet operational.

17
HTB QC next steps

• Extension of QC1 to all measured parameters
• Implementation of spatial test
• Availability of end-user flags through
  Researchers Interface
• Addition of technical flagging to CDW?
• Special challenge for dense mesoscale networks
• Large number of stations gt maintenance based on
  immediate response too expensive gt new methods
  and tools needed for QC, network diagnostics and
  maintenance!