Observational Data Standard, v 1.0

1
Observational Data Standard, v 1.0
A Provisional Standard to Facilitate Data
Sharing and Aggregation

• Lynn Kutner, Donna Reynolds, Jennifer Nichols,
  Kristin Barker, Kristin Snow

April 2007
2
Working Group

• NatureServe
• Lynn Kutner
• Carol Fogelsong
• Donna Reynolds
• Jennifer Nichols
• Bruce Stein
• Rickie White
• Keith Carr
• Kelly Gravuer
• Larry Master
• Kristin Barker

• Member Programs
• Karen Cieminski (MN)
• Karen Walker (MT)
• Pete Sorrill (ON)
• John Fleckenstein (WA)
• Nicole Firlotte (MB)
• Jim Morefield (NV)
• Tim Howard (NY)
• Melanie Arnett (WY)
• Partners
• Parks Canada
• Canadian Wildlife Service
• Cornell Lab of Ornithology
• Larry Morse

3
Reviewers

• Member Programs Alberta, British Columbia,
  Connecticut, Idaho, Maine, Manitoba, Maryland,
  Massachusetts, Minnesota, Montana, Nevada, New
  Hampshire, New York, Ontario, Pennsylvania,
  Saskatchewan, Texas, Vermont, Virginia,
  Washington, West Virginia, Wyoming
• NatureServe Central
• NatureServe Canada
• Canadian Wildlife Service
• Cornell Lab of Ornithology
• Information Center for the Environment (ICE),
  University of California, Davis
• Parks Canada
• U.S. National Park Service
• VegBank
• Washington Department of Fish and Wildlife

4
Development of the Standard
5
Process

• Funded by the Gordon and Betty Moore Foundation
• NatureServe is a co-sponsor of the Taxonomic
  Databases Working Group (TDWG) Observational Data
  Subgroup this is a provisional standard as an
  input for that groups work towards an
  international observational data standard
• Multi-institutional Observation Working Group
• Compared 16 observation databases to identify
  breadth and commonalities among attributes
  Salvias, ABCD, VegBank, Darwin Core2, Member
  Program custom databases
• Project began mid-2005. Version 1.0 published
  September 2006
• Initial implementation in NatureServes Kestrel
• Revisions expected

6
What is an Observation?

• An observation characterizes evidence for
  the presence or absence of an organism or set of
  organisms through a data collection event at a
  location.

7
Goals Requirements

• Useableminimal requirements, simple
• Extensible, e.g., discipline-specific data
• Universal
• core set of concepts, broadly applicable, for
  data sharing and aggregation
• compatible with (and incorporating) existing
  standards
• Software-independent
• Accommodate
• species AND ecological communities (inc. plots)
• historical data
• different protocols
• negative (absence) data
• data quality and validation attributes
• monitoring

8
Overview of the Standard

• http//www.natureserve.org/prodServices/obsStandar
  d.jsp

9
Major Entities

• Observation
• Observation Grouping
• Survey
• Species List
• Project (group of surveys)
• Protocol (methods)
• Search Area
• External Documentation

10
Ranking of Attributes

• Required cannot be null, but in some cases can
  be unknown
• Priority Core should always be recorded if
  possible
• Core important but not required
• (not always available or
  applicable)
• Additional supplementary data

11
Attribute Table
12
Observation Required Attributes

• What Scientific name
• Calypte anna or
• Quercus velutina / Ilex opaca Forest or
• a community
• Where GIS shape, coordinates, or text
• New Mexico Eddy County, 3 mi S of Artesia along
  Rt. 285
• When Date (to year, at least)
• 30 November 1978
• Who Observer Name(s) or Unknown
• D.J. Davis

13
Observation What

• Scientific Name
• Author of scientific name
• Concept reference name used in concept ref
• Name Type taxon or ecological community
• Identification Confidence high, medium, low
• Concept Fit extent to which observed community
  fits with published concept
• Secondary Designation other communities / taxa
  that this observation may relate to
• Verbatim Scientific Name
• GUID global universal identifier of a taxonomic
  concept

14
Observation Where

• Link to GIS feature (point, line, polygon)
• Latitude, Longitude, Datum
• Location Description (text)
• OR
• Country
• Mapping Accuracy distance within which the
  location of the observation has believed to have
  been captured
• Location confidence high, medium, low
• Location fuzzed indicator and/or description of
  process if data have been randomized /
  generalized

15
Observation When

• Date of the observation includes either full
  date, partial date, or range of dates
• Date Accuracy proposed values accurate within
  1 day, 1 week, 1 month, 3 months, 1 year, 3
  years, 10 years, gt10 years
• Observation Start Time and End Time

16
Observation Who

• Observer Name full name(s) of the person(s) who
  collected data
• Observer Affiliation
• Observer Postal Address, Phone Number, E-mail
  Address
• Observer Role primary observer, observer,
  collector, submitter, verifier

17
Observation How

• Protocol protocol used to collect data
• Found Indicator used to indicate negative
  (absence) data
• Confidence confidence in search result (High,
  Medium, Low)
• Level of Effort effort expended on a given
  visit
• Additional Inventory Needed Is additional
  inventory of the area needed for this element? (Y
  / N)
• Evidence Type sample values Specimen,
  Sighting, Literature, Photograph, Tracks
• Evidence Comments

Observation Evidence
18
Observation Biology

• Number of Individuals (species)
• Estimated / Observed whether number is estimate
  or actual count
• Reproductive Evidence element reproducing at
  the location? (Y / N)
• Condition of Element description of the quality
  of element (e.g., health, alive or dead)
• Density / Distribution distribution of the
  element on the landscape (e.g., solitary
  individual, patchy, scattered, solid cover)
• Strata vegetation strata in the community
• Origin sample values Native, Nonindigenous,
  Unknown/Undetermined
• Invasiveness Comments notes about the degree of
  invasiveness

19
Observation Environment

• Habitat Description text description of the
  local or surrounding habitat
• IUCN Habitat Category habitat type, selected
  from a picklist
• Condition of Site condition of the surroundings
  (e.g., flooded, burned, etc.)
• Weather description of the weather conditions
  at the time of the visit
• IUCN Threat Category primary threat, selected
  from a picklist
• Threat Comments notes about primary or other
  threats
• Management Activities activities conducted
  during a particular visit (e.g., pulling or
  pesticides applied to invasives)
• Management Needs most important management
  needs (for enhancement or control) at this
  observation site

20
Other Information

• Data Sensitive Is the observation information
  sensitive? (Y / N)
• Reason Data Sensitive primary reason why data
  are sensitive
• General Comments notes about observation not
  addressed elsewhere
• Internal Notes comments or issues about the
  observation that are internal to the organization
  that created the record
• Associated Element Scientific Name
• Associated Element Origin sample values Native,
  Nonindigenous, Unknown/Undetermined
• Associated Element Relationship Comments

Associated Elements
21
Major Entities

• Observation
• Observation Grouping
• Survey
• Species List
• Project (group of surveys)
• Search Area (inc. plot) - GIS Shape
• Protocol (methods)
• Documentation

22
Observation Grouping

• Name of Grouping
• Criteria the common characteristic or other
  criteria used to group the observations (same
  element, same location, or any other criteria)
• Owner the person who created the observation
  grouping
• Monitoring Comments changes over time and
  trends within the grouping

23
Species List

• For community plot data
• observation ID
• search area ID (plot)
• scientific name / concept
• stratum
• cover class
• percent cover
• number (per species)
• May make an observation record for each species,
  but not required

24
Next Steps

• Current and future software development projects
  at NatureServe
• Kestrel
• Handheld field data collection tool - funding
  received from NSF
• XML schema, mapping to existing TDWG standards
• TDWG Observational Data Subgroup

25
Kestrel NatureServes Web Application for
Observations Data Management

• XML-based
• Web interface
• Extensible Data model is dynamically generated
• Libraries and Templates
• Open Source

26
(No Transcript)
27
Kestrel Architecture
28
Application Framework
Kestrel Web User Interface
KestrelXML Web Service
Oracle Database
29
Observation Standard Support

• Core Attributes
• Who, what, where, when
• Managed as relational fields
• Extended Attributes
• Managed as XML

30
Attribute Extensibility

• Individual Attributes defined in XML
• Help to describe observation or survey
• Non-trivial effort
• Signed by author, optionally by organization

31
Example Attribute ObservationType
32
Example Attribute Acidity
33
Attribute Collection

• Group of Extended Attributes
• Strongly associated with observation or survey
• Signed by author/organization
• Work together
• Ad-hoc OR
• Protocol support, e.g.
• Heritage Methodology
• CI Rapid Assessment
• Citizen Bird Survey

34
Timeline

• Parks Canada June 2007 very basic
  functionality
• NSF Handheld Data Collection Tool June 2009
  accommodate plots, multivalued data, etc.

35
Questions / Discussion