Beyond%20BirdSpot%20learnings%20and%20a%20proposal

1
Beyond BirdSpotlearnings and a proposal

• L. Shyamal
• LSHYAMAL_at_GMAIL.COM
• 16 Nov 2005

2
What BirdSpot was meant to do

• Compile records of birds from all available
  published material seen to build a true picture
  of distribution in a database system making it
  amenable for searches and analyses
• thereby also making it a bibliography of bird
  reports
• Start collecting trip reports to build more
  accurate snapshots of birdlife in various parts
  of the country - provide reliable measures of
  commonness (frequency of sighting) and seasonal
  status
• trip report being lists of all bird species seen
  on a particular trip to a location at a
  particular point of time along with estimates of
  time spent in field
• Generate simple visualizations of the data and
  generate reports
• Ensure quality of data by user a feeling of
  ownership for the data and by keeping it offline
• most users have no motivation to store junk /
  frivolous data on their own systems
• Ensure preservation of the data safe by
  replicating across a user community
• Data sharing achieved on a peer to peer basis so
  that users can build their own compilation of
  data based on their judgement of accuracy of
  observers - sharing via email

3
The implementation

• Visual Basic 6.0 - MS Access 97
• User interface - guiding principles
• high data/ink ratio (Edward Tufte)
• high interaction (Ben Schniederman / SpotFire
  inspired)
• Database schema
• separation of the Trip concept from stray records
  concept
• use of a single linear taxonomic definition list
• Installer
• Inno Setup
• MDAC installation not sufficiently well
  integrated

4
The database schema
5
Data entry

• Initially data entered using MS Excel
• Modified using a data grid
• data grid modifications - short lists of species
• checking of input data using database
• Fuzzy text mining and matching (Fixit) - not
  integrated
• Current data contents almost entirely entered by
  author
• data entry has taken over at least 5 years for
  the current status

6
Current User Interface Geographic view

• Gets list of species for location or region
  marked along with consolidated reporting rates
• Generates HTML report
• Shows sources of information
• Background maps can be changed
• Map can be customized
• Rectangular and lasso selector supported for
  region marking
• Tool tips provide lat-long info
• Maps have to be orthonormal raster images of
  fixed size
• Dialog itself is not resizable - inability to
  manage layout easily

Help and other items relegated to a flat strip at
the bottom
7
Current User Interface Species view

• Gets locations for a selected species
• Generates HTML report
• Shows sources of information
• Background maps can be changed
• Map can be customized
• Month range highlighting
• Shading of spots based on report count
• Gridding and rank based spot generation

Help and other items relegated to a flat strip at
the bottom
8
Current User Interface Data entry

• Initially data entered using MS Excel
• Modified using a data grid
• data grid modifications
• short lists of species
• checking of input data using database
• Fuzzy text mining and matching (Fixit) - not
  integrated
• Current data contents almost entirely entered by
  author
• data entry has taken over at least 5 years for
  the current status

9
Current User Interface Exporter

• Separate application
• Generates a text output with multiple sections
• file needs to be emailed to configured database
  compiler
• Has not be used by many users

10
Current User Interface Importer

• Separate application
• Parses text file sections and inserts into
  database
• No checking for duplication done
• Has not be used by many users

11
Suggestions, feedback and problems

• Unable to resize window
• How do I store only my data ?
• Near zero data contribution from user base
• Failure to handle multiple species treatments
• Lack of support for reassignment or editing of
  reports
• Lack of support for version history maintenance
• Installation troubles especially due to MDAC
• Difficulty of georeferencing
• How do I modify this to work with
  butterflies//... ?
• Cant this also have photographs and
  identification information ?
• Slow speed of operation especially ranking and
  gridding
• Failure to reflect changes immediately
  (requirement to flatten data)

12
What is needed

• Web based system with backend database
• Taxonomic subsystem
• User subsystem
• Database maintenance
• Species information
• Multimedia storage
• Reporting and analysis
• Utilities for database synchronization via web
  services
• Offline data entry and analysis tools
• Support for linking with multimedia database and
  wiki style identification and species material
• Participation - data entry, compilation of
  species information, contribution of images,
  recordings, text and reference material
  compilation from existing reference material
• Prevention of large scale data loss, by allowing
  all users to synchronize various parts of the
  data into offline systems
• Ensure high quality data by ensuring that users
  first enter into their personal offline systems
  and then synchronize the data with that on server
• All new users are registered through references
  or invitations given by existing users

13
Backend database

• Species concept
• Should essentially support the idea of species
  being a named set of organisms
• Should support the idea of multiple sets of
  species - overlapping, non-overlapping, equal and
  containment
• Should support the idea of nested hierarchies of
  these sets for higher taxon levels
• Should provide tools for marking and updating of
  lists and taxonomic concepts (see TDWG, Taxonomic
  Concept Transfer Schema)
• User management
• Should track user activities
• Database control
• Should maintain database history and support
  undo/rollback
• Should automatically taint bad records or users
• data grid modifications - short lists of species
• Should aid in easier data entry
• Should try to emphasise trip lists and dissuade
  stray record entry except for rarer species
• Should support synchronization, version control,
  merging and diff-ing apart from ease of
  replication

14
Web Interfaces

• Querying
• temporal brushing
• spatial querying
• Reporting
• maps
• Seasonal histograms
• latitudinal trends, gridding, smoothing,
  contouring
• Data entry
• Fuzzy text matching, text mining (extracting from
  emails)
• User management
• Browsing
• Image upload
• Authoring

15
Offline client utilities

• Taxonomy editor
• Database sychronization
• Data entry
• Data analysis
• Secure authenticated communication with website
• Allow low bandwidth users to work with breaks in
  communications
• Offline operations and as web-based application
  should have similar capabilities