ViPER Video Annotation and Performance Evaluation

1
ViPERVideo Annotation and Performance Evaluation

• viper-toolkit.sf.net

2
The ViPER Toolkit

• Unified video performance evaluation resource,
  including
• ViPER-GT a Java toolkit for marking up videos
  with truth data.
• ViPER-PE a command line tool for comparing
  truth data to result data.
• A set of scripts for running several sets of
  results with different options and generating
  graphs.

3
The Video Performance Evaluation Resource
Schema Mapping
Metrics
Filters
Ground Truth Editor
Truth Data
Performance Evaluation Tool
Video Analysis Algorithm
Result Data
Video Analysis Algorithm
Result Data
Video Analysis Algorithm
Result Data
Evaluation Results
4
ViPERGround Truth Editing

• viper-toolkit.sf.net

5
Ground Truth Editing
6
Ground Truth Editing

• Lets the user specify the task and the
  interpretation.
• Improved (Verson 4)
• Schema editor.
• Timeline view.
• Supports undo/redo.
• All new video annotation widget.
• MPEG-2 support on Windows.

7
Latest Version in Series
8
Time Line View

• Provides summary and editing of annotation.
• Very useful for tracking annotation, great for
  event or activity annotation.

9
Time Line View

• Provides summary of ground truth.
• Direct manipulation.
• Quick editing of activities, events, and other
  descriptors.
• Some ability to modify descriptors with dynamic
  attributes directly, if not the attribute values.
• Feedback for indirect manipulation.
• Easier to notice massive changes.

10
Play When Valid Feature

• New feature divides movie into valid segments.
• Useful for dividing movie and for playback.
• Combined with Generate I-Frame Descriptor
  plug-in, supports i-frame-only mark-up.

11
Enhanced Keyboard Editing

• Support for real-time mark-up of events and
  activities.
• Keys for creating and deleting activities.
• Keys for controlling rate of display (jog dials).
• Enhance mark-up of spatial data.
• Keys for creating, editing of a single
  descriptor's attribute.
• Overall attempt to minimize effort in a GOMS
  model.
• Mouse events unnecessary except for polygon
  editing.

12
Video Frame View Enhancements

• Display with Respect To
• Play back the movie with a spatial object
  remaining static and the video frame shifting
  behind it.
• Good for editing several things that move
  together.
• Enhanced editing
• More direct manipulation controls.
• Zooming enhancements, including zoom history,
  scroll-wheel-zoom, and center-click-panning.

13
Schema Editor

• Works on live data.
• Dont have to muck about with the file format as
  much.

14
Demonstration
15
ViPER-GT Internals
ViPER-GT A Video Ground Truth Annotation Tool
ViPER Metadata API
AppLoader Plug-In Manager
Jena
Core GT API
Pure Java MPEG Decoder
Schema Editor
Plug-Ins
Plug-Ins
Plug-Ins
Native Decoders VirtualDub QuickTime JMF
16

• VideoAnnEx
• IBM AlphaWorks MPEG-7 Editor
• OntoLog (OWL)
• Jon Hegglands RDF Video Ontology Editor
• Informedia
• CMU Digital Video Library
• PhotoStuff
• Still image annotation for the semantic web
• ANVIL
• Any one-off tool a developer has created to test
  their stuff