Computer Vision for Tangible User Interfaces

1
Computer Vision for Tangible User Interfaces
2
Computer Vision

• Method for extracting information from images and
  video streams
• Includes object identification and tracking
• Commonly combined with video projectors in TUI
  applications to augment physical objects with
  digital information.

3
Computer Vision

• Strengths
• Fast way to do mockups and prototypes
• Can track position and orientation of multiple
  objects
• Weaknesses
• Can be sensitive to lighting and obstructions
• Not portable
• Often not robust
• Computationally expensive

4
Java Media Framework

• A Java library that will interface with most web
  cams
• Makes it easy to capture still frames from a
  video stream
• Sample code available on the compTUI website
• http//java.sun.com/products/java-media/jmf/

5
ARToolkit

• Open source library
• Tracks the position and orientation of physical
  markers
• Real time performance
• Markers use simple black squares
• Pattern matching allows any marker patterns to be
  used
• http//www.hitl.washington.edu/artoolkit/

6
Intels Open Source Computer Vision Library
(opencv)

• Real time computer vision library
• Open source
• Object Identification
• Face Recognition
• Gesture Recognition
• Motion Tracking
• Tracks the position and orientation of physical
  markers
• Real time performance
• http//www.intel.com/technology/computing/opencv/

7
TopCodes

• Tangible Object Placement Codes
• Designed for fast and accurate computer vision
• Each TopCode encodes a 13-bit number
• Can track position, size, and angular orientation
  of multiple objects
• Self-calibrating
• 100 Java
• Code available on the compTUI website