Themes and Issues in Vision

1
Themes and Issues in Vision

• Jim Rehg
• CS 4495/7495 Computer Vision
• Lecture 3
• Friday Aug 23, 2002

2
Summary From Last Class

• Characteristics
• Vision is a Constructive Process
• Vision Solves Specific Tasks in Specific Contexts
• Key Issues
• Invariance

3
Organization of 3-D Scenes

• How is it that we perceive coherent, distinct 3-D
  objects given only a 2-D photo? Especially when
• Objects may be partially occluded
• We may have had no prior experience with these
  objects

4
Vision is Fundamentally Ill-Posed

• There are an infinite number of possible scenes
  that could produce the pixels in a single image

Camera
Surface
5
Integration of Information

• Spatial information in the image must be
  integrated to perceive a coherent object

Fixations for Give the ages of the people
Fixations for What were they doing before the
guest arrived?
Fixation experiments Yarbus 1961
6
Integration of Information

• Prior knowledge must be used to weed out
  unlikely interpretations.
• Possible sources of knowledge
• Experience with real-world objects (manipulating
  objects, walking around them, etc.)
• Built-in priors (e.g. infant response to faces)

7
What is the right spatial scale for integration?
(Image courtesy of Bill Freeman)
8
What is the right spatial scale for integration?
(Image courtesy of Bill Freeman)
9
Identical local evidence...
(Courtesy of Bill Freeman)
10
different interpretations
(Courtesy of Bill Freeman)
11
Information must propagate over the image
Local information...
...must propagate
(Courtesy of Bill Freeman)
12
Summary

• Characteristics of vision
• Vision is a Constructive Process
• Vision Solves Specific Tasks in Specific Contexts
• Vision Reconstructs Plausible 3-D Interpretations
  of 2-D Images
• Key Issues
• Invariance
• Integration of spatial information
• Integration of prior knowledge

13
Perception of Function

• How do we know what objects are for?

14
Some Applications of Vision

• Object Recognition
• Photobook
• Motion Analysis
• StrikeZone
• Structure from Motion
• Special effects

15
Recognition of Objects
Grouping is affected by top-down processes
Context influences recognition of parts
16
Photobook System
First widely-used face analysis system. Developed
by Turk et. al. at MIT Media Lab
17
Video-based Tracking of a Baseball Pitch for
Broadcast Television
18
Processed
19
Result of baseball extraction
20
Stereo vision

• Triangulate on two images of the same point to
  recover depth.
• Feature matching across views
• Calibrated cameras

Left
Right
Matching correlation windows across scan lines
21
Stereo Results

• Trinocular stereo system available from Point
  Gray Research for 5K.

22
Structure from Motion
Static scene
Unconstrained motion of (hand-held) camera,
including zooming

• Generalization of stereo framework
• Unconstrained motion of single uncalibrated
  camera
• Feature tracking across motion sequence.
• Good recent results, but solutions are somewhat
  brittle.

23
Structure from motion results
Input frames (3 of 5)
Output 3-D model
(Images courtesy of Luc Van Gool)
24
Special Effects in The Matrix

• Recently used to create Bullet Time
  special-effect in The Matrix (www.whatisthematrix.
  com)
• Linear array of cameras replaces moving camera.
• Green screen makes segmentation easy.