Outline

1
Outline

• Theoretical approaches to computer vision
• Visual perception as information processing
• Problems in Computer Vision
• Classification
• Segmentation
• Recognition
• Motion analysis

2
Visual Perception as an Inverse Problem

• Retinal images are generated by the light
  reflected from the 3-D world
• The image formation is determined by the laws of
  optics
• The area of image rendering is called computer
  graphics
• Vision as an inverse problem
• Get from optical images of scenes back to
  knowledge of the objects that gave rise to them

3
Vision as a Heuristic Process

• Visual system makes a lot of assumptions about
  the nature of the environment and conditions
  under which it is viewed
• These assumptions constrain the inverse problem
  enough to make it solvable most of the time
• The resulting solution will be veridical if the
  assumptions are true
• Vision is a heuristic process in which inferences
  are made about the most likely environmental
  condition that could have produced a given image

4
Perception as Bayesian Inference

• Images I are observations
• Scene properties S are not known
• p(S) specifies the prior knowledge about the
  scene
• The knowledge you have without looking at the
  image
• Bayes rule

5
Four Stages of Visual Processing

• Image-based stage
• Surface-based stage
• Object-based stage
• Category-based stage

6
Image-based Stages

• Most theorists agree that initial stage is not
  the only representation based on a
  two-dimensional retinal organization
• It includes image-processing operations
• Local edge and line detection
• Region detection
• Correspondence between left and right eyes
• Marr called this representation primal sketches
• Raw primal sketch
• Full primal sketch

7
Representation in Early Vision

• Local spatial/frequency representation
• The representation should be
• Local
• Orientation-tuned
• Frequency-tuned
• Gabor filters
• Wavelet transformation
• Image compression

8
Gabor Filters
9
Surface-based Stage

• Recovery of intrinsic properties of visible
  surfaces
• Surface layout
• The spatial distribution of visible surfaces
  within the 3-D environment
• Explicit surface-based representation
• 2.5-D sketch
• Intrinsic images
• Intrinsic properties to surfaces

10
Surface-Based Stage cont.

• Surface primitives
• Local patches of 2-D surface within a 3-D space
• Three-dimensional geometry
• Projective geometry
• Viewer-centered reference frame

11
Surface-Based Stage cont.

• Cues for surface representation
• Stereopsis
• Motion parallax
• Shading and shadows
• Pictorial properties
• Texture
• Size
• Shape
• Occlusion

12
Object-Based Stage

• Some form of true 3-D representation
• Includes unseen and occluded surfaces
• Explicit representations of whole objects
• Two ways of constructing object representation
• Extend the surface-based representation
• Infer 3-D objects from 2-D images

13
Object-Based Stage cont.

• Volumetric primitives
• Descriptions of truly 3-D volumes
• Three-dimensional geometry
• Geometry in 3-D space
• Object-based reference frame
• Spatial relations among the volumetric primitives
  are represented by intrinsic structures among
  volumetric structures

14
Category-Based Stage

• Final stage concerns with recovering fully the
  functional properties of objects
• Functional properties through categorization
• Properties directly from visible characteristics

15
Top-down vs. Bottom-up Processes

• Bottom-up processing
• Data driven processing
• Take a lower-level representation as input and
  create or modify a higher-level representation
• Top-down processing
• Expectation-driven processing
• Processes that take a higher-level representation
  as input and produce or modify a lower-level
  representation

16
Neural Network Approaches

• Neural networks are based on the assumptions that
  human vision depends heavily on the massively
  parallel structure of neural circuits in the
  brain
• Multiple Layer Perceptrons
• Input layer
• Hidden layer
• Output layer

17
Problems in Computer Vision

• Given a matrix of numbers representing an image,
  or a sequence of images, how to generate a
  perceptually meaningful description of the
  matrix?
• An image can be a color image, gray level image,
  or other format such as remote sensing images
• A two-dimensional matrix represents a single
  image
• A three-dimensional matrix represents a sequence
  of images
• A video sequence is a 3-D matrix
• A movie is also a 3-D matrix

18
Image Classification

• Given some types through examples, identify the
  type of a new image

19
Image Segmentation

• Partition the images into homogenous regions
• Widely studied problem
• A very difficult problem
• An important problem

20
Object Recognition

• Object recognition
• Recognize objects in a constrained environment
• Identify objects from images

21
Video Sequence Analysis

• Motion analysis
• Compute motion from images
• Motion segmentation
• Video sequence analysis
• Derive models automatically
• Enhanced TV