What%20we%20didn

1
What we didnt have time for

• CS664 Lecture 26
• Thursday 12/02/04
• Some slides c/o Dan Huttenlocher, Stefano Soatto,
  Sebastian Thrun

2
Administrivia

• Final project is due at noon on Friday 12/17
• Write-up only (5MB max)
• Be sure to include some pictures
• Send me email if you missed any quiz for a good
  reason

3
Outline

• Geometry
• Graph-based segmentation
• Statistics

4
Geometry
5
Homogeneous coordinates

• Identify a point in the image plane with ray
  passing through that point (pixel)
• (x,y) (? x, ? y, ?) for non-zero ?
• (X,Y,Z) (X/Z,Y/Z,1) for non-zero Z

6
Advantages

• Many non-linear operations become linear in
  homogeneous coordinates
• Example (X,Y,Z) projects to (fX/Z,fY/Z)

7
Camera projection matrix
8
Epipolar geometry
epipolar line
epipolar plane
Stefano Soatto (c) 2002
epipole
9
Pencil of planes

• Different epipolar planes for different scene
  points x
• Plane defined by camera origins x

10
Epipolar lines are important

• For pixel p in I there is a corresponding
  epipolar line in I
• This allows us to limit the search!
• Generalization of stereo to arbitrary camera
  positions
• Classical stereo has parallel cameras

11
Example verged stereo
12
Examples motion
Parallel toImage Plane
Forward
13
Essential matrix E

• Ex is perpendicular to xs epipolar line in the
  other image
• So if x corresponds to x then
• xTEx 0
• Captures the scene geometry
• We assume the cameras are calibrated
• Otherwise we get the fundamental matrix

14
Estimating the geometry

• The essential matrix has 5 parameters
• Can estimate from 5 corresponding points
• Fundamental matrix has 7
• The question of how few perfect correspondences
  do you need has spawned an unfortunately large
  literature

15
Yet more optimization

• We can estimate the essential matrix from a bunch
  of point matches
• A similar technique can be used to compute
  structure from motion
• Bundle adjustment

16
RANSAC (line fitting)

• Variant of generate-and-test
• Pick a small set of points at random
• Fit them via least squares
• Points far from this line are outliers
• Repeat until you find a line with very few
  outliers

17
RANSAC (camera geometry)

• Pick a small set of corresponding pixels
• At least 5 (essential) or 7 (fundamental)
• Compute the matrix from these
• See how many corresponding pixels this matrix
  explains

18
Graph-based Segmentation
19
Segmentation by min cut
Image Pixels
From Khurram Hassan-Shafique CAP5415 Computer
Vision 2003
20
Min cuts dont segment well
Slide from Khurram Hassan-Shafique CAP5415
Computer Vision 2003
21
Normalized cuts

• Instead of the min cut, minimize
• Measure of dis-similarity between the sets A and
  B
• NP-hard to minimize
• Rely on continuous approximation

22
Normalized cuts examples
23
Limitations of normalized cuts

• Works by binary partitioning
• Slow and memory-intensive
• Textured backgrounds are problems

24
Other graph-based methods

• Many other variants on min cuts
• Typical cuts, nested cuts, etc.
• No clear winner for segmentation
• Perhaps mean shift?

25
MST-based segmentation

• Minimum spanning tree is the cheapest way to
  connect all pixels into a single component (or
  region)
• Merge two components when the cheapest edge
  between them is cheap compared to a measure of
  the internal variation
• Provably good segmentation under a fairly natural
  definition
• Neither too coarse nor too fine

26
Example output

• Solves many problems with normalized cuts

27
More statistics
28
Dimensionality reduction

• We can represent orange points only by their v1
  coordinate

29
Eigenfaces

• An n-pixel image is a point in ltn
• Find low-dimensional representation of face
  images (from a training set)
• Recognition by finding the closest face in face
  space

30
Markov Random Fields
image pixels (vertices)
neighborhood relationships (n-links)
MRF defining property
Hammersley-Clifford Theorem
31
MAP estimation of an MRF
32
Energy minimization