TwoView Geometry

1
Two-View Geometry

• Jim Rehg
• CS 4495/7495
• Many slides by Frank Dellaert

2
Outline

• Intro
• Camera Review
• Stereo triangulation
• Geometry of 2 views
• Essential Matrix
• Fundamental Matrix
• Estimating E/F from point-matches

3
Why Consider Multiple Views?
P'
P
X
x'
x
Answer To extract 3D structure via
triangulation.
4
Stereo Rig
Top View
Matches on Scanlines

• Convenient when searching for correspondences.

5
Feature Matching !
6
Real World Challenges
Bad News Good correspondences are hard to find

• Good news Geometry constrains possible
  correspondences.
• 4 DOF between x and x' only 3 DOF in X.
• Constraint is manifest in the Fundamental matrix
• F can be calculated either from camera matrices
  or a set of good correspondences.

7
Geometry of 2 views ?

• What if we do not know R,t ?
• Caveat
• My exposition follows book conventions
• but more intuitive (IMHO)
• Different from Hartley Zisserman !
• FP use RT-RTt camera matrices
• HZ uses Rt

8
Epipolar Geometry
P
p
p
C
C
MI0
MRT-RTt
9
Image of Camera Center
epipole
MI0
MRT-RTt
10
ExampleCameras Point at Each Other
Top View
Epipolar Lines
11
Epipoles

• Camera Center C in first view
• Origin C in second view

12
Image of Camera Ray ?
epipole
MI0
MRT-RTt
13
Point at infinity

• Given p, what is corresponding point at infinity
  x 0 ?
• Answer for any camera MAa
• A-1 Infinite homography
• In our case MRT-RTt

14
Sidebar Infinite Homographies

• Homography between
• image plane
• plane at infinity
• Navigation by the stars
• Image of stars function of rotation R only
  !
• Traveling on a sphere rotates viewer

15
Essential Matrix
16
Epipolar Line Calculation

• 1) Point 1 epipole et
• 2) Point 2 point at infinity
• 3) Epipolar line join of points 1 and 2

17
Epipolar Lines
p
C
C
MRT-RTt
MI0
18
Epipolar lines
p?Rp
et
19
Epipolar Plane
P
p
p
C
et
e
C
MRT-RTt
MI0
20
Essential Matrix

• mapping from p to l
• E 33 matrix
• Because p is on l, we have

21
Es Degrees of Freedom

• R,t 6 DOF
• However, scale ambiguity !
• 5 DOF

22
Fundamental Matrix
FUNDAMENTAL
23
Uncalibrated Case
p
C
C
MKRT-RTt
MKI0
24
Uncalibrated Case, Forsyth Ponce Version
Fundamental Matrix (Faugeras and Luong, 1992)
25
Fundamental Matrix

• mapping from p to l
• F 33 matrix
• Because p is on l, we have

26
Properties of the Fundamental Matrix

• Fp is the epipolar line associated with p.
• FTp is the epipolar line associated with p.
• FTe0 and Fe0.
• F is singular.

27
Estimation of Fundamental Matrix

• Given point correspondences, we can estimate F
  and solve for camera geometry
• Linear solution known as 8-point algorithm, due
  to Longuet-Higgins (1981)
• Naïve implementation can be numerically unstable
• 8-point algorithm revived by Hartley (1995) via
  proper numerical conditioning
• Nonlinear solution by Luong et. al. (1993)

28
The Eight-Point Algorithm (Longuet-Higgins, 1981)
29
The Normalized Eight-Point Algorithm (Hartley,
1995)

• Center the image data at the origin, and scale
  it so the mean squared distance between the
  origin and the data points is 2 pixels
• qi T pi qi T pi.
• Use the eight-point algorithm to compute F from
  the points qi and qi.
• Enforce the rank-2 constraint.
• Output T-1F T.

30
Non-Linear Least-Squares Approach (Luong et al.,
1993)
Minimize
with respect to the coefficients of F , using an
appropriate rank-2 parameterization.
31
Fundamental Matrix in Point Matching

• Given discrete feature points in two-frames, the
  set of all possible matches must satisfy the
  Fundamental Matrix
• But, how do we enforce this constraint when
• We dont know F, and
• We dont know the correct correspondences needed
  to compute F ?

32
RANSAC for Point Matching

• A standard solution is provided by the RANSAC
  algorithm
• Basic idea is to sample a set of point
  correspondences
• Use the sample to estimate a motion model
  (mapping between 2 views)
• e.g. translation, homography, Fundamental matrix,
  etc.
• See whether the remaining points can provide
  support for this solution

33
Example Mosaicking with homographies
www.cs.cmu.edu/dellaert/mosaicking
34
Omnidirectional example
Images by Branislav Micusik, Tomas Pajdla,
cmp.felk.cvut.cz/ demos/Fishepip/
35
Basic RANSAC

• Fitting a straight line

36
Discard Outliers

• No point with dgtt
• RANSAC
• RANdom SAmple Consensus
• Fischler Bolles 1981
• Copes with a large proportion of outliers

37
Main Idea

• Select 2 points at random
• Fit a line
• Support number of inliers
• Line with most inliers wins

38
Why will this work ?
The best line has the most support More support
-gt Better fit
39
In General

• Fit a more general model
• Sample minimal subset
• Translation ?
• Homography ?
• Fundamental Matrix ?

40
RANSAC

• Objective
• Robust fit of a model to data S
• Algorithm
• Randomly select s points
• Instantiate a model
• Get consensus set Si
• If SigtT, terminate and return model
• Repeat for N trials, return model with max Si

41
Additional RANSAC Topics

• Given a noise model for the points
• We can compute the distance threshold at a
  desired significance level
• We can compute the expected number of samples
  needed to find the correct model
• We can also sample adaptively

42
Additional Details on RANSAC

• The remaining slides are optional material, they
  will not be included in the exams

43
Distance Threshold

• Requires noise distribution
• Gaussian noise with ?
• Chi-squared distribution with DOF m
• 95 cumulative
• Line, F m1, t3.84 ?2
• Translation, homography m2, t5.99\ ?2
• I.e. -gt 95 prob that dltt is inlier

44
How many samples ?

• We want at least one sample with all inliers
• Cant guarantee probability p
• E.g. p 0.99

45
Calculate N

• If w proportion of inliers 1-etha
• P(sample with all inliers)ws
• P(sample with an outlier)1-ws
• P(N samples an outlier)(1-ws)N
• We want P(N samples an outlier)lt1-p
• (1-ws)Nlt1-p
• Ngtlog(1-p)/log(1-ws)

46
Example

• P0.99
• s2, etha5 gt N2
• s2, etha50 gt N17
• s4, etha5 gt N3
• s4, etha50 gt N72
• s8, etha5 gt N5
• s8, etha50 gt N1177

47
Remarks

• N f(etha), not the number of points
• N increases steeply with s

48
Threshold T

• Remember terminate if SigtT
• Rule of thumb T ? inliers
• So, T(1-etha)n

49
Adaptive N

• When etha is unknown ?
• Start with etha50, Ninf
• Repeat
• Sample s, fit model
• -gt update etha as outliers/n
• -gt set Nf(etha,s,p)
• Terminate when N samples seen