CS 39549525: Spring 2003

1
CS 395/495-25 Spring 2003

• IBMR Week 8A
• Chapter 8... Epipolar Geometry Two Cameras
• Jack Tumblin
• jet_at_cs.northwestern.edu

2
IBMR-Related Seminars

• Light Scattering ModelsFor Rendering Human Hair
• Steve Marschner, Cornell University Friday May
  23 300pm, Rm 381, CS Dept.

3
Reminders

• ProjA graded Good Job! 90,95, 110
• ProjB graded Good! minor H confusions.
• MidTerm graded novel solutions encouraged.
• ProjC due Friday, May 16 many recd...
• ProjD posted, due Friday May 30
• (Last Weeks IAC notes revised...)
• Take-Home Final Exam Thurs June 5, due June 11

4
Camera Matrix P links P3?P2

• Basic camera
• x P0 X where P0 K 0
• World-space camera
• translate X to camera location C, then
  rotate x PX (P0RT) X
• Rewrite as
• P KR -RC
• Redundant notationP M p4M KRp4 -K
  R C

5
Chapter 7 More One-Camera Details

• Full 3x4 camera matrix P maps P3world to P2 image
• ? What does it do to basic 3D world shapes?
• Forward Projection
• Line / Ray in world ? Line/Ray in image
• Ray in P3 is X(?) A ?B
• Camera changes to P2 x(?) PA ?PB

yc
A
PA
?B
p
C
f
zc
xc
6
Chapter 7 More One-Camera Details

• Full 3x4 camera matrix P maps P3world to P2 image
• ? What does it do to basic 3D world shapes?
• Backward Projection
• Line L in image ? Plane ?L in world
• Recall Line L in P2 (a 3-vector) L x1 x2
  x3T
• Plane ?L in P3 (a 4-vector)
• ?L PTL

p11 p21 p31 p12 p22 p32p13 p23 p33p14
p24 p34
x1x2x3
yc
L
zc
p
C
f
xc
7
Cameras as Protractors

• World-Space Direction D xc, yc, zc, 0T
• Direction from image point x D (KR)-1x
• Point C and points x1,x2 form angle ? (pg
  199)
• (K-TK-1) ? Image of Absolute Conic
• Line L makes plane ? with normal n KT L (in
  camera coords)

8
Cameras as Protractors

• P (??) (K-TK-1). OK. Now what was ?? again?
• Absolute Conic ?? all imaginary points on ??
• Satisfies BOTH x12 x22 x32 0 AND x42 0
• Finds right angles-- if D1 ? D2, then D1T
  ??D2 0
• Dual of absolute conic is Dual Quadric Q?
  all imaginary planes ? ? to ??, tangent to Q?
• Finds right angles-- if ?1 ? ?2, then ?1T
  Q??2 0
• can write ?? or Q? as the same matrix

1 0 0 00 1 0 00 0 1 00 0 0 0
9
Cameras as Protractors

• Clever vanishing point trick
• Perpendicular lines in image?
• Find their vanishing pts. by construction
• Use v1T? v2 0, stack, solve for ? (K-TK-1)

v3
v2
v1
10
Cameras as Protractors

• P ?? (K-TK-1) ? Image of Absolute Conic
• Just ??as has a dual Q?, ? has dual ?
• ? ?-1 K KT
• The dual conic ? is the image of Q? , so
• ? P (Q?) P( )
• Vanishing points v1,v2 of 2 ? world-space
  lines v1T? v2 0
• Vanishing lines L1, L2 of 2 ? world-space
  planes L1T? L2 0

11
Movement Detection?

• Can we do it from images only?
• 2D projective transforms often LOOK like 3-D
• External cam. calib. affects all elements of P
• YES. Camera moved if--only-ifCamera-ray points
  (C?x?X1,X2, etc) will
• map to LINE (not a point) in the other image
• Epipolar Line l image of L
• Parallax x1?x2 vector

X2
X1
x2
x1
L
x
C
l
C
12
Epipolar Geometry Chapter 8

• Basic idea
• Given ONLY images from 2 cameras C, C
• Different views of same objects X, butwe dont
  know world-space points X.
• If we choose an x, how can we find x ?
• How are x, x linked?

X
x
x
C
C
13
Epipolar Geometry Chapter 8

• Basic idea
• 2 cameras located at C, C in world space.
• Find baseline through camera centers C, C
• Baseline hits image planes at epipoles
• Notice baseline and Xform a plane...
• Many OTHER planes thru baseline ...

X
x
x
baseline
C
C
14
Epipolar Geometry Chapter 8

• Basic idea
• 2 cameras located at C, C in world space.
• Find baseline through camera centers C, C
• Baseline hits image planes at epipoles
• Family of planes thru baseline are all the
  epipolar planes
• Image of planes lines epipolar lines
• Lines intersect at epipolar points in both
  images.

x
x
baseline
C
C
15
Epipolar Geometry Chapter 8
X
epipolar plane ?
epipolar line L
epipolar line L
x
x
baseline
C
e
e
C

• Summary
• Connect cameras C, C with a baseline, which
• hits image planes at epipoles e, e.
• Chose any world pt X, then ?? everything is
  coplanar! epipolar plane ? includes image points
  x, x, and these connect to epipoles e,e by
  epipolar lines L, L

16
Epipolar Geometry
X
epipolar plane ?
epipolar line L
epipolar line L
x
x
baseline
C
e
e
C

• Useful properties
• Every image point x maps to an epipolar line
  Lalso
• Epipoles e,e each cameras view of the other
• All epipolar lines L pass through epipole e
• Epipolar Line L is (image of the C?X ray...)
• Epipolar Line L links (image of C) to (image of
  X)

17
Fundamental Matrix Fx L
X
epipolar plane ?
epipolar line L
epipolar line L
x
x
baseline
C
e
e
C

• One Matrix Summarizes ALL of Epipolar Geometry
• Fundamental Matrix F 3x3, rank 2.
• Maps image point x to image point x xT F x
  0
• but F is only Rank 2 given only x, F cannot
  find x for you!!
• Maps image point x to epipolar line L F x
  L

18
Fundamental Matrix Fx L
X
epipolar plane ?
epipolar line L
epipolar line L
X(?)
x
x
baseline
C
e
C

• 1) How do we find F?
• If we know the camera matrices P and P
• (we almost never do), book derives (pg 224)
• F e? P P ?!?!What?!?! point e cross
  product with a matrix pp ?!?!

(Recall P PT(PPT)-1, the pseudo-inverse)
19
Fundamental Matrix Fx L
X
epipolar plane ?
epipolar line L
epipolar line L
X(?)
x
x
baseline
C
e
C

• F e? P P But whats this? NEW TRICK
• Cross Product written as matrix multiply (pg.
  554)
• a ? b ?
  a?b
• Note a ? b -b ? a a?b (aTb?)T

a1 a2 a3
b1 b2 b3
a2b3 a3b2 a3b1 a1b3 a1b2 a2b1
0 -a3 a2 a3 0 -a1-a2 a1 0
b1 b2 b3
skew symmetric matrix
20
Fundamental Matrix Fx L
X
epipolar plane ?
epipolar line L
epipolar line L
X(?)
x
x
baseline
C
e
C

• 2) What is F if we DONT know the cameras P,
  P,but we DO know some corresp. point pairs (x,
  x)?
• F finds epipolar line L from point x Fx L
• (Recall that if (any) point x is on line a L,
  then xT L 0)
• Substitute Fx for L xTF x 0
• AHA! we can find F using DLT-like method! see
  Chap. 10

21
Fundamental Matrix Summary
(pg. 226)

• F is 3x3 matrix, maps P2?P2, rank 2, 7-DOF
• If world space pt X ? image space pts. x and x
  then xTF x 0
• Every image pt has epipolar line in the other
  image Fx L FTx L
• Baseline pierces image planes at epipoles e, e
• Fe 0 FTe 0

22
Fundamental Matrix Summary
(pg. 226)

• F is 3x3 matrix, maps P2?P2, rank 2, 7-DOF
• Given camera matrices P, P, find F matrix by
• F e? P P (recall e is image of C e
  PC)
• F is unaffected by any proj. transforms done on
  BOTH cameras(PH, PH) has same F matrix as (P,
  P) for any full-rank H
• (e.g. F measures camera C vs. Camera C only,
  no matter where you put them)

23
Fundamental Matrix Uses

• Special case camera translate only (no
  rotations)
• Camera matrices are P K I 0 , P KI t
  
• where K is internal calib., t is 3D translation
  vector
• F matrix simplifies to F e?
• Epipolar lines are all parallel to direction t
• x,x displacement depends only on t 3D depth z
• x x (Kt)(1/zc)

24
Fundamental Matrix Uses

• Can we find a camera matrix from motion
  fundamental F?
• Let one camera position define the worlds
  coordsP P0 K I 0 , and other is P
  M m KR -RC
• where K is internal calib., R is rotation C is
  position
• F matrix simplifies to F m?M
• If we know how we moved the camera (R,C matrices)
  then find F by correspondence and solve for K.
  (pg 237)
• No R, C matrices? Use Essential Matrix (pg 238)

25
Fundamental Matrix Properties
(pg. 226)

• Why bother with F?
• Can find it from image pt. correspondences only
• Works even for mismatched cameras (example
  100-year time-lapse of Eiffel tower)
• Choose your own world-space coordinate system.
• SVD lets us recover P0, P camera matrices from F
• (See 8.6 The Essential matrixpg 240)
• Complete 2-camera mapping from world??image
• 2 images corresponding point pairs (xi,xi)?F
• Let camera coords 3D world coords, then
  (xi,xi)?Xi

26
Correspondence Problem

• Where Computer Vision, IBMR part ways
• Fundamental Matrix Corresponding point pairs
  (x,x) ?
• How can we blunt the correspondence problem?

27
END