Computer Graphics

1
Computer Graphics

• Recitation 12

2
The plan today

• Projective geometry basics
• Reminder of homogeneous coordinates
• Algebraic properties
• Epipolar geometry
• Pinhole camera model
• Fundamental matrix of two views
• Reprojection using the fundamental matrix

3
Motivation

• Applications in computer vision
• Object recognition in images
• Reconstruction of 3D models from images

4
Motivation image based rendering

• We are given several images of a 3D world.
• Want to generate images of novel views without
  reconstructing the 3D model and rendering it
• Reconstructing 3D model is a hard task
• Rendering complex 3D models under complex
  lighting environment is hard

5
Motivation image based rendering

• Input two images of a statue

6
Motivation image based rendering

• Output images of novel views, without
  reconstruction of 3D!!

7
Holes may appear

• Due to disocclusion

8
Holes may appear

• Due to discretization

9
Projective geometry

• Images of 3D world are some projective
  transformations 3D ? 2D
• Expressed in matrix (linear) form when
  homogeneous coordinates are used

10
Relation between different images
The different images are related by Fundamental
Matrices
11
Projective transformations

• A general projective transformation
• Preserves collinearity (three points on a line
  will stay on the line)
• Doesnt preserve parallelism, lengths, angles

12
Projective transformations

• Projective transformation is not linear
• Points at infinity may be mapped to finite points
  in the image and vice versa

13
Homogeneous coordinates

• Add one dimension to the representation
• Any linear transformation in homogeneous
  coordinates is in fact projective transformation

14
Homogeneous coordinates

• The representation is unique up to scale

15
Homogeneous coordinates

• Points at infinity

16
Representation of 2D lines

• ax by c 0
• Homogeneous vector (a, b, c)T
• (?a, ?b, ?c)T, ? ? 0 represents the same line
• The vector (0, 0, 1) represents the line at
  infinity

17
Incidence relation

• Point p (x, y, 1)T is on line l (a, b, c)T
  ? lTp pTl 0

18
Incidence relation

• A line through two points p1 and p2 is l p1 ?
  p2
• p1T l ltp1, p1 ? p2gt 0
• p2T l ltp2, p1 ? p2gt 0
• Therefore, both p1 and p2 lie on l

19
Lines intersection

• Two lines in the plane always intersect (maybe at
  infinite point)
• The intersection point of l1 and l2 is p l1 ?
  l2
• l1Tp ltl1, l1 ? l2gt 0
• l2Tp ltl2, l1 ? l2gt 0
• Therefore, p lies on both l1 and l2

20
Projective transformation

• Linear transformation of homogeneous coordinates
• From 3D model to 2D image 3?4 matrix

21
Simple pinhole camera model

• The camera center C is in the origin of R3
• The image plane is z f

Y
Z
f
X
22
Simple pinhole camera model

• Local coordinates in the image plane

principal point
Y
y
Z
x
X
23
Simple pinhole camera model

• The simple camera matrix

y
Iy / f y / z Iy fy/z
Iy
z
f
24
Simple pinhole camera model

• The simple camera matrix

25
General pinhole camera model

• 1) The image plane origin may be translated

Y
(px, py)
Z
X
26
General pinhole camera model

• K is called calibration matrix (internal camera
  parameters)

27
General pinhole camera model

• 2) The world coordinates may be rotated and
  translated

camera center (3?1)
rotation 3?3 matrix from world to camera
Y
(px, py)
Z
X
28
Geometry of two views
P
p
p
C
C
29
The epipoles

• e is the image of C (second camera origin) under
  the first camera

P
p
p
C
e
e
C
30
The epipoles

• Clearly, C, C, e, e, P, p, p all lie in the
  same plane!
• A plane that contains the baseline CC is called
  epipolar plane

P
p
p
C
e
e
C
31
Epipolar lines

• Epipolar plane intersects the image plane at
  epipolar line
• The epipolar line will always pass through the
  epipole

P
p
p
C
e
e
C
32
Epipolar lines

• Epipolar line l is the image of the ray CP in
  the second camera

P
Q
p
p
C
e
e
C
33
Much nicer figure
C
C
34
Another nice figure
C
C
35
Relation between corresponding points

• p is the image of P in the first camera
• p is the corresponding point in the second
  camera
• p must lie on the intersection of the image
  plane with the epipolar plane through C, p, e!

P
l
p
p
e
C
e
C
36
The basic incidence equation

• p must lie on the epipolar line in the second
  image plane
• pT l 0
• It can be shown that the epipolar line l is
• l Fp
• where F is a 3?3 matrix that depends on the
  camera matrices only

37
The fundamental matrix

• F is a 3?3 matrix
• Expresses the incidence relation
• pT F p 0
• Fp is the epipolar line,
• so we get back pT l 0

38
Properties of the fundamental matrix

• Rank 2 (because Fe 0)
• Thus, 7 degrees of freedom out of 9 coefficients
• one constraint is det F 0
• scaling multiplying F by a constant doesnt
  change anything because of homogeneity

39
Finding the fundamental matrix

• Given 2 images with unknown camera matrices
• We can find F by 7 point correspondences (gives
  us 7 equations for the coefficients of F)
• Practical algorithms find many more
  correspondences and find a least-squares solution
  more robust

40
Generating novel views
C3
e23
e21
e23
e12
e31
C2
C2
e13
C1
We have three cameras C1, C2, C3 eij denotes
intersection of CiCj with the image plane of Ci
41
Generating novel views
C3
p
e23
e21
P
e23
e12
p
p
e31
C2
C2
e13
C1
The image of P in the 3rd camera is the
intersection of the epipolar lines from C1, C2
42
The algorithm

• Given
• Fundamental matrices
• Point correspondences between the two given
  images
• This is found using optical flow algorithm
• For each correspondence p ? p find the position
  of the corresponding point p in the third view
• Copy the intensity to p

p
p
p
43
Generating novel views

• Denote the fundamental matrix from Cj to Ci by
  Fij (i, j 1, 2, 3)
• The intersection of the epipolar lines is given
  by
• p (F31 p)?(F32 p)

44
Degeneracies

• Cant reproject points on the plane of C1, C2 ,
  C3 (no proper intersection of the epipolar lines)
• Numerical problems for all points close to that
  plane
• Epipolar geometry is not defined when Ci Ci
  (only rotational movement of the camera)
• Cant reconstruct third view if C1, C2 , C3 are
  on one line

45
Good luck!