CS 44957495 Computer Vision Single View Metrology

1
CS 4495/7495 Computer VisionSingle View
Metrology

• Many Slides by Antonio Criminisi
• Some Slides by Frank Dellaert and Forsyth Ponce

2
What is hard?

• The imaging process introduces perspective
  distortion
• In the imaging process depth is lost

3
Height measurements
How tall is this person ?
4
Forensic Science measuring heights of suspects
Vanishing line
Reference height
Reference height
5
The Virtual Flagellation
Piero della Francescas Flagellation
6
The Virtual St. Jerome
Henry V Steinwicks St Jerome in his Study
7
The Virtual Music Lesson
Vermeers Music Lesson
8
Geometric cues photograph
9
Geometric cues paintings
Masaccios Trinity
10
Join cross product !

• Join of two lines is a pointpl1xl2
• Join of two points is a linelp1xp2

11
Automatic estimation of vanishing points and lines
12
Joining two parallel lines ?
(a,b,c)
(a,b,d)
13
Points at Infinity !
(-b,a,0)T
Line at infinity linf(0,0,1)T
j
l(a,b,c)T
i
(-b,a,0)T
14
Method 1 Determinants

• 2D Line between two 2D points
• Point x on l must be linear comb of p1 p2

15
Lines
cameras01.m
16
Method 2 Homogenous Equations

• we want lTp0 for all lines !

Overdetermined system of homogeneous equations
17
cameras03.m
18
Solving Homogeneous Equations
19
SVD Figure
AU S VT
4x3
4x4
3x3
R3P2
column space
row space
nullspace(AT)
nullspace(A)empty
20
Fundamental Theorem (Strang)
Ax
Strang Course 18.06 Linear Algebra, on MIT
OpenCourseWare, ocw.mit.edu
21
Applied to line fitting
cameras04.m
Caveat Normalization matters !
22
Automatic estimation of vanishing points and lines
RANSAC algorithm
Candidate vanishing point
23
Automatic estimation of vanishing points and lines
Output Vanishing points and lines for the 3
dominant directions.
24
A special case, planes
Observer
2D
2D
Image plane (retina, film, canvas)
World plane
A plane to plane projective transformation
25
Analysing patterns and shapes
What is the shape of the b/w floor pattern?
The floor (enlarged)
Automatically rectified floor
26
Analysing patterns and shapes
What is the (complicated) shape of the floor
pattern?
Automatically rectified floor
St. Lucy Altarpiece, D. Veneziano
27
Rectifying Slanted Views
28
Rectifying slanted views
Corrected image (front-to-parallel)
29
SVD For Homography

• Cross-product Trick
• Same optimization algorithm(Direct Linear
  Transfer)

30
Measuring distances
2.2m
2.8m
2.2m
31
3D-2D Projective mapping
Projection Matrix (3x4)
32
Projective Camera Matrix
56 DOF 11 !
33
Cross-Ratios
Take notes
ZcD(X,C) is height of camera Zd(X,X) is height
of object
34
Height measurements(Other way)
Vanishing line (horizon)
35
Measuring heights of people1. Calibration step
Vanishing line (horizon)
36
Measuring heights of people2. Measurement step
Vanishing line (horizon)
37
Measuring heights of people
Here we go !
38
Assessing geometric accuracy
Are the heights of the 2 groups of people
consistent with each other?
Flagellation, Piero della Francesca
Estimated relative heights
39
Assessing geometric accuracy
The Marriage of the Virgin, Raphael
Estimated relative heights
40
Occlusion filling

• Geometric filling by exploiting
• symmetries
• repeated regular patterns
• Texture synthesis
• repeated stochastic patterns