3D reconstruction from uncalibrated images

1
3D reconstruction from uncalibrated images

• Young Ki Baik
• CV Lab

2
Contents

• Introduce
• Basic geometrical theory
• Overview 3D reconstruction
• Conditions for 3D reconstruction and Solution
• Correspondence
• Camera parameters and motion
• Results
• Experimental results and demonstration
• Future Works

3
Introduction(1)

• Mapping to images

3D point
3D object
mapping
Image plane
Camera
Camera
Camera system for obtaining images
4
Introduction(2)

• 3D reconstruction from images

• Point correspondence
• Camera parameter and motion

3D point
3D object
Camera
Camera
3D reconstruction system to make 3D object
5
3D reconstruction from uncalibrated images

• Overview

Image Sequence
Feature Extraction/ Matching
Relating Image
Projective Reconstruction
Auto-Calibration
Dense Matching
3D Model Building
6
Conditions for 3D reconstruction

• Correspondence
• Feature extraction
• Harris corner method
• SIFT method
• Scale Invariant Feature Transform
• Initial feature matching
• Template matching (Image base descriptor)
• Descriptor (SIFT-d, PCA-d, SIFT-dPCA-d, )
• Feature matching
• RANdom SAmple Consensus
• To eliminate outlier

7
Conditions for 3D reconstruction

• Correspondence
• Guide matching
• To get more correspondence
• Using previous features and Geometry information

About 2 times more correspondence
Geometry based distance value using fundamental
matrix
Correlation based cost value
8
Conditions for 3D reconstruction

• Camera parameter and motion (Using
  Self-calibration)
• Dual Absolute Conic
• Hartley 94 / Hartley 99, David Nistér IJCV 2004
  ( cheirality solution )
• Dual Absolute Quadric
• Triggs97
• M.Pollefeys et al. PAMI98, ECCV 2002, IJCV 2004

Dual Absolute Quadric
M. Pollefeys
9
Conditions for 3D reconstruction

• Constraints for self-calibration
• Constant internal parameter
• Fixed camera
• K1 K2
• Known internal parameter
• Rectangular pixel s 0
• Square pixel s 0, fx fy
• Principle point known ( ux , uy ) image center

10
Experiments and results

• Result using rig
• Rig
• Calibration using vanishing point
• DAQ (using weighted linear equation)

Using the calibration rig information
Using the manual vanishing points input
Self-calibration result using rig correspondence
only
Self-calibration result is similar to the method
using calibration rig.
11
Experiments and results

• Real scene test
• Assuming that self-calibration works well

12
Experiments and results

• Manual input to check self-calibration results
• Points Correspondence information
• Line Connection information

13
Experiments and results

• Test 1 (Pinball machine 3 images)

• Key points
• Match

Fig.1 Fig.2 Fig.3
5476 5609 8530
Fig.1-2 Fig.2-3 Fig.1-2-3
Initial match 146 196 41
RANSAC 104 124 41
Guide match 230 160 67
RANSAC 281 202 67
14
Experiments and results

• Test 2 (Mask 3 images)

• Key points
• Match

Fig.1 Fig.2 Fig.3
1837 1420 1888
Fig.1-2 Fig.2-3 Fig.1-2-3
Initial match 102 158 4
RANSAC 35 100 4
Guide match 150 258 23
RANSAC 78 186 23
15
Experiments and results

• Test 3 (Building 6 images)

• Key points
• Match

Fig.1 Fig.2 Fig.6
959 1064 1177
Fig.1-2 Fig.2-3 Fig.16
Initial match 386 377 30
RANSAC 227 254 30
Guide match 465 484 35
RANSAC 308 309 35
16
Experiments and results

• Test 4 (House 5 images)

• Key points
• Match

Fig.1 Fig.2 Fig.5
3013 3084 2873
Fig.1-2 Fig.2-3 Fig.15
Initial match 1023 973 15
RANSAC 656 716 15
Guide match 1186 1216 54
RANSAC 911 909 54
17
Future works

• Quasi-Dense matching technique and reconstruction
• To get more reliable results
• Full side 3D reconstruction
• Using attaching algorithm
• Bundle adjustment algorithm
• To reduce error
• Full 3D reconstruction system
• Dense matching and 3D modeling