Structure%20from%20Motion - PowerPoint PPT Presentation

About This Presentation
Title:

Structure%20from%20Motion

Description:

Structure from Motion (With a digression on SVD) – PowerPoint PPT presentation

Number of Views:106
Avg rating:3.0/5.0
Slides: 36
Provided by: Szymo92
Category:

less

Transcript and Presenter's Notes

Title: Structure%20from%20Motion


1
Structure from Motion
  • (With a digression on SVD)

2
Structure from Motion
  • For now, static scene and moving camera
  • Equivalently, rigidly moving scene andstatic
    camera
  • Limiting case of stereo with many cameras
  • Limiting case of multiview camera calibration
    with unknown target
  • Given n points and N camera positions, have 2nN
    equations and 3n6N unknowns

3
Approaches
  • Obtaining point correspondences
  • Optical flow
  • Stereo methods correlation, feature matching
  • Solving for points and camera motion
  • Nonlinear minimization (bundle adjustment)
  • Various approximations

4
Orthographic Approximation
  • Simplest SFM case camera approximated by
    orthographic projection

Perspective
Orthographic
5
Weak Perspective
  • An orthographic assumption is sometimes well
    approximated by a telephoto lens

Weak Perspective
6
Consequences ofOrthographic Projection
  • Scene can be recovered up to scale
  • Translation perpendicular to image planecan
    never be recovered

7
Orthographic Structure from Motion
  • Method due to Tomasi Kanade, 1992
  • Assume n points in space p1 pn
  • Observed at N points in time at image coordinates
    (xij, yij)
  • Feature tracking, optical flow, etc.

8
Orthographic Structure from Motion
  • Write down matrix of data

Points ?
Frames ?
9
Orthographic Structure from Motion
  • Step 1 find translation
  • Translation parallel to viewingdirection can not
    be obtained
  • Translation perpendicular to viewing direction
    equals motion of average position of all points

10
Orthographic Structure from Motion
  • Subtract average of each row

11
Orthographic Structure from Motion
  • Step 2 try to find rotation
  • Rotation at each frame defines local coordinate
    axes , , and
  • Then

12
Orthographic Structure from Motion
  • So, can write where R is a
    rotation matrix and S is a shape matrix

13
Orthographic Structure from Motion
  • Goal is to factor
  • Before we do, observe that rank( ) 3(in
    ideal case with no noise)
  • Proof
  • Rank of R is 3 unless no rotation
  • Rank of S is 3 iff have noncoplanar points
  • Product of 2 matrices of rank 3 has rank 3
  • With noise, rank( ) might be gt 3

14
DigressionSingular Value Decomposition (SVD)
  • Handy mathematical technique that has application
    to many problems
  • Given any m?n matrix A, algorithm to find
    matrices U, V, and W such that
  • A U W VT
  • U is m?n and orthonormal
  • V is n?n and orthonormal
  • W is n?n and diagonal

15
SVD
  • Treat as black box code widely available
    (svd(A,0) in Matlab)

16
SVD
  • The wi are called the singular values of A
  • If A is singular, some of the wi will be 0
  • In general rank(A) number of nonzero wi
  • SVD is mostly unique (up to permutation of
    singular values, or if some wi are equal)

17
SVD and Inverses
  • Why is SVD so useful?
  • Application 1 inverses
  • A-1(VT)-1 W-1 U-1 V W-1 UT
  • This fails when some wi are 0
  • Its supposed to fail singular matrix
  • Pseudoinverse if wi0, set 1/wi to 0 (!)
  • Closest matrix to inverse
  • Defined for all (even non-square) matrices

18
SVD and Least Squares
  • Solving Axb by least squares
  • xpseudoinverse(A) times b
  • Compute pseudoinverse using SVD
  • Lets you see if data is singular
  • Even if not singular, ratio of max to min
    singular values (condition number) tells you how
    stable the solution will be
  • Set 1/wi to 0 if wi is small (even if not exactly
    0)

19
SVD and Eigenvectors
  • Let AUWVT, and let xi be ith column of V
  • Consider ATA xi
  • So elements of W are squared eigenvalues and
    columns of V are eigenvectors of ATA

20
SVD and Eigenvectors
  • Eigenvectors of ATA turned up in finding
    least-squares solution to Ax0
  • Solution is eigenvector corresponding to smallest
    eigenvalue

21
SVD and Matrix Similarity
  • One common definition for the norm of a matrix is
    the Frobenius norm
  • Frobenius norm can be computed from SVD
  • So changes to a matrix can be evaluated by
    looking at changes to singular values

22
SVD and Matrix Similarity
  • Suppose you want to find best rank-k
    approximation to A
  • Answer set all but the largest k singular values
    to zero
  • Can form compact representation by eliminating
    columns of U and V corresponding to zeroed wi

23
SVD and PCA
  • Principal Components Analysis (PCA)
    approximating a high-dimensional data set with a
    lower-dimensional subspace

Original axes
24
SVD and PCA
  • Data matrix with points as rows, take SVD
  • Columns of Vk are principal components
  • Value of wi gives importance of each component

25
SVD and Orthogonalization
  • The matrix U is the closest orthonormal matrix
    to A
  • Yet another useful application of the
    matrix-approximation properties of SVD
  • Much more stable numerically thanGraham-Schmidt
    orthogonalization
  • Find rotation given general affine matrix

26
End of Digression
  • Goal is to factor into R and S
  • Lets apply SVD!
  • But should have rank 3 ?all but 3 of the wi
    should be 0
  • Extract the top 3 wi, together with the
    corresponding columns of U and V

27
Factoring for Orthographic Structure from Motion
  • After extracting columns, U3 has dimensions 2N?3
    (just what we wanted for R)
  • W3V3T has dimensions 3?n (just what we wanted for
    S)
  • So, let RU3, SW3V3T

28
Affine Structure from Motion
  • The i and j entries of R are not, in general,
    unit length and perpendicular
  • We have found motion (and therefore shape)up to
    an affine transformation
  • This is the best we could do if we didntassume
    orthographic camera

29
Ensuring Orthogonality
  • Since can be factored as R S, it can also
    be factored as (RQ)(Q-1S), for any Q
  • So, search for Q such that R R Q has the
    properties we want

30
Ensuring Orthogonality
  • Want or
  • Let T QQT
  • Equations for elements of T solve byleast
    squares
  • Ambiguity add constraints

31
Ensuring Orthogonality
  • Have found T QQT
  • Find Q by taking square root of T
  • Cholesky decomposition if T is positive definite
  • General algorithms (e.g. sqrtm in Matlab)

32
Orthogonal Structure from Motion
  • Lets recap
  • Write down matrix of observations
  • Find translation from avg. position
  • Subtract translation
  • Factor matrix using SVD
  • Write down equations for orthogonalization
  • Solve using least squares, square root
  • At end, get matrix R R Q of camera
    positionsand matrix S Q-1S of 3D points

33
Results
  • Image sequence

Tomasi Kanade
34
Results
  • Tracked features

Tomasi Kanade
35
Results
  • Reconstructed shape

Front view
Top view
Tomasi Kanade
Write a Comment
User Comments (0)
About PowerShow.com