Bild 1

1
Differential-Algebraic Multiview Constraints
Anders Heyden Fredrik Nyberg Applied Mathematics
Group Malmö University Sweden heyden,fredrik.nyb
erg_at_ts.mah.se
2
Main results

• In this work we present a novel type of multiview
  constraints, called differential-algebraic
  multiview constraints.
• These constraints are useful for on-line
  structure and motion estimation, e.g. based on
  filtering techniques.
• They enable linear update of current motion
  estimate (for calibrated cameras) based on at
  least three corresponding points.

3
Contents

• The structure and motion problem
• Discrete methods
• Multilinear constraints
• Linear methods
• Continuous methods
• Motion estimation
• Structure estimation
• Hybrid methods
• Matching constraint tracking
• Differential-Algebraic matching constraints

4
The Structure and Motion Problem

• Reconstruct the three-dimensional world from a
  number of its two-dimensional perspective images
• Calculate at the same time the position and
  orientation of the camera at the different
  imaging instants
• This talk concentrates on the motion estimation
  and recursive methods aiming at real-time
  applications

5
Different camera models
Calibrated camera K known lxKR j -RtX ) y
R j Rt X, with xKy. Uncalibrated camera K
unknown lxPX ) x PX . This talk mainly
deals with calibrated cameras, but the techniques
can be applied to uncalibrated cameras as well.
6
Matrix formulation
Consider one object point X and its m images
lixiPiXi, i1, . ,m
i.e. rank(M) lt m4 .
7
The epipolar constraint
Consider minors obtained from three rows from one
image block and three rows from another
which gives the bilinear epipolar constraint
8
The trifocal constraint
Consider minors obtained from three rows from one
image block, two rows from another and two rows
from a third
which gives the trilinear constraints
9
Structure and Motion Estimation

• Use eight corresponding points in two images or
  seven corresponding points in three images to
  estimate F or T linearly.
• Extract the camera matrices from F or T (linear
  algorithms)
• Estimate 3D-coordinates of feature points using
  the camera equations (linearly), called
  intersection
• These methods ignores the nonlinearities
• In the calibrated case there are severe
  non-linearities! (cf. The Kruppa equations)

10
Motion Models

• Discrete formulation

• Continuous formulation

11
Image Acquisition Models

• Assume a calibrated standard pinhole camera model

• From discrete system formulation

• From continuous system formulation

• Assume image coordinates normalized such that the
  last homogeneous component 1

• Introduce

12
Standard Epipolar Constraint

• Discrete measurement version from

which results in
13
Continuous epipolar constraint
Start with the camera matrix equation and its
derivative
Using
Define
gives
Multiply the second equation above with
where u denotes image velocities x
14
Structure and Motion estimation from the
continuous epipolar constraint

• Use cec to estimate v and w (nonlinear!)
• Estimate v first
• Then w
• Use the motion parameters and the camera matrix
  equation to estimate the structure

15
Combining discrete and continuous measurments

• Assume relatively closely spaced discrete time
  perspective observation of a rigid object moving
  relative to a calibrated camera

• Recursively estimate both the 3D position and the
  motion parameters at time t, given the set of
  perspective measurements up to that time instant

16
The essential matrix increment
Investigate the incremental change in E
using
17
The hybrid matching constraint
Inserting the expression for the essential matrix
increment into the epipolar constraint gives
The so called hybrid epipolar constraint
18
Motion estimation from HEC

• The HEC are linear in the motion parameters, w, d
• The motion parameters, w and d, can be estimated
  from at least 6 point-matches

19
Differential-Algebraic Epipolar Constraint
Start from
20
DAEC
Use the first order approximations in Dt
and the image motion field
This results in
which implies
21
DAEC

• Computing the 4 x 4 minors of MCDEC results in
• Minors containing the first three rows give the
  standard epipolar constraint
• Minors containing two rows out of the first three
  give constraints linear in wt and dt - in total
  nine such constraints, with two linearly
  independent
• Minors containing the last three rows give
  constraints nonlinear in the parameters wt and dt

22
DAEC
where
23
Recursive motion estimation algorithm
1. Initialize motion parameters (e.g. using the
continuous epipolar constraint) 2. For each new
image estimate the motion parameters, ? and d,
linearly using DAEC and at least three
corresponding points 3. Update the motion
paramters R and b according to
4. Update the structure parameters based on the
new motion estimates 5. Goto 2
24
Extensions 1 Trifocal hybrid matching
constraints

• Minors containing only one row out of the first
  three gives the previously derived
  differential-algebraic epipolar constraint.
• Minors containing only one row out of the last
  three gives the standard discrete trifocal
  constraints.
• Minors containing two rows out of the first
  three, one row out of the middle three rows and
  two rows out of the last three rows give 27
  linear constraints in the motion parameters

25
Extensions 2 Moving Stereo Head

• Minors containing only one row out of rows 4 to
  6 gives the previously derived differential-algebr
  aic epipolar constraint between views 1 and 2.
• Minors containing only one row out of the last
  three rows gives the previously derived
  differential-algebraic epipolar constraint
  between views 2 and 1.
• Minors containing at least two rows out of row 4
  to 6 and at least two rows out of the last three
  rows, unfortunately gives either trivial
  constraints or non-linear constraints in the
  parameters.

26
Conclusion

• Recursive motion estimation algorithm based on
  DAEC.
• Only three observed object point matches needed
• Update is performed using linear constraints
  only
• First order approximations employed
• Requires image motion field information

Additional work

• Combination with structure estimation using EKF
• Error feedback step to improve estimates
• Please, see poster!

27
Thank You!
heyden_at_ts.mah.se