Motion II

1
- Motion II -

• Estimation of Motion field /
• 3-D construction from motion
• Yongjik Kim

2
Outline

• Estimating the motion field
• Differential technique
• Optical flow algorithm (Trucco)
• Filling in optical flow / segmentation of
  multiple objects (Horn)
• Validity of optical flow
• Feature-based technique
• Feature matching / Feature tracking
• Recovering 3-D motion structure

3
Estimating the Motion Field

• Ref) Trucco, Chapter 8.4
• Differential Techniques based on spatial
  temporal variations of the image at all pixels.
• Matching (feature-based) techniques rely on
  special image points (features) and track them
  through frames.

4
Differential Techniques Optical Flow

• Optical Flow Algorithm (Trucco, p196)
• For each pixel p,
• we must satisfy ?I v dI/dt 0
• Assumption we assume that this equation holds
  in the neighborhood of p with constant v.
• we write this equation for a small (typically
  5x5) patch centered at p.
• Then we find least square fit of v - this is the
  calculated optical flow for pixel p.

5
Differential Techniques Optical Flow

• We assumed that ?I v dI/dt 0 holds in the
  neighborhood of p with constant v.
• Can we justify it?
• Yes In case of rigid motion, the motion field
  of a moving plane is a quadratic polynomial in
  the coordinates (x, y, f) of the image points.
• Therefore, if the object is smooth rigid, we
  can assume the motion field varies smoothly.
• cf) Trucco p187

6
Filling in Optical Flow Information

• Ref) Horn Chapter 12
• Ref) Determining Optical Flow, Horn Schunck
  (Artificial Intelligence 17(1981) p185-203)
• For any point, we can have optical flow
  information in only one direction (parallel to ?I
  orthogonal to the boundary).
• We must fill the image with these information
  from boundaries.

7
Filling in Optical Flow Information

• How?
• Two criteria
• es The optical flow must be smooth
• ec The error in the optical flow constraint
  equation must be small
• Iteratively minimize es k ec
• cf) Horn, p284

8
Filling in Optical Flow Information

• Results
• Horn, Chapter 12, p290-292
• Horn Schunck

9
Discontinuities in Optical Flow

• If the scene is composed of multiple objects,
  there are discontinuities in the optical flow.
• We need discontinuity information ( object
  segmentation) to refine optical flow.
• On the other hand, we need optical flow to find
  discontinuities.
• Solution iteratively refine both segmentation
  and optical flow

10
Validity of Optical Flow

• The optical flow equation assumes that image
  brightness remains constant. Is that valid?
• Ref) Trucco, p194
• Even with simple Lambertian reflectance, image
  brightness is constant only in case of
• pure translation, or
• when the illumination direction is parallel to
  the angular velocity (i.e. the axis of rotation).

11
Validity of Optical Flow

• Therefore, in general, the optical flow is almost
  always different from the motion field!
• The error is small if image gradient is high.

12
Feature-based Techniques

• Ref) Trucco, p198-203
• They only get sparse motion field - motion
  vectors are known only at feature points.
• Two-frame method Feature matching
• How to find features?
• Use optical flow algorithm (least square fit) -
  if the calculated optical flow v is confident
  enough (i.e., if its covariance matrix is smaller
  enough), we consider it a feature point.

13
Feature Matching (continued)

• Algorithm
• Initially set displacement field using optical
  flow alrogithm.
• For each feature point p in image 1,
• 1. Warp its neighborhood Q1 according to
  current displacement vector in order to get Q.
• 2. From Q and corresponding section Q2 of image
  B, find optical flow at p ( new displacement
  vector) and image difference between Q and Q2.
• 3. If image difference is below threshold, exit.
  Otherwise go to step 1.

14
Feature Tracking

• Multiple-frame Method Feature Tracking
• similar to feature matching, iterated between
  frame 0 1, and then 1 2, and then 2 3, and
  so on...
• use knowledge of prior frames to estimate the
  position of feature points at next frame
• cf) Trucco, p201 - see the uncertainty (white
  cross) decreasing
• cf) Trucco, p203 - correspondence problem

15
3-D Motion Structure fromSparce Motion Field

• Feature-based technique gives us only sparse
  motion field we have to extract information
  about (dense) 3-D motion field 3-D structure
  from it!
• Factorization method
• If the camera model is orthographic, and the
  motion is rigid, the (2N n) matrix of n feature
  points at N frames has at most rank 3 huge
  intercorrelation!
• Use SVD to calculate 3D motion structure.

16
Motion of Rigid Objects

• Ref) Trucco, Chapter 8.2
• Any rigid body motion, at a given instant, can
  be decomposed into translation rotation with
  respect to a given point.
• Ex) a rolling ball

17
Motion of Rigid Objects

• In particular, a rigid body motion can be
  decomposed into
• translation
• rotation about the origin in the camera reference
  frame
• cf) Trucco, p183
• From now on, we will mean by rotation a
  rotation defined like above.

18
Motion Parallax

• Imagine two points instantaneously coincident in
  the image coordinate.
• In general, their apparent motion (motion field)
  doesnt have to be the same.
• But the difference in their apprent motion will
  always point toward the epipole!
• epipole vanishing point of the motion field if
  there were no rotation
• cf) Trucco, p188-190

19
3-D Motion Structure fromDense Motion Field
(sketch)

• If we assume the motion field is continuous, we
  can use adjacent points instead of coinciding
  points to calculate direction to epipole.
• A pair of adjacent points determines a line
  intersecting epipole - with many pairs, we can
  use least square fit to find epipole.
• Once we find the epipole, we can calculate
  angular velocity then we have 3D motion.