Motion analysis

1
Motion analysis

• By Horst Haussecker and Hagen Spies

2
Applications of optical flow

• Motion detection
• Motion compensation
• Motion-based data compression
• 3-D scene reconstruction
• Autonomous navigation

3
Introduction

• From sequences of 2-D images the only accessible
  motion parameter is the optical flow, f, an
  approximation of the 2-D motion field u, on the
  image sensor.
• The motion field is given as the projection of
  the 3-D motion of points in the scene onto the
  image sensor.

4
Difficulties

• Difference in the optical flow and the real
  motion field.
• A priori assumption on brightness changes, object
  properties, and the relation between relative 3-D
  scene motion and the projection onto the 2-D
  image sensor are necessary for quantitative scene
  analysis.
• Transparent overlay of multiple motions,
  occlusions, illumination changes, nonrigid
  motion, stop-and-shoot motion, low
  signal-to-noise (SNR) levels, aperture problem
  and correspondence problem

5
Example Sphere
6

• The simple example with the sphere shows that
  errors have to be detected and quantified.
• Camera calibration is an important step towards
  quantitative image analysis.

7
Optical flow

• Moving patterns cause temporal variations of the
  image brightness. The relationship between
  brightness changes and the optical field f
  constitutes the basis for a variety of
  approaches, such as deferential, spatiotemporal
  energy-based, tensor-based, and phase-based
  techniques. Analyzing the relationship between
  the temporal variations of image intensity or the
  spatiotemporal frequency distribution in the
  Fourier domain serves as an attempt to estimate
  the optical field.

8
Brightness change constraint

• A common assumption on optical flow is that the
  image brightness g(x,t) at a point xx,yT at
  time t should only change because of motion.
  Thus, the total time derivative
• needs to equal zero.
• Aperture problem
• One equation and two unknowns

9
Aperture problem
10
Optical flow in spatiotemporal images

• Instead of restricting the analysis to two
  consecutive images the brightness pattern g(x,t)
  can be extended in both space and time, forming a
  3-D spatiotemporal image.
• Let r r1, r2, r3 be the
  vector pointing into the direction of constant
  brightness within the 3-D xt-domain.
• The optical flow computation is reduced to an
  orientation analysis in spatiotemporal images,
  that is, an estimate of the 3-D vector r.

11
Examples of spatiotemporal images for synthetic
test patterns moving with constant velocity
12
Motion constraint in Fourier domain

• Let g(x,t) be an image sequence of any pattern
  moving with constant velocity, causing the
  optical flow f at any point in the image plane.
  The resulting spatiotemporal structure can be
  described by
• The spatiotemporal Fourier transform g(k,w) is
  given by
• where g(k) is the spatial Fourier transform.
  Fourier spectrum of a pattern moving with
  constant velocity condenses to a plane in Fourier
  space.

13
The equation of the plane in Fourier domain is
given by the argument of the delta distribution
in the previous equation
Taking the derivatives of w(k,f) with respect to
kx and ky yields both components of the optical
flow
The Fourier transform does not necessarily have
to be applied to the whole image. For local
estimates, multiplication with an appropriate
window function prior to transformation restricts
the spectrum to a local neighborhood. It is,
however, not possible to perform a Fourier
transformation for a single pixel. The smaller
the window, the more blurred the spectrum becomes
14
Spatiotemporal frequency domain
15
Sampling theorem

• How fast patterns of a certain size are allowed
  to move is given by the sampling theorem
• It is not the size of the object, but rather the
  smallest wave number contained in the Fourier
  spectrum of the object that is the limiting
  factor. A large disk-shaped object can suffer
  from temporal aliasing right at its edge, where
  high wave numbers are located.

16
Correspondence and flow

• Correspondence problem
• Aperture and sampling theorem are special cases
  of the general correspondence problem
• Flow versus correspondence
• Correspondence-based techniques are less
  sensitive to illumination changes. They are also
  capable of estimating long-range displacements of
  distinct features that violate the temporal
  sampling theorem.
• Correlation-based approaches are extremely
  sensitive to periodic structures.
• If the temporal sampling theorem can be assured
  to be fulfilled, optical flow based techniques
  are generally the better choice. In other cases,
  when large displacements of small structures are
  expected, correlation-based approaches usually
  perform better.