Optical Flow Estimation using Variational Techniques

1
Optical Flow Estimation using Variational
Techniques
Darya Frolova
2
Part I
Mathematical Preliminaries
3
Agenda

• Optical Flow - what is this?
• - different
  approaches
• - formulation of
  minimization problem for Optical Flow
• - existence of
  solution (using main theorem)

4
minimization problem
Theorem
What happens when A is not a compact? What
happens when functional F is not
continuous? Does the minimum exist?
5
New Theorem
Theorem
A is a compact set
function F is continuous on A
6
Bounded Variation -1D case
Consider
x2
a
x1
xn -1
b

Definition
f has bounded variation over
if exists const M such that
for all
7
Bounded Variation
Definition
The space of functions of bounded variation on ?
is denoted BV (?)
Total variation of function f can be
represented as follows
f
where - function f is compactly supported
(is zero outside of a compact set),
8
If f is differentiable, then
Supremum for and
Then variation of function f
Function f does not need to be differentiable
9
Bounded Variation ND case
bounded open subset, function
Variation of over
f
where
10
Bounded Variation example
11
Definitions
Banach space complete, normed linear space
Definition
Space of functions of bounded variation BV(?) is
a Banach space
Definition
Space X is said to be complete, when any Cauchy
sequence xn from X converges
Definition
Consider sequence xn from X. If
natural, such that for any m, n gt N
, then xn - Cauchy sequence
12
Definition
13
Dual Space
Definition
Let X denote a real Banach space. Dual space
of X X space of linear bounded
operators X
f X ? R
14
Dual space
Definition
X is called reflexive if ( X )
( if bidual space of X is equal to X )
Reflexive
Not reflexive
e.g. the space of sequences l8
e.g. finite-dimensional (normed) spaces,
Hilbert spaces
l8 xn
15
Topologies on X
Definition
Sequence xn from X strongly converges to
point x if
If sequence strongly converges, then it weakly
converges
16
Topologies on X
Definition
Sequence fn from X strongly converges to
f if
Definition
Sequence fn from X weakly converges to
f if
for every x from X
17
Direct Method
Problem
f X ? R , where X is a Banach space
Does the solution exist?
The proof consists of three steps , which is
called Direct Method of Variational Calculus
18
Theorem
If
functional f is coercive and lower
semicontinuous on X, X is Banach and reflexive
space
Then
functional f has its minimum on X
19
Direct Method
Step A
a) If f is coercive, then minimizing sequence
is bounded
Step B
Step C
20
Step A
f (x), x? X ? R
There exists an infimum of the set f (x),
x? X ? R
and exists a sequence ? R , which converges to
this infimum
21
Step B (a)
To prove
If f is coercive, then minimizing sequence is
bounded
Proof
So, unbounded xn cannot be minimizing
sequence.
22
Step B (b)
To prove
If X is reflexive, then there exists weakly
convergent subsequence of minimizing sequence
Proof
We proved that minimizing sequence is
bounded. If X is reflexive ( ( X ) X )
then using theorem about weak sequential
compactness, which states that
for any bounded sequence in reflexive Banach
space there exists weakly convergent subsequence
23
Step C
To prove
If f is lower semicontinuous at x0 ? x0 is
a minimum point of f
Proof
24
Part II
Optical Flow
25
Image Sequence
Sequence of images contains information about the
scene, We want to estimate motion (using
variational formulation)
26
2D motion field
3D motion field
I1
2D motion field
Projection on the image plane of the 3D velocity
of the scene
I2
Image intensity
Optical center
Motion vector - ?
27
Optical Flow
What we are able to perceive is just an apparent
motion, called Optical flow (motion,
observable only through intensity variations)
28
Optical flow-methods
29
Brightness constancy
Intensity of a point keeps constant along its
trajectory (reasonable for small displacements)
Start from point x0 at time t0. Trajectory ? (
t, x (t) )
30
Brightness constancy
Given sequence I (t, x) and time t0
Find the velocity v (x) such that
31
Aperture problem
32
Solving the aperture problem
Rigid deformations are not considered (object
moves locally in one direction)
Sensitive to noise
33
Weighted least squares
Velocities are constant in small window (spatial
neighborhood)
w (x) is a window function ( gives more
influence to the
constraint at the center of the

neighborhood than at the periphery)
Too local, no global regularity
34
Regularizing the velocity field
S (v)
A (v)
Velocity should change slowly in spatial domain
(in image plain)
Horn and Schunck
35
Discontinuities
But smoothing term does not allow to save
discontinuities
Synthetic example (method of Horn and Schunck)
Discontinuities near edges are lost
36
Discontinuity-preserving approach
Where function ? permits noise removal and edge
conservation
Black et.al, Cohen, Kumar, Balas, Tannenbaum,
Blanc-Feraud
Suter, Gupta and Prince, Guichard and Rudin
37
Discontinuity-preserving approach summary
Given sequence I (t, x)
Find velocity field v that minimizes the energy
functional E
38
Smoothing term
Function ? R ? R is strictly convex,
nondecreasing ? (0) 0 (without loss of
generality)
? (s)
39
Homogeneous term
Idea if there is no texture (there is no
gradient), then there is no
possibility to correctly estimate
the flow field
So, we may force it to be zero
Without loss of generality
40
Existence of solution
Theorem
Under the following hypotheses
41
Existence of a solution
Proof
Using direct method of Variational Calculus
Step A
Construct minimizing sequence vn for
functional E
a) If E is coercive, then minimizing sequence
is bounded
Step B
b) If BV (?) is reflexive, then there exists
weakly convergent subsequence of minimizing
sequence
If E is lower semicontinuous at point x0
then x0 is a point of minimum
Step C
42
Existence of a solution
c is bounded
?
?
?
So, functional E is coercive
43
Existence of a solution
Proof
Using direct method of Variational Calculus
Step A
Construct minimizing sequence vn for
functional E
a) If E is coercive, then minimizing sequence
is bounded
Step B
b) If BV (?) is reflexive, then there exists
weakly convergent subsequence of minimizing
sequence
If E is lower semicontinuous at point x0
then x0 is a point of minimum
Step C
44
Existence of a solution
Space of functions of bounded variation is not
reflexive ( BV ) ? BV
But it has such a property that every
bounded sequence Ij from BV (?) has a
subsequence that weakly converges to
some element I from BV (?)
45
Existence of a solution
Proof
Using direct method of Variational Calculus
Step A
Construct minimizing sequence vn for
functional E
a) If E is coercive, then minimizing sequence
is bounded
Step B
b) If BV (?) is reflexive, then there exists
weakly convergent subsequence of minimizing
sequence
If E is lower semicontinuous at point x0
then x0 is a point of minimum
Step C
46
The End