Title: Computer Vision
1Computer Vision
Some slides from K. H. Shafique
http//www.cs.ucf.edu/courses/cap6411/cap5415/
and T. Darrell
2Correspondence
Time t
Time t dt
3Motion Field vs. Optical Flow
- Scene flow 3D velocities of scene points.
- Motion field 2D projection of scene flow.
- Optical flow Approximation of motion field
derived from apparent motion of brightness
patterns in image.
4Motion Field vs. Optical Flow
- Consider perfectly uniform sphere rotating in
front of camera. - Motion field follows surface points.
- Optical flow is zero since motion is not visible.
- Now consider stationary sphere with moving light
source. - Motion field is zero.
- But optical flow results from changing shading.
But, in general, optical flow is a reliable
indicator of motion field.
5Applications
- Object tracking
- Video compression
- Structure from motion
- Segmentation
- Correct for camera jitter (stabilization)
- Combining overlapping images (panoramic image
construction)
6Optical Flow Problem
- How to estimate pixel motion from one image to
another?
7Computing Optical Flow
- Assumption 1 Brightness is constant.
- Assumption 2 Motion is small.
(from Taylor series expansion)
8Computing Optical Flow
In the limit as u and v goes to zero, the
equation becomes exact
(optical flow equation)
9Computing Optical Flow
- At each pixel, we have one equation, two
unknowns. - This means that only the flow component in the
gradient direction can be determined.
(optical flow equation)
The motion is parallel to the edge, and it cannot
be determined.
This is called the aperture problem.
10Computing Optical Flow
- We need more constraints.
- The most commonly used assumption is that optical
flow changes smoothly locally.
11Computing Optical Flow
- One method The (u,v) is constant within a small
neighborhood of a pixel.
Optical flow equation
Use a 5x5 window
Two unknowns, 25 equation !
12Computing Optical Flow
- Find minimum least squares solution
Lucas Kanade method
13Computing Optical Flow
- When is This Solvable?
- ATA should be invertible
- ATA should not be too small. Other wise noise
will be amplified when inverted.)
14Computing Optical Flow
- What are the potential causes of errors in this
procedure? - Brightness constancy is not satisfied
- The motion is not small
- A point does not move like its neighbors
- window size is too large
15Improving accuracy
- Recall our small motion assumption
- This is not exact
- To do better, we need to add higher order terms
back in
- This is a polynomial root finding problem
- Can solve using Newtons method
- Also known as Newton-Raphson method
- Lucas-Kanade method does one iteration of
Newtons method - Better results are obtained via more iterations
16Iterative Refinement
- Iterative Lucas-Kanade Algorithm
- Estimate velocity at each pixel by solving
Lucas-Kanade equations - Warp H towards I using the estimated flow field
- - use image warping techniques
- Repeat until convergence
17Revisiting the small motion assumption
- What can we do when the motion is not small?
18Reduce the resolution!
19Coarse-to-fine optical flow estimation
20Coarse-to-fine optical flow estimation
run iterative L-K
21Multi-resolution Lucas-Kanade Algorithm
22Block-Based Motion Estimation
23Block-Based Motion Estimation
24Block-Based Motion Estimation
Hierarchical search
25Parametric (Global) Motion
- Sometimes few parameters are enough to represent
the motion of whole image
translation
rotation
scale
26Parametric (Global) Motion
27Parametric (Global) Motion
28Parametric (Global) Motion
29Iterative Refinement
- Iterative Estimation
- Estimate parameters by solving the linear system.
- Warp H towards I using the estimated flow field
- - use image warping techniques
- Repeat until convergence or a fixed number of
iterations
30Coarse-to-fine global flow estimation
Compute Flow Iteratively
31Global Flow
Find features Match features Fit parametric model
Application Mosaic construction
32Image Warping
33Image Warping
Interpolate to find the intensity at (x,y)
Pixel values are known in this image
Pixel values are to found in this image
34Other Parametric Motion Models
Perspective
and
Pseudo-Perspective Approximation to perspective.
(Planar scene Perspective projection)