Object Tracking

1
Object Tracking

• Computer Vision Course Presentation
• by Wei-Chao Chen
• April 05, 2000

2
Overview

• What is object tracking?
• Camera or/and objects are moving with respect to
  each other.
• Using knowledge of current state of objects and
  new input image(s), calculate the new position of
  objects.
• Real-time performance requirements.

3
Overview

• Applications of Object Tracking
• Behavioral Analysis of objects or human, such as
  gesture analysis, facial expression analysis,
  etc.
• Robotics such as autonomous vehicle.
• Smart Camera such as video surveillance, etc.

4
Overview
Tracking with automatic model switching
Facial expression tracking separated from pose.
5
Overview
Tracking using Hermes curve evolution and PDE for
incremental updates.
Video Surveillance with behavioral model and
Bayesian networks.
6
Overview

• Research focuses on tracking
• Feature detection
• Contour detection,
• Energy Minimization.
• Recursion algorithm.
• Adaptive modeling for enhanced prediction.
• Modeling techniques for behavioral analysis.

7
Related Topics

• Camera Position Estimation
• With moving camera with known internal
  parameters, extract the camera motion.

static object
For each i, we have
pt11
pt22
pt21
Find more than 8 points and the essential matrix
E1,2 is solved.
pt12
Ct1
Ct2
E1,2
8
Related Topics

• Optical Flow
• The velocity field of each points on the images.
• Can be used as input for object tracking.
• Search for similar flow vectors to locate the
  tracked objects.

9
Mathematics for Object Tracking

• Problem Statement
• Given certain measurements in the system, solve
  for the system states.
• Example In HiBall tracker, given the orientation
  of beacon related to the tracker (measurements),
  calculate the orientation/location of the tracker
  (system states).
• Both measurements and system states might contain
  error.

10
Mathematics for Object Tracking
uncertainty
measurement error
vi
wi-1
System
Measuring Device
ai
xi
g(..)
f(..)
previous state(s)
11
Mathematics for Object Tracking

• Notations Definitions
• ai state vector of the system at instance i.
• xi measurement vector at instance i.
• fi(xi , ai , vi ) 0 measurement equation,
  where vi is the measurement uncertainty.
• ai g(a i-1 , wi-1 ) state model of the
  system, where wi represents the uncertainty of
  the model at instance i. a.k.a. plant equation.

12
Mathematics for Object Tracking

• Deterministic least-squares method
• Given enough measurements, over-constraint the
  system to find the unique state solution.
• Recursive least-squares method
• Given under-constraint measurements and previous
  states, find the next system state.
• Kalman filtering method
• Also a recursive method.
• Optimal solution of given constraints.

13
Deterministic Methods

• Example
• For linear measurements with constant state
• f(x, a) x - Ha 0,
• Goal optimize the function
• f T (x, a) f (x, a).
• The solution of state is found by
• â (HTH)-1HT x.
• The pseudo-inverse of H results in the
  least-square solution of a.

14
Deterministic Method

• Advantage
• The method is conceptually easy.
• Disadvantage
• Need to keep records of enough numbers of
  measurements.
• Difficult to solve in practice
• large set of linear equations in the above
  example need to be solved.

15
Recursive Method

• Address the problem of deterministic method.
• We want to have incremental method.
• We dont want to keep lots of earlier states.
• We dont want to wait until all measurements
  arrives.

16
Recursive Method

• Again, let fi(xi, a) be the measurement function.

Assume f is linear and let fi xi - Hiai
0. Let
We have HiT Hi H(i-1)T Hi-1 HiT Hi Let Pi
(HiTHi)-1
17
Recursive Method
Optimize the following equation
We shall eventally get
The above equation shows the updated state âi
using only previous state âi-1 and measurement
matrix Hi.
18
Kalman Filter Method

• How about noise in the system and measurements?
• There will always be errors in measurements
• Imprecise instrumentation,
• Ambiguity,
• Finite resolution of measurements.
• Model of the system g() might not be accurate
• Imprecise modeling of system.
• Uncertainty of previously calculated state(s) due
  to imprecise measurements.
• Solution tolerate the noise.

19
Kalman Filter Method

• Kalman filter is optimal because it
  incorporates
• knowledge of the system and measurement
  device dynamics,
• the statistical description of the system
  noises, measurement errors, and uncertainty in
  the dynamics models,
• any available information about initial
  conditions of the variables of interest.

Peter S. Maybeck, Stochastic models, estimation,
and control
20
Kalman Filter Method
Fig Typical Kalman filter application
Peter S. Maybeck, Stochastic models, estimation,
and control
21
Kalman Filter Method

• Linear Kalman Filter Equations

Assume we have the following plant and
measurement equations
Error wi , vi are both assumed to be a
normal-distribution random variable.
22
Kalman Filter Method
- Define Ri, Qi Ri measurement error
covariance cov(vi) Qi state model error
covariance cov(wi) Both Ri and Qi can be
calculated incrementally. - Define Pi Pi
State estimation error covariance. Pi can be
calculated by E(ai-âi)(ai-âi)T
23
Kalman Filter Method
Linear Kalman Equations are found by optimizing
Pi
Cool... but what do they mean?
24
Kalman Filter Method
Prediction Phase
Correction Phase
25
Kalman Filter Method
measurement error
state error
K
Qi
Ri
â
â
Prediction Phase
Correction Phase
P
P
26
Kalman Filter Method
Startup of recursion requires 1. Estimated
initial states b0. 2. Weights on estimation of
b0, denoted as M0. 3. Initial error covariances
R1 and Q1. Kalman filter starts by
calculating K0 M0H(HM0HT R1)-1 and P0
(M0-1 HT(R1)-1H)-1
27
2D Token Tracking

• Input A sequence of images with known temporal
  order.
• Goal Given a set of 2D lines in one image, track
  their movements on the images.
• Assumption
• There exist some edge-detector to extract edges.
• Both camera object can move in the world.
• No internal camera parameters are required.

28
2D Token Tracking
29
2D Token Tracking

• Problem Abstraction
• Identify the system states
• Find a representation of line
• The collection of line position, size,
  orientation, their derivatives are the states.
• Error in the modeling is allowed.
• Find the measurement equation
• Found by doing edge detection and match with
  predicted line location.

30
2D Token Tracking
Represent a line with a, b, x0, l
where xayb 0 x0 Ordinate of midpoint of
line a, b Orientation of line l Length of
line The plant equation can be written as
followed
31
2D Token Tracking
Measurement function can be written as
below xi ri ri-1 Hai vi where r0 is
the predicted measurement, and H extracts a
subset of the system state as the measurements. T
o complete the modeling, we need
covariance matrices Qi and Ri. Ri is explicitly
defined with the representation of line segment,
and Qi is chosen by the user, according to the
precision of edge detector.
32
2D Token Tracking
Initialization of the process 1. Initialize
state b0 and M0 (estimated weights of b0),
assuming segments have no motion/acceleration
2. Calculate K0 and P0 and the recursion starts.
33
3D Token Tracking

• Similar to 2D, except the 3D information of
  objects need to be known.
• This implies calibrated stereo rig of cameras.
• Representation of lines and state of the system
  are directly extended from 2D to 3D.
• Edge detectors are changed to feature-based 3D
  reconstructor.

34
3D Token Tracking
35
References

• Faugera, O. Three-Dimensional Computer Vision -
  Chapter 8 Tracking Tokens over Time.
• ICCV 98
• Welch, G. and Bishop G. An Introduction to the
  Kalman Filter.