Multi-scale Visual Tracking by Sequential Belief Propagation

1
Multi-scale Visual Tracking by Sequential Belief
Propagation
Gang Hua, Ying Wu Dept. Electrical Computer
Engr. Northwestern University Evanston, IL
60208 yingwu,ganghua_at_ece.northwestern.edu
2
Abrupt Motion

• sudden changes of target dynamics
• frame dropping
• large camera motion
• etc.

3
Challenges

• Most existing visual tracking methods assume
  either small motion or accurate motion models
• Abrupt motion violates them
• Hierarchical search is not enough
• Unidirectional information flow
• Error accumulation from coarse to fine
• No mechanism to recover failure in coarse scales

4
Our Idea

• Different scales provide different salient visual
  features
• Bi-directional information flow among different
  scales should help
• Different scales collaborate

5
Our Formulation

• A Markov network
• XXi ,i1..Ltarget state in different scales
• ZZi ,i1..LImage observation of the target in
  different scales
• Undirected link Potential function
  ?ij(fi(Xi),fj(Xj)),
• Directed linkObservation function Pi(ZiXi)
• The task is to infer Pi (XiZ), i1..L

Fig.1. Markov Network (MN)
6
Belief propagation (BP)

• The joint posterior
• Belief propagation Pearl88, Freeman99

7
Dynamic Markov Network

• XtXt,i ,i1..LTarget states at time t
• ZtZt,i ,i1..LImage observations at time t
• P(Xt,iXt-1,i)Dynamic model in the ith scale
• ZtZk, k1..tImage observation up to time t

Fig.2. Dynamic Markov Network (DMN) modeling
target dynamics
8
Bayesian inference in DMN

• Markovian assumption
• The Bayesian inference is
• Independent dynamics model

9
Sequential BP

• Message Passing in DMN
• Belief update in DMN

10
Sequential BP Monte Carlo

• To handle non-Gaussian densities
• Monte Carlo implementation
• A set of collaborative particle filters

11
Algorithm
12
Experiments bouncing ball

• Sudden dynamics changes fail the single particle
  filters

• The tracking result of the Sequential BP

13
Experiments dropping frames

• Dropping 9/10 of the video frames

• BP iteration at a specific time instant

14
Experiments shaking camera
15
Experiments scale changes
16
Conclusion future work

• Contributions
• A new multi-scale tracking approach
• A rigorous statistical formulation
• A sequential BP algorithm with Monte Carlo
• Future work
• Theoretic study comparison of the BP with the
  mean field variational approach
• Learning model parameters