Occlusion Tracking Using Logic Models

1
Occlusion Tracking Using Logic Models

• James H. von Brecht
• Sheshadri R. Thiruvenkadam
• Tony F. Chan

This research supported by ONR grant
N00014-06-1-0345, NSF grants DMS-0601395,
DMS-0610079 and ARO MURI grant 50363-MA-MUR
2
Tracking Under Occlusions

• Objective Video tracking of objects under
  occlusions and in complex backgrounds.
• Assumptions
• Availability of prior shape.
• Affine motion.
• Approach
• Use prior shape information to locate and
  segment object in current frame.
• Incorporate shape prior intelligently using Logic
  Models

3
Tracking Under Occlusions

• When occlusions are present, image data alone can
  be unreliable, and is not sufficient to detect
  the object of interest.
• Must incorporate some prior knowledge about the
  object for successful detection when occluded.
• Prior Shape
• Motion Model
• Other energy minimization approaches incorporate
  prior shape as an additive constraint.
• Energy IMAGE SHAPE
• Additive constraints depend upon a notion of
  scale, and hence are sensitive to undesirable
  local minima issues.
• Using Logic Models, we couple prior shape and
  image data into one energy term
• Energy IMAGE/SHAPE

SIP 2007
4
Logic Models

• Given two (registered) images, we can combine the
  objects in each image according to a pre-selected
  logical operation (AND/OR) into one segmentation
  for both images

5
Segmentation Energy

• Segmentation is achieved by minimizing a
  functional of the familiar form
• The image terms and vary depending
  upon the choice of Logic Model, and combine the
  images and in a particular way in order
  to achieve the desired segmentation.
• is the Heaviside function, is the level-set
  function used for segmentation, and is some
  regularization term. For example,
• Note that we must minimize a different energy if
  our choice of logic model were to change (must
  automate this choice for tracking).

6
Segmentation Energy

• The image terms extend the Chan-Vese segmentation
  energy to handle multiple images
• Depending upon the desired Logic Model, we have

SIP 2007
7
Incorporating Prior Shape

• By applying the correct Logic Model, we can
  utilize available prior shape information to
  locate and segment an occluded object.
• Shape prior is represented as a binary image
  i.e.
• Correct logical combination depends upon the
  similarity of the objects intensity and the
  occlusions intensity.
• Similar gt AND
• Dissimilar gt OR

Original Image/ Shape Prior
AND Model
OR Model
8
Incorporating Prior Shape

• Knowing the correct Logic Model to apply allows
  us to avoid segmentation local minima which can
  occur by simply introducing shape additively
• Additionally, using the opposite Logic Model (OR
  in the above example) by mistake results in an
  incorrect segmentation must determine correct
  model automatically!

AND Model
Chan-Vese Shape
OR Model
9
Automatic Use of Logic Models

• Given two segmentation results, from OR and
  from AND, the correct model will deviate
  least from the shape prior, and thus minimizes
  the quantity
• To enforce this in practice,we first compute
  and , check the above term for each, then
  choose the minimizer as the segmentation result.

10
Tracking With Logic Models

• We assume the object in the current frame of
  a video sequence can be represented as an affine
  transformation of the shape prior modulo
  occlusion.
• Thus, we wish to minimize our segmentation energy
  both with respect to the active contour and
  the transformation -

11
Tracking with Logic Models

• General algorithm (sequential segmentation)-
• Use the result from the previous
  frame to generate an initial guess for the
  current frame .
• Compute via gradient descent on the AND
  energy with as initial data.
• Compute via gradient descent on the OR
  energy with as initial data.
• Check for each and
  select as whichever gives a minimum.

12
Limitations and Future Work

• Naïve approach to tracking
• No explicit motion model.
• Can be sensitive to registration local minima
  when image quality is poor.
• Sequential segmentation can fail if object motion
  is too great between frames.
• Cannot handle total occlusions.
• Computationally expensive.
• Future work
• Incorporate logical framework into more
  sophisticated Bayesian-based tracking techniques,
  e.g. Particle Filtering, Kalman Filtering.
• Goal robust, real-time tracking.

13
Sample Results
14
Sample Results