Augmented Reality: Object Tracking and Active Appearance Model - PowerPoint PPT Presentation

1 / 32
About This Presentation
Title:

Augmented Reality: Object Tracking and Active Appearance Model

Description:

Good filtering algorithms. Adequate dynamical models. Shape/appearance models need work ... AAM model instance with shape parameters p and appearance parameters ... – PowerPoint PPT presentation

Number of Views:64
Avg rating:3.0/5.0
Slides: 33
Provided by: chan162
Category:

less

Transcript and Presenter's Notes

Title: Augmented Reality: Object Tracking and Active Appearance Model


1
Augmented RealityObject Tracking and Active
Appearance Model
  • Presented by Pat Chan
  • 01/03/2005
  • Group Meeting

2
Outline
  • Introduction to Augmented Reality
  • Object Tracking
  • Active Appearance Model (AAM)
  • Object Tracking with AAM
  • Future Direction
  • Conclusion

3
Introduction
  • An Augmented Reality system supplements the real
    world with virtual objects that appear to coexist
    in the same space as the real world
  • Properties
  • Combine real and virtual objects in a real
    environment
  • Runs interactively, and in real time
  • Registers(aligns) real and virtual objects with
    each other

4
Introduction
  • Display
  • Presenting virtual objects on real environment
  • Tracking
  • Following users and virtual objects movements
    by means of a special device or techniques
  • 3D Modeling
  • Forming virtual object
  • Registration
  • Blending real and virtual objects

5
Object Tracking
  • Visual content can be modeled as a hierarchy of
    abstractions.
  • At the first level are the raw pixels with color
    or brightness information.
  • Further processing yields features such as edges,
    corners, lines, curves, and color regions.
  • A higher abstraction layer may combine and
    interpret these features as objects and their
    attributes.

6
Object Tracking
  • Accurately tracking the users position is
    crucial for AR registration
  • The objective is to obtain an accurate estimate
    of the position (x,y) of the object tracked
  • Tracking correspondence constraints
    estimation
  • Tracking objects is a sequence of video frames is
    composed of two main stages
  • Isolation of objects from background in each
    frames
  • Association of objects in successive frames in
    order to trace them

7
Object Tracking
  • Object Tracking in image processing is usually
    based on reference image of the object, or
    properties of the objects.
  • Tracking techniques
  • Kalman filtering
  • Correlation-based tracking,
  • Change-based tracking
  • 2D layer tracking
  • tracking of articulated objects

8
Object Tracking
  • Object Tracking can be briefly divides into
    following stages
  • Input (object and camera)
  • Finding correspondence
  • Motion Estimation
  • Corrective Feedback
  • Occlusion Detection

9
Input
  • Tracking algorithms can be classified into
  • Single object Single Camera
  • Single object Multiple Cameras
  • Multiple object Single Camera
  • Multiple objects Multiple Cameras

10
Single Object Single Camera
  • Accurate camera calibration and scene model
  • Suffers from Occlusions
  • Not robust and object dependant

11
Single Object Multiple Camera
  • Accurate point correspondence between scenes
  • Occlusions can be minimized or even avoided
  • Redundant information for better estimation
  • Multiple camera Communication problem

12
Possible Solution
13
Static Point Correspondence
  • The output of the tracking stage is
  • A simple scene model is used to get real
    estimation of coordinates
  • Both Affine and Perspective models were used for
    the scene modeling
  • Static corresponding points were used for
    parameter estimation
  • Least mean squares was used to improve parameter
    estimation

14
Dynamic Point Correspondence
15
Block-Based Motion Estimation
  • Typically, in object tracking precise sub-pixel
    optical flow estimation is not needed.
  • Motion can be in the order of several pixels,
    thereby precluding use of gradient methods.
  • A simple sum of squared differences error
    criterion coupled with full search in a limited
    region around the tracking window can be applied.

16
Adaptive Window Sizing
  • Although simple block-based motion estimation may
    work reasonably well when motion is purely
    translational
  • It can lose the object if its relative size
    changes.
  • If the objects camera field of view shrinks, the
    SSD error is strongly influenced by the
    background.
  • If the objects camera field of view grows, the
    window fails to make use of entire object
    information and can slip away.

17
Four Corner Method
  • This technique divides the rectangular object
    window into 4 basic regions - each one quadrant.
  • Motion vectors are calculated for each subregion
    and each controls one of four corners.
  • Translational motion is captured by all four
    moving equally, while window size is modulated
    when motion is differential.
  • Resultant tracking window can be non-rectangular,
    i.e., any quadrilateral approximated by four
    rectangles with a shared center corner.

18
Example Four Corner Method
Synthetically generated test sequences
19
Correlative Method
  • Four corner method is strongly subject to error
    accumulation which can result in drift of one or
    more of the tracking window quadrants.
  • Once drift occurs, sizing of window is highly
    inaccurate.
  • Need a method that has some corrective feedback
    so window can converge to correct size even after
    some errors.
  • Correlation of current object features to some
    template view is one solution.

20
Correlative Method (cont)
  • Basic form of technique involves storing initial
    view of object as a reference image.
  • Block matching is performed through a combined
    interframe and correlative MSE
  • where sc(x0,y0,0) is the resized stored template
    image.
  • Furthermore, minimum correlative MSE is used to
    direct resizing of current window.

21
Example Correlative Method
22
Occlusion Detection
  • Each camera must possess an ability to assess the
    validity of its tracking (e.g. to detect
    occlusion).
  • Comparing the minimum error at each point to some
    absolute threshold is problematic since error can
    grow even when tracking is still valid.
  • Threshold must be adaptive to current conditions.
  • One solution is to use a threshold of k (constant
    gt 1) times the moving average of the MSE.
  • Thus, only steep changes in error trigger
    indication of possibly wrong tracking.

23
Improvements
  • Things can be improved
  • Good filtering algorithms
  • Adequate dynamical models
  • Shape/appearance models need work

24
Active Appearance Models (AAMs)
  • Active Appearance Models are generative models
    commonly used to model faces
  • Can also be useful for other phenomena
  • Matching object classes
  • Deformable appearance models

25
Active Appearance Models (AAMs)
  • 2D linear shape is defined by 2D triangulated
    mesh and in particular the vertex locations of
    the mesh.
  • Shape s can be expressed as a base shape s0.
  • pi are the shape parameter.
  • s0 is the mean shape and the matrices si are the
    eigenvectors corresponding to the m largest
    eigenvalues

26
Active Appearance Models (AAMs)
  • The appearance of an independent AAM is defined
    within the base mesh s0. A(u) defined over the
    pixels u ? s0
  • A(u) can be expressed as a base appearance A0(u)
    plus a linear combination of l appearance
  • Coefficients ?i are the appearance parameters.

27
Active Appearance Models (AAMs)
  • The AAM model instance with shape parameters p
    and appearance parameters ? is then created by
    warping the appearance A from the base mesh s0 to
    the model shape s.

Piecewise affine warp W(u p) (1) for any pixel
u in s0 find out which triangle it lies in, (2)
warp u with the affine warp for that triangle.
M(W(up))
28
Fitting AAMs
  • Minimize the error between I (u) and M(W(u p))
    A(u).
  • If u is a pixel in s0, then the corresponding
    pixel in the input image I is W(u p).
  • At pixel u the AAM has the appearance
  • At pixel W(u p), the input image has the
    intensity I (W(u p)).
  • Minimize the sum of squares of the difference
    between these two quantities

29
Object Tracking with AAM
  • Objects can be tracked with the trained AAM
  • 3-D face tracking with AAM search
  • Pose estimation with AAM

30
Example
  • The training set consisted of five images of a
    DAT tape cassette
  • DAT cassette was annotated using 12 landmarks
  • Upon the five training images, a two-level
    multi-scale AAM was built.

aam_tracking_mpeg4.avi
31
Future Direction
  • Propose a general object tracking algorithm with
    the help of AAM
  • Improve the accuracy of the object tracking
    algorithm
  • Improve the fitting speed of the AAM

32
Conclusion
  • Introduction on Augmented Reality
  • Survey on Object Tracking
  • Introduction Active Appearance Model
  • Improve the accuracy of object tracking by AAM
  • Proposed our future research direction
Write a Comment
User Comments (0)
About PowerShow.com