RECOGNIZING FACIAL EXPRESSIONS

1
RECOGNIZING FACIAL EXPRESSIONS THROUGH TRACKING
Salih Burak Gokturk
2
OVERVIEW

• MOTIVATION
• PREVIOUS WORK
• PROBLEM DESCRIPTION
• THEORY OF TRACKING
• TRACKING VIDEOS
• THEORY OF SVM

3
MOTIVATION
What does expression recognition mean?
Guessing the meaning of facial deformations.
What are the possible applications ?

• Any interactive scenerio.
• Video conferencing

Why do we want to use 3-D information ?

• 2-D system is very dependent on the view yet
• computationally simple.

4
Components of the recognition system
Analysis -Face Tracking

• Intelligence
• Support Vector Machine
• Classifier

Shape Parameters
5
OVERVIEW

• MOTIVATION
• PREVIOUS WORK
• PROBLEM DESCRIPTION
• THEORY OF TRACKING
• TRACKING VIDEOS
• THEORY OF SVM

6
SNAKE MODEL

• Introduced by Kass and Witkin.
• Energy Minimization Problem

7
MODEL BASED 3-D FACE TRACKING

• DeCarlo and Metaxas, 96, very accurate
  tracking.
• Eisert and Girod, 98, used in video-conferencing
  application, with efficient and accurate
  compression.
• Gokturk et. al., 00, brings a data driven
  approach where the face model is learnt from
  stereo tracking.
• Model based approaches.
• Track all the points together with n-dimensional
  freedom on the shape.
• Based on Lukas-Tomasi-Kanade optical flow
  tracker.

8
OVERVIEW

• MOTIVATION
• PREVIOUS WORK
• PROBLEM DESCRIPTION
• THEORY OF TRACKING
• TRACKING VIDEOS
• THEORY OF SVM

9
PROBLEM DESCRIPTION(Tracking 1)
10
PROBLEM DESCRIPTION(Tracking 2)
X(t)
TIME t1
I(t1)
11
PROBLEM DESCRIPTION (Recognition)
Training
Testing
Data
Classifier
New Data
Output
12
OVERVIEW

• MOTIVATION
• PREVIOUS WORK
• PROBLEM DESCRIPTION
• THEORY OF TRACKING
• TRACKING VIDEOS
• THEORY OF SVM

13
ASSUMPTIONS

• Cameras are calibrated.
• The person should move slow unless the
• camera is fast enough for motion capture.
• The mesh is initialized to the first image.
• The user performs the expressions known
• to the computer

14
Stereo Tracking
Monocular Tracking
Learn Shape
Data
- shape is learnt from stereo learning in our case

• p - degrees of freedom

15
LUKAS TOMASI KANADE OPTICAL FLOW TRACKER
Time t
Il(xi(t))
?

• - For robustness, u and v are estimated using a
  neighbourhood around the point.

Time t1
Il(xi(t1))
16
LUKAS TOMASI KANADE OPTICAL FLOW TRACKER EXTENDED
TO 3D
X(t1)
X(t)
?
TIME t1
I(t1)
17
OVERVIEW

• MOTIVATION
• PREVIOUS WORK
• PROBLEM DESCRIPTION
• THEORY OF TRACKING
• TRACKING VIDEOS
• THEORY OF SVM

18
STEREO LEARNING
19
FACE TRACKING
- The deformation space of a particular
individual is learnt - That particular individual
is tracked using a mono camera
- Tracked parameters ?, R, T.
20
UNIVERSAL FACE TRACKER

• The deformation space of a subset of people
  learnt
• - The shapes are aligned and PCA is applied on
  this set

21
OVERVIEW

• MOTIVATION
• PREVIOUS WORK
• PROBLEM DESCRIPTION
• THEORY OF TRACKING
• TRACKING VIDEOS
• THEORY OF SVM

22
Support Vector Machines (SVM)
Training
Testing
Data
Classifier
New Data
Output
- Best discriminating hyperplane between two
class of objects

• Distinguish the vectors that carry the
• relevant information (support vectors)

• if nonlinear data, map the data to
• high dimensional domain, then apply
• the SVM.

23
What needs to be done

• Choose the appropriate data input for SVM
  classifier. Using ? vector might not help. Create
  more intelligent vectors in that case.
• Choose an appropriate kernel (transformation
  function) for SVM.
• Apply SVM in a one to many fashion.
• Combination of intelligent features and SVM
  should give robust results.

Expected Contributions

• Show that 3-D model based tracking is suitable
  for further applications.
• Support vector machine is a suitable classifier
  for expression recognition