Classifying Facial Actions

1
Classifying Facial Actions

• Cory Marqusee, UD
• Department of Electrical and Computer Engineering
• University of California at San Diego
• From the IEEE Transactions on Pattern Analysis
• and Machine Vision Volume 21, Number 10
• Donato, et. al, October 1999

2
Overview

• The Facial Action Coding System
• The Image Database
• Optical Flow Analysis
• Holistic Analysis
• Local Representations
• Naive and Expert Human Coders

3
The Facial Action Coding System

• Based on facial actions (FACS), or component
  motions.
• Human experts
• require 100 hrs. of training
• take 1 hr. to score a 1 min. video
• Areas of use
• Basic human research.
• Human-computer interaction systems.
• Previous automated vision systems required dots
  attached to the face.

4
The Facial Action Coding System

• Facial signals are derived in terms of component
  motions, or facial actions (FACS).
• Ekman and Friesen define 46 Action Units (AUs)
  for each independent motion of the fade.
• Each expression is composed of a set of AUs.
• Upper facial muscles with Aus 1, 2, 4, 6 and 7
  illustrated.

5
Classified Facial Actions
6
Image Database

• 24 Subjects, 150 distinct actions.
• 1,100 sequences of 6 images.
• From a neutral expression to high magnitude
  muscle contraction.
• Divided between upper and lower face categories.
• 640 x 480 pixel images were used.
• Preprocessing
• Faces were aligned.
• Eyes were rotated to horizontal.
• Scaled, and then cropped in 60 x 90 pixels
  containing either region of interest (upper or
  lower face).
• Eye and mouth centers were warped to coincide
  across all images to remove variations in facial
  shape.

7
Optical Flow Analysis

• Local Velocity Extraction
• Local Smoothing
• Classification

8
Local Velocity Extraction

• A sequence of three images are needed at t
• Flow Field
• Response distributions

9
Local Smoothing

• Optical Estimate U
• Computer Iteratively using
• Cycle of iterations stops when

10
Classification and Results

• S(novel feature vector, training feature vector)
• Many systems employ regularization procedures.
• Smoothed optic flow gave poor results
• High spatial resolution optic flow is required.

11
Optic Flow for AU1 (no smoothing)
12
Holistic Analysis

• Principal Component Analysis EigenActions
• Local Feature Analysis (LFA)
• Sparsification of LFA
• Independent Component Analysis

13
Principal Component Analysis

• Principal components obtained by calculating the
  eignevectors of the covariance matrix, S, of the
  delta-images, X.

14
Principal Component Analysis
15
Local Feature Extraction
16
Sparsification of LFA

• Images are projected into a subspace in which the
  classes are maximally separated.

17
Independent Component Analysis

• A generalization of PCAs that learn the
  high-order moments.
• Pixel gray values (edges)
• Shape
• Curvature

18
Independent Component Analysis
19
Local Representations

• Local PCA
• Gabor Wavelet Representation
• PCA Jets

20
Local PCA
21
Gabor Wavelet Representation
22
PCA Jets
23
Naive and Expert Human Coders

• Naive Subjects
• 10 volunteers with no prior knowledge on
  expression measurement.
• Given 30-60 min.
• Experts
• Given 114 upper face and 93 lover face images.
• Given 20-75 min.