Facial Animation

1
Facial Animation

• Wilson Chang
• Paul Salmon
• April 9, 1999
• Computer Animation
• University of Wisconsin-Madison

2
Papers Used

• Bregler C.,Covell M.,Slaney M., Video Rewrite
  Driving Visual Speech with Audio. In SIGGRAPH 97
  Conference Proceedings. ACM SIGGRAPH, August 1997
• Guenter B.,Grimm C.,Wood D., Malvar H., Pighin F,
  Making Faces. In SIGGRAPH 98 Conference
  Proceedings. ACM SIGGRAPH, July 1998
• Pighin F, Hecker J., Lischinski D., Szeliski R.,
  Salesin D., Synthesizing Realistic Facial
  Expressions from Photographs. In SIGGRAPH 1998.
• Waters K., A Muscle Model for Animating
  Three-Dimensional Facial Expression. In SIGGRAPH
  1987.

3
Motivation

• Creation of Virtual Characters
• Teleconferencing Video Compression
• Simulated Movement
• Facial Surgery Planning

4
Three general catagories

• 2-D Facial Model
• 3-D Facial Model
• Muscular Model

5
Why facial animation is hard.

• Humans are very good at reading expressions.
• Any slight deviation from a correct expression
  will be immediately noticed.
• Deep-rooted instinct.

6
2-D Facial Animation

• Video Rewrite - modify and sync an actors lip
  motion to a new soundtrack.
• Keyframe approach.
• Uses vision techniques to track mouth movement.

7
Video Rewrite registration

• Hand annotation of 26 images with 54 eigenpoints
  each.
• Morph pairs to 351 images.
• Learn eigenpoint model.
• Warp images to standard reference plane.
• Eigenpoint analysis.

8
Audio Analysis

• Video Rewrite uses TIMIT speech database.
• Triphones - emphasize middle.
• teapot /SIL-T-IY/, /T-IY-P/, /IY-P-AA/,
  /P-AA-T/, /AA-T-SIL/

9
Video Synthesis

• Triphone Footage selection
• error ?Dp (1- ?)Ds
• Dp poneme-context distance.
• Ds distance between lip shapes.
• Overall Lip Width Height
• Inner Lip Height
• Height of Visible Teeth

10
Finish Synthesis

• Compress and Stretch video.
• Align and blend mouth to face.

11
Results

• Good Sync and natural articulation.
• Missing Triphones result in unnatural speech

12
Making Faces

• Motion capture.
• 3D mesh via Cyberware scanner.
• Deformed by
• Position of 128 Dots
• Manual identification - 1st frame
• Tracked by vision techniques
• Texture Extraction
• Dot removal.
• Cylindrical map.

13
Synthesizing Realistic Facial Expressions from
Photographs

• 3D facial models derived from photographs.
• Smooth transitioning between model expressions.
• Adaptation from one model to another.

14
Model Fitting

• Generic 3D mesh model.
• Pose Recovery - using multiple subject views
• Identify feature points.
• Deduce camera pose.
• Iteratively refine the generic face model.

15
Model Fitting

• Scattered Data Interpolation
• Interpolate mesh between feature points.
• Uses radial basis functions.
• Correspondence based shape refinement
• Use less accurate correspondences.
• Polylines for eyebrows, eyelids, lips, etc.
• Not used in pose processing due to error.

16
Texture Extraction

• View independent vs View dependent.
• Weight maps- bias selection of original
  photograph
• Self-occlusion.
• Smoothness.
• Positional certainty.
• View similarity.

17
View Dependent Texture Extraction

• Select best photographs.
• Draw model for each photograph.
• Blend rendered image.
• Pros
• adds detail.
• Cons
• sensitive to original photo.
• More memory, slower.

18
View Independent Texture Extraction

• Blend photographs to form single texture.
• Map onto virtual cylinder.
• Blurry

Dependent
Independent
19
Special Case Textures

• Fine Detail - hair.
• Occlusion - eyes, teeth.
• Intricate Projection - ears.
• Shadowing - eyes, teeth
• Solutions
• Use photo with highest visibility.
• Simulate shadowing

20
Expression Morphing

• Simplified by common mesh.
• Linearly interpolated vertices.
• Blend result of rendering with each texture.
• Synthesize new expressions via
• Global blend.
• Regional blend.
• Painterly interface.

21
Results

• Morphed expressions with different human
  subjects

22
Muscular Modeling

• Easy generalized across models.
• 22 muscle groups
• Facial Action Coding System (Ekman, Wallace) -
  Action Unit parameterization

23
Anatomy
24
Skin as Mesh

• Nodal mobility
• Tensile Strength of skin
• Proximity to muscle attachment
• Depth of tissue proximity to bone
• Elasticity interaction with other muscles
• Network of springs
• p F/k

25
Mesh expression examples
26
Muscle types modeled

• Linear/parallel muscles
• Sphincter muscles

27
Linear/parallel muscles
28
Sphincter muscles
29
Animating

• Not in paper
• Build a library
• Abstract language
• Keyframe

30
Our conclusions

• Good results between models.
• Relatively inexpensive equipment.
• Notable manual processing.