Character Animation Facial Animation

1
Character AnimationFacial Animation

• Computer Animation

2
Facial Animation

• The human face ...
• Provides a great of visual information in
  character animation
• Is the prime source of emotional information
• Facial motion, unlike other body parts, is
  primarily dependent on muscle as opposed to
  skeletal structures

3
Facial Animation

• In modeling the human face, we need to be
  concerned about four aspects
• Structure Which elements move the most and how
  to represent this in geometry
• Appearance What the face looks like
• Emotion Which facial components are involved in
  each emotional expression
• Speech Primarily lip-syncing

4
Facial Modelling

• The human face has a complex surface geometry
• The challenge in modelling a face is provide
  sufficient structural detail for those areas that
  require it, while minimizing data

5
The Basic TechniqueStep 1

• Outline major features and contours
• Note gap in mouth

6
The Basic TechniqueStep 2

• Add more contour rings
• Usually at least 2 to 3 rings

7
The Basic TechniqueStep 2

• Images can be used as templates for designing a
  character
• This is called rotoscoping
• Note how more curves are included for eye sockets
  and mouth
• This is only done for half of the face

8
The Basic TechniqueStep 3

• Add cross sections trying to keep ring like
  structure and 4 sides where possible
• Try to keep ring type connections and quads

9
The Basic TechniqueStep 4

• If needed, add more segments to define a
  ridge/lip edge and interior

10
The Basic TechniqueSteps 5-7

• Step 5
• Take this flat surface and pull it into 3d
• Step 6
• Tweak all points in 3d as needed
• Additional splices may be needed

• Step 7
• Add the back of the head and neck
• Create mirror structure and attach
• Model the underside of the chin

11
Adding Depth

• Once a flat, 2-D mesh is created using a front
  view image, various points can be raised to add
  depth
• This is called lofting
• At this stage, additional control points and
  edges can be added

12
Extending Arcs to Back

• Once this mask has been created, various curves
  are extended to form the back of the head
• Additional curves are added, to complete the
  heads mesh
• But we still only have half of a head

13
Copy, Flip Attach

• Once a half head has been created, a copy is made
• This copy is then flipped along the X-axis
  (mirrored) and attached to the original half

14
Mouth Cavity

• A mouth cavity can be modelled as a sack-like
  shape connected to the inside of the lips
• This allows for a closeup of the mouth and what
  appears to be the inside of the head
• The tongue and teeth are usually created as
  separate objects

15
Ears

• Ears are modeled separately and then merged to
  the head model, welding and fixing splines as
  needed
• The trick is to keep only the amount of detail
  required, while maintaining the proper shape

16
Eyes

• Eyes are generally modeled as spheres placed
  inside the head cavity
• Irises and such are aspects of a face's visual
  appearance

17
AppearanceFacial Features

• Often it is easier to represent facial features
  via a texture map than via geometry

18
AppearanceTextures Maps

• First, a surface geometry is created as described
  above

19
AppearanceTextures Maps

• Next, several maps are created by scanning a
  subject
• Diffuse map
• Ambient amp
• Specular map

20
AppearanceTextures Maps

• The surface is unwound, much like unrolling a
  cylinder
• The maps are applied to the surface, which is
  then rewound to its original form

21
AppearanceTextures Maps

• This technique adequately addresses simulating
  geometric features
• However, it fails with respect to skin

22
AppearanceSkin

• Real skin is translucent
• Light is scattered through the various layers,
  and reflected from a sub-layer
• Bidirectional reflectance distribution
  functions(BRDF) are often used

BRDF
No BRDF
23
Bidirectional Reflectance Distribution Function
(BRDF)?

• 4-dimensional function that defines how light is
  reflected at an opaque surface
• Takes an incoming light direction, ?i , and
  outgoing direction, ?? , both defined with
  respect to the surface normal n
• Returns the ratio of reflected radiance exiting
  along to the irradiance incident on the surface
  from ?i

24
Bidirectional Reflectance Distribution Function
(BRDF)?

• The BRDF was first defined by Edward Nicodemus in
  the mid-sixties. The modern definition
  iswhere L is the radiance, E is the
  irradiance, and ?i is the angle made between ?i
  and the surface normal, n.

25
Bidirectional Reflectance Distribution Function
(BRDF)?

• D.K. McAllister et al proposed texture maps that
  contain at each texel all the parameters of a
  Lafortune representation BRDF as a compact, but
  quite general surface appearance representation
• Their method for renders such surfaces rapidly
• They also propose a method of rendering spatial
  bi-directional reflectance distribution functions
  using prefiltered environment maps
• Only one set of maps is required for rendering
  the different BRDFs stored at each texel over the
  surface
• McAllister, D.K., et al, Efficient rendering of
  spatial bi-directional reflectance distribution
  functions, Proceedings of the ACM
  SIGGRAPH/EUROGRAPHICS conference on Graphics
  hardware, 2002, pp. 79-88.

26
Facial Action Coding System (FACS)?

• Based on the book of Carl-Herman Hjortsjö, "Man's
  Face and Mimic i.e. Facial Language"
• Originally developed by Paul Ekman and Wallace
  Friesen in 1976, to taxonomize every conceivable
  human facial expression

27
Facial Action Coding System (FACS)?

• Purpose
• To determine how the contraction of each facial
  muscle (singly and in combination with other
  muscles) changes the appearance of the face
• To identify the specific changes that occurr with
  muscular contractions and how best to
  differentiate one from another

28
Facial Action Coding System (FACS)?

• Action Units (AU) are muscles or muscle groups
  that control specific facial actions
• One can manually code nearly any anatomically
  possible facial expression, decomposing it into
  the specific AUs and their temporal segments that
  produced the expression

29
Facial Action Coding System (FACS)?

• FACS defines 32 AUs, which are a contraction or
  relaxation of one or more muscles
• It also defines a number of Action Descriptors,
  which differ from AUs

30
Facial Action Coding System (FACS)?

• FACS can be used to distinguish two types of
  smiles as follows
• insincere and voluntary Pan American smile
  contraction of zygomatic major alone
• sincere and involuntary Duchenne smile
  contraction of zygomatic major and inferior part
  of orbicularis oculi

31
Facial Action Coding System (FACS)?

• In computer animation, emotional expressions can
  be produced by interpreting FACS definitions
• That is, for a given expression, specified AUs
  can be engaged

32
Action Units

• Inner Brow RaiserFrontalis, Pars Medialis
• Outer Brow RaiserFrontalis, Pars Lateralis
• Brow Lowerer Depressor Glabellae Depressor
  Supercilli
• Upper Lid Raiser Levator Palebrae Superioris
• Cheek Raiser Orbicularis Oculi, Pars Orbitalis
• Lid Tightener Orbicularis Oculi, Pars Palebralis
  
• Lips Toward Orbicularis Oris Each Other
• Nose Wrinkler Levator Labii Superioris, Alaeque
  Nasi
• ... and so on

33
EmotionsA Little Test

• Which is
• Joy (Happiness)?
• Sadness
• Surprise
• Anger
• Disgust
• Fear
• Note direction of movement due to muscles

34
Joy (Happiness)?

• Mouth
• Horizontally stretched
• Corners raised
• Eyes
• Slightly closed
• Possible crows feet at corner
• Eyebrows relaxed

35
Sadness

• Mouth
• Corners of lips pulled down
• Raised cheeks
• Possibly dropped jaw (for agony and crying)?
• Eyes
• Upper eyelids drop
• Inner corners of eyebrows raised and brought
  together

36
Surprise

• Fear has been studied more than any other emotion
• Hard to distinguish surprise from fear
• Surprise is a temporary expression, which often
  converts into other expressions

37
Anger

• Jaw tightly clenched
• If teeth exposed, mouth has rectangular shape
• Thinning of lips
• Subtle but frequent
• Earliest sign
• Lower jaw can be forward

38
Anger

• Glaring eyes
• Raised upper eyelids
• Possibly tensed lower lids
• Muscle of difficulty
• Tighten up area around eyes
• Mental and physical difficulty cause it to
  contract
• Eyebrows lowered and brought together

39
Disgust Contempt

• Active muscles around nose
• Eyes can be relaxed compared to anger
• Can be limited to one side of the face
• Lowered eyebrows
• Raised upper lip

40
Fear

• Eyes
• Upper eyelids raised as high as possible
• Tensed lower lids
• Eyebrows raised and slightly close together
• Mouth
• Lips stretched horizontally
• Possibly dropped jaw

41
EmotionsA Little Test

• How well did you do?

Anger
Sadness
Surprise
Fear
Disgust
Joy
42
Animating Emotions

• Notice the reduction in facial detail
• Exaggeration is often needed to make the emotion
  clear to the viewer

43
Facial Animation

• Shapes/morph targets
• Bones/cages
• Skeleton-muscle systems
• Motion capture (on points on the face)?
• Knowledge based solver deformations

44
Facial AnimationShaped-Based Systems

• Offer a fast playback as well as a high degree of
  fidelity of expressions
• Involves modeling portions of the face mesh to
  approximate expressions and visemes and then
  blending the different sub meshes, known as morph
  targets or shapes
• Drawbacks
• Involve intensive manual labor
• Are specific to each character
• Must be animated by slider parameter tables.

45
Facial AnimationSkeletal Muscle Systems

• Physically-based head models
• Physical and anatomical characteristics of bones,
  tissues, and skin are simulated to provide a
  realistic appearance (e.g. spring-like
  elasticity)?
• Can be very powerful for creating realism but the
  complexity of facial structures make them
  computationally expensive, and difficult to create

46
Facial AnimationSkeletal Muscle Systems

• It may be argued that physically-based models are
  not a very efficient choice in many applications
• This does not deny the advantages of
  physically-based models and the fact that they
  can even be used within the context of
  parameterized models to provide local details
  when needed
• See Waters, Terzopoulos, Kahler, and Seidel
  (among others)?

47
Facial Animation'Envelope Bones'

• aka 'Cages'
• Commonly used in games
• Produce simple and fast models, but are not prone
  to portray subtlety

48
Facial AnimationMotion Capture

• Uses cameras placed around a subject
• Subject is generally fitted either with
  reflectors (passive motion capture) or sources
  (active motion capture) that precisely determine
  the subject's position in space
• Data recorded by the cameras is then digitized
  and converted into a three-dimensional computer
  model of the subject

49
Facial AnimationMotion Capture

• Until recently, the size of the detectors/sources
  used by motion capture systems made the
  technology inappropriate for facial capture
• However, miniaturization and other advancements
  have made motion capture a viable tool for
  computer facial animation

50
Facial AnimationMotion Capture

• Facial motion capture was used extensively in
  Polar Express by Image Metrics where hundreds
  of motion points were captured.
• This film was very accomplished and while it
  attempted to recreate realism, it was criticized
  for having fallen in the 'uncanny valley', the
  realm where animation realism is sufficient for
  human recognition but fails to convey the
  emotional message

51
Facial AnimationMotion Capture

• The main difficulties of motion capture are the
  quality of the data which may include vibration
  as well as the retargeting of the geometry of the
  points

52
Face Animation Languages

• Virtual Human Markup Language (VHML)?
• Face Modeling Language (FML)?

53
Face Animation LanguagesVHML

• First I speak with an angry voice and look
  very angry,
• but suddenly I change to look more
  surprised.

54
Face Animation LanguagesFML

• end"2000" /
• end"2000" /
• repeat"kbdF3_up"
• end"2000" event_value"F1_up" /
• end"2000" event_value"F2_up" /

55
Lip-Syncing

• Motion capture
• Dots are placed at key points on actors face
• As actor speaks, dots are tracked and positions
  recorded

56
Phonemes

• Mouth orientation based on vocal sounds
• Note sound groupings