Face re-orientation using ellipsoid model in video conference

1
Face re-orientation using ellipsoid model in
video conference
Hey Ben! Speak up and look at me!
eh I am!
Ben Yip Jesse S Jin The University of Sydney
2
Background

• Known since 1969
• Hardware solutions not easily accessible
• Monocular approaches
• Involves predefined face mask
• Stereo binocular approaches
• Personalized face model, feature points mapping
• Multiple viewpoints approaches
• Occluded feature points, eye gaze detection
• Our approach
• Monocular, with no face model
• Rotate ellipsoid / anti-rotate ellipsoid operation

3
Deviated angle head movement

• Camera to be placed on top of monitor
• Vertical deviated angle depends on the setting
  (e.g. 20)
• Head rotation angle is substantially less (e.g.
  7), assuming there is a pivot point
• Head pose determination

4
Rotate ellipsoid operation

• Model a downward looking head/face using
  ellipsoid, and then rotate the head upward

5
Rotate ellipsoid operation

• Model 1 models the head
• Model 2 models the face
• Measures top, left, bottom, right of the
  ellipsoid
• Estimates z values and pivot point of rotation

6
Anti-rotate ellipsoid operation

• Model the front view of the head / face, then
  anti-rotate upwards on the downward looking view
• Anti-rotate is NOT the same as rotate the inverse
  angle (because there are projections on 2D)

7
Anti-rotate ellipsoid operation
8
Anti-rotate ellipsoid operation

• Model 3 models the head
• Model 4 models the face
• The formulas are

9
Performance

• 19 feature points to compare
• Data is scaled to ES Units (eye separation units
  in MPEG-4)
• Middle point of the eyes are aligned in the
  centre for comparison
• Anti-rotate ellipsoid operation is better in
  general

10
(No Transcript)
11
Eyelids correction

• Image warping to lift up the eyelids
• Pixel operation on the eye glare transformation

12
Result

13
Result

14
Result
15
Summary

• Anti-rotate ellipsoid operation is better
• Not much differences between modeling a face or a
  head
• Advantages
• Faster than face model based approach
• No texture mapping, registration
• Foreground objects are preserved
• Result is not perfect, but acceptable

16
Thank You!
questions?