Gesichtsanimation f

1
Gesichtsanimation für Video-konferenzanwendungen
VisionIC

• Jürgen Rurainsky, Dr. Peter Eisert
• BildsignalverarbeitungHeinrich-Hertz-Institut,
  Berlin
• 05/07/12

2
Outline

• VisionIC
• Advanced Videophone
• Communication model (phase 1)
• Face Feature Detection
• Communication model (phase 2)
• Head modeling
• Model-based Video Coding
• Analysis by Synthesis
• Application
• Animation

3
VisionIC

• Entwicklung einer intelligenten Vision-Plattform
  samt Pilotanwendungen für den Massenmarkt
• Parallel-Prozessor Architektur, CMOS-Kamera mit
  integrierter Merkmalskaskade (pulskodierte
  neuronale Netze)
• Anwendungen
• Advanced Videophone (HHI)
• Dialogmodem
• Kfz-Vollerfassung

4
Advanced Videophone

• Kommunikationstreffpunkt in einer virtuellen
  Umgebung
• Videotelefondienst mit Merkmalen einer immersiven
  Telekonferenz
• Portable und stationäre Endgeräte (PDA, Laptop,
  Multi-Kamera-System)
• 3D-Bildverarbeitungstechnologien (Analyse,
  Synthese, 3D-Modellierung, MPEG-4 Codierung)
• Kopf-Schulter-Ansichten animiert mit Facial
  Animation Parameters (FAP)
• Ziel Kommunikation mit nur einigen kbit/s (Video)

5
Communication model (phase 1)
Face Feature Detection
6
Face Feature Detection (eye)

• Face color segmentation
• Face geometry
• Binary iris template matching
• Iris diameter calculated from face geometry
  (rough assumption)
• Deformable iris mapping
• Deformable eye lid mapping with anchor point
• Eye geometry calculated from face geometry
  (assumption)

7
Face Feature Detection (mouth)

• Face geometry
• Binary mouth template matching
• Search region extraction related to face geometry
• Bézier curve fitting

• Bézier curve fitting with a deformable curve (5
  control points)
• Intensity
• Extreme values

8
Communication model (phase 2)
Face Feature Tracking
3D-Head modeling
9
Head modeling
Source Jörgen Ahlberg (Linköping University)
10
Model-based Video Coding
11
Analysis by Synthesis
model frame
camera frame
difference

• Exploits all pixels in the image
• Minimization of the image differences in an
  analysis-synthesis loop

12
Applications

• Low bit-rate video coding(UMTS, mobile
  applications, avatars)
• Virtual Conferencing(VisionIC, BMBF project)

13
Facial Animation
original sequence
synthesized sequence
bit-rate lt 1 kbit/s
14
Animation of different People
15
Danke
16
Example
17
Face Feature Detection (chin line)
18
(No Transcript)