George Goudelis

1
Exploiting dynamic video and 3D information for
face verification
George Goudelis
Department of Informatics Aristotle University of
Thessaloniki GREECE and Eurecom Sofia Antipolis
FRANCE
e-mail goudelis_at_aiia.csd.auth.gr
2
Introduction

• Using 3D face models for face recognition\verific
  ation is one of the latest and still emerging
  topics on the field of face biometrics
• Many researchers argue on how effective it is to
  use 3D models instead of 2D images while others
  disagree about if better results can be obtained
  by the use of 3D data.
• In our work we try to investigate the above
  matter and give an idea about how better (if
  there is any), the two different kind of data is
  against each other

3
3-D Data

• The extra information that 3D data contain
  comparing to 2D is depth which is usually
  represented on the z-axes
• A lot of researchers work using the 2.5D
  information
• 2.5D is a simplified 3D (x, y, z) surface
  representation that contains at most one depth
  value (z direction) for every point in the (x, y)
  plane

4
3D Data Advantages

• Independence of pose and distance (metrics of the
  face are always correct)
• More information of the model (depth information)
  
• 2D systems have no break fake resistance due to
  their nature. They can be easily fooled by
  presenting a simple print-out picture in front of
  the camera
• Good resolution of the 2D image is required.

5
3D Disadvantages

• The quality of data affects the precision of mesh
  model and further affects the recognition
  accuracy
• Other factors such as glasses, hairstyle,
  moustache and respirator, affect the 3D capture
  and further the recognition performance
• Quite intrusive. Enrolment procedure requires
  collaboration from the subject
• Time consuming. 3D representations require time

6
3D Models Capture

• In our investigation of information content of 3D
  data in comparison with 2D the Geometrix -face
  vision system was used
• Geometrix is a face scanner that uses two stereo
  cameras (up and down)
• The stereo images produced are used for the 3D
  representation of the object

7
Geometrix- Face Vision System

• In order to capture the model the user must sit
  at a fixed distance of about 1 meter from the
  camera and stay stable for about 4 seconds
• Two lights are attached to the camera and
  eliminate the problems occurred by the different
  lighting conditions
• The software needs 30 to 40 seconds to calculate
  the 3D face model, the 3D shape and the 3D meshes
• The model is obtained from two 2D images

8
Database

• For the needs of our experiments a database was
  specially created
• The database consists of 20 persons x 2 sessions
  and video
• Images of many subjects with and without glasses
  have been captured
• In video all subjects have been asked to turn
  their heads in all possible directions

9
Database

• The database was incorporated in a larger
  database of 50 persons made by Eurecom
• The video was captured by a conventional web cam
• The video is used for the extraction of the
  required poses for face verification
• An automated method for pose extraction has been
  developed

10
Database samples
11
Pose Extraction Method

• The method extracts automatically the poses that
  are classified according to the angle of view.
• While a person is moving his head in different
  directions the method identifies the pose
• It uses the Mutual Information shared between two
  frames to identify and extract any required pose.
• An image taken from a different session
  representing a pose is compared using mutual
  information with each of the frames within the
  video sequence

12
Pose Extraction Method

• The method has been tested on XM2VTS video
  database
• Experiments on 120 subjects (4 sessions each)
  show that the algorithm is able to extract the
  99,8 percent of the requested poses while it
  show to be robust in small variations of scaling
  and illumination

13
Comparing 3D to 2DExperimental Procedure

• The depth maps of the 3D models where extracted
• From video, frontal pose and profiles were
  extracted as well
• Verification results were performed using
  eigenfaces
• The eigenvectors for the 2D images and the depth
  maps were calculated respectively
• The results are fused

14
Verification Results3D vs 2D Multi-view
ROC curve for frontal plus profile
15
Verification Results3D vs 2D Multi-view

• ROC curve for frontal plus depth

16
Conclusion

• First experimental results indicated that the 3D
  data offer limited performance improvement
  against multi-view 2D
• However only eigenfaces have been used so far
• exploitation of discriminant information it may
  help achieving better results
• Other algorithms will be applied as well
• Experiments on a larger database could provide
  more accurate results