Multimodal People Recognition Project Proposal for E6882 Multimedia Security System

1
Multimodal People Recognition Project Proposal
for E6882 Multimedia Security System

• Student Wenwei Wang
• Advisor Prof Ching-Yung Lin
• April 5, 2006

2
1. Project Subject

• Multimodality person recognition in this project
  is defined as the recognition of each person in a
  group using their face ID, voice ID, color
  appearance ID, sound source, and etc.
• It Enhances the efficiency and the robustness of
  person identification algorithm

3
2. Multimodality person recognition Applications

• Multimodal Bio ID more reliable
• Secured site access
• Nursing Room
• Automatic Multimedia Meeting Recording who says
  what
• Automatic Banking
• Public security
• And more

4
3. Key Words

• Multimodality
• Unconstrained Audio and Video
• Face recognition
• Speaker identification
• Face Depth Estimate
• Probability Color Model
• Event Detection
• Background Adaption
• Classifier Fusion
• Confidence score

5
4. Project Scope

• One-person project
• Bottom line Investigate the emerging techniques
  in the subject field by summarizing and
  discussing some advanced algorithms.
• Top line Conduct experiments on some interesting
  algorithms using MatLab, or if any, try out some
  improvements

6
5. Project Schedule and Evaluation

• Proposal April 5
• Final Presentation May 10
• Submission May 13
• Evaluation? Yeah, please tell me how you evaluate
  my project.
• I need a A, anyway

7
6. Who are doing Multimodality Research

• ATR media Integration Communications Lab
  Kenji Mase
• CMUs Interactive Systems Laboratories
  Alexander Waibel, Jie Yang, Paul Duchnowski (now
  at MIT)
• Compaq Cambridge Research laboratory - Vladimir
  Pavlovic, James M. Rehg
• Georgia Tech Computational Perceptual Lab - Aaron
  Bobick, Frank Dellaert ,Irfan A. Essa, Thad
  Starner
• IBM Research DreamSpace Project - Mark Lucente
• IMAG CLIPS - Laurence Nigay, Joëlle Coutaz
• IMAG GRAVIR - James L. Crowley and Francois
  Berard
• MIT AI Lab Vision Interface Group - Trevor
  Darrell
• MIT Media Lab () - Justine Cassell, Alex
  Pentland, Kristinn R. Thórisson
• OGI Center for Human-Computer Communication -
  Phil Cohen, Sharon Oviatt
• Rutgers University Center for Advanced
  Information Processing - James L. Flanagan,
  Grigore Burdea, Casimir A. Kulikowski, Ivan
  Marsic, Peter Meer, Joseph Wilder, Attila Medl
• University of Maryland Perceptive Interface and
  Reality Lab Yaser Yacob, Larry S. Davis, Ramani
  Duraiswami, David Harwood
• ESPRIT Project 8579/MIAMI Multimodal Integration
  for Advanced Multimedia Interfaces
• Federated Laboratory for Advanced Displays and
  Interactive Displays

8
7. Technical Papers Collected and Collecting

• J. Yang, X. Zhu, R. Gross, J. Kominek, Y. Pan, A.
  Waibel at CMU Multimodal People ID for a
  Multimedia Meeting Brower
• T. Choudhury, B. Clarkson, T. Jebara, A. Pentland
  at MIT Multimodal Person Recognition using
  Unconstrained Audio and Video
• R. Stiefelhagen, X. Chen, at CMU Capturing
  Interactions in Meetings with Omnidirectional
  Cameras
• A. Hauptmann, J. Gao, R. Yan, Y. Qi, H. Wactlar
  at CMU, Automated Analysis of Nursing Home
  Observations
• C. Broun, X. Zhang at GIT Multimodal fusion of
  polynomial classifiers for automatic person
  recognition
• And more, and more

9
8. Reference Books Collected

• Prof. C-Y Lin Class notes, Lectures, and etc.
• T. Moon, W. Stirling Methematical Methods and
  Algorithms for Signal Processing.
• S. Y. Kung, M. Y. Pak, S.H. Lin Biometrics
  Authentication
• R. Gonzalez, R. Woods Digital Image Processing.
• B. Gold, N. Morgan Speech and Audio Signal
  Processing
• S. K. Mitra Digital Signal Processing