Head Tracking in Meeting Scenarios

1
Head Tracking in Meeting Scenarios

• Sascha Schreiber

2
Overview

• Video processing system for person action
  recognition
• Single person head tracking using elliptical
  structures
• Multiple person tracking
• Summary and outlook

3
System Setup
Video stream
Tracked persons
Kalman filtering
Temporal segmentation
HMM classification

1. Face/Head Tracking Face Detection Particle
   Filtering
2. Feature extraction Global Motion Features
3. Kalman filtering necessary for disturbed feature
   streams
4. Temporal segmentation Bayesian Information
   Criterion
5. HMM classification 6 models, one for each gesture

Gesture inventory

• Standing up
• Sitting down
• Pointing

• Nodding
• Shaking head
• Writing

4
Single Person Tracking

• Former system
• Initialization by pyramid sampling
• ? slow, only frontal to half-profile views of
  faces can be detected
• Weighting of particles by the output of a neural
  network
• ? Loss of track, if no face visible (e.g. person
  leans forward)
• ? Particles have to be placed quite exactly

5
Single Person Tracking

• Actual system
• Initialization by skin colored regions

• Former system
• Initialization by pyramid sampling
• ? slow, only frontal views of faces can be
  detected

1) Skin color blob detection
2) Search for valid blobs ? ratio major
axis/minor axis
6
Single Person Tracking

• Former system
• Weighting of particles by the output of a neural
  network
• ? Loss of track, if no face visible (e.g. person
  leans forward)
• ? Particles have to be placed quite exactly

• Actual system
• Weighting of particles using ellipses

1) Compute gradient image
7
Single Person Tracking

• Comparision of tracking results

Neural net
Ellipses
8
Multiple Person Tracking

• Problem
• Particles concentrate on location with highest
  head likelihood
• Solution
• Introduction of Super particles representing
  the number of persons in scenario

9
Multiple Person Tracking

• Weighting of Superparticles

1) Counting blobs in binary skin color mask ? n
blobs
10
Multiple Person Tracking

• Weighting of Superparticles

11
Multiple Person Tracking

• Weighting of Superparticles

12
Multiple Person Tracking

• Demovideo Multiple Person Tracking

13
Summarization Outlook

• Robust head tracking
• ? Improving tracking algorithm
• Features computed relative to the head position
• ? Tracking hands as additional feature
• Temporal segmentation facilitated by tracking
• Combined tracking and recognition

14
Head Tracking in Meeting Scenarios

• Sascha Schreiber