Tracking-dependent and interactive video projection (Big Brother project)

1

• Tracking-dependent and interactive video
  projection (Big Brother project)

2

• Team

Matei Mancas Jonathan Demeyer Thierry Ravet
Donald Glowinski Gualtiero Volpe
Pierre Bretéché
3

• System Architecture

4

• Summary

• Robust tracking
• Modeling human motion attention of a visual
  scene

5

• Summary

• Robust tracking Tracking should be robust to
  severe illumination changes !
• Modeling human motion attention of a visual
  scene

6
Robust tracking data acquisition
Color tracking of red hats
IR tracking of lights on the top of the red hats
7
Robust tracking data fusion
Projective transform
8
Robust tracking data fusion
Selected points (Matlab)
Transformation matrix (EyesWeb)
Image warping (EyesWeb)
9
Robust tracking data fusion
RED color tracking / GREEN IR tracking / BLUE
Final fused tracking
10

• Summary

• Robust tracking
• Modeling human motion attention of a visual
  scene

11

• Summary

• Robust tracking
• Modeling human motion attention of a visual
  sceneRegions of interest should be dynamically
  highlighted and visual effects corresponding to
  outstanding events should be displayed

12
Attention in space (blob speed)
HOT RED most important / DARK RED less
important
Global contrast !
13
Attention in time (blob speed)
HOT RED really interesting / BLACK really
boring
Rarity on 4 seconds
14
Attention in time (quantity of motion)
HOT RED really interesting / BLACK really
boring
Rarity on 4 seconds
15

• Conclusion

• Summary
• Multi-blob real time tracking (on IR images and
  color images)
• Image registration using projective warping
• Data fusion using a weighted combination of blob
  positions based on confidence levels for each
  modality
• Demonstration of motion attention in space
  (instantaneous) and in time (short-time memory)
  -gt motion is not necessarily salient it depends
  on the context
• Use of blob speed and silhouette quantity of
  motion features

• So we worked hard ...

16

• Conclusion

• ... but we will work even harder
• Refinement of confidence level for each modality
• Better hardware is needed adapted optics for
  the cameras, and a color camera with the same
  characteristics than the IR camera easy modality
  registration
• A computer with several firewire ports easy
  modality synchronization
• Use of other features for attention for tracked
  paths (direction, acceleration, direction
  variation, trajectory curvature, ) and for
  gesture expressivity (energy, internal/external
  quantity of motion, )
• Long-time memory attention (flickering lights,
  commonly used paths, ) to get higher level
  motion segmentation
• More efficient instantaneous attention (space)
  which works also with moving cameras and for
  surprising behaviors in crowds

17

• Thank you !