Active Range Imaging Datasets for Indoor Surveillance

1
Active Range Imaging Datasets for Indoor
Surveillance
C. Distante, G. Diraco, A. Leone Institute for
Microelectronics and Microsystems CNR, Lecce
(Italy)
2

• Introduction
• Outline of active range vision
• ? Range imaging technologies
• ? Properties of Time-Of-Flight range sensors
• Active range vision Vs Passive vision
• ? Comparison between TOF camera and stereo
  vision
• ? Advantages and drawbacks in surveillance
  contexts
• Datasets for indoor surveillance
• Case study
• ? TOF sensor-based fall detection
• Conclusions

3
In the last years several active range sensors
have been presented (Canesta Inc., Mesa Imaging
AG, 3DV Systems Ltd, ). The ability to
describe scenes in three dimensions opens new
scenarios, providing new opportunities in
different applications, including visual
monitoring (object detection, tracking,
recognition, image understanding), security,
biometrics, automotive, robotics, medical
imaging, Active range sensors provide depth
information allowing to use algorithms much less
complex and allows problems to be approached in a
new, robust and cost-efficient way. Datasets
are presented in order to suggest a common basis
for comparative analysis of vision algorithms.

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

4
Range Imaging (RIM) is the fusion of two
different technologies, integrating depth
measurement and imaging aspects. Its a new
measurement technique, not yet well-known and
investigated.

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

Depth Measurement Techniques Taxonomy
Contactless depth measurement
Triangulation (stereoscopy, structured light -
sub-millimeter resolution)
Interferometry (light source scanning -
sub-micrometer resolution)
Time-Of-Flight (millimeter resolution)
Pulse (Direct measure)
Continuous wave modulation (Indirect measure,
eye-safe)
5
Principles of phase shift modulation-based TOF
sensors

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

The depth estimation is realized by measuring the
phase shift f of the signal round-trip from the
device to the target and back.
6

• Main features of modulation-based TOF sensors
• standard CMOS technology
• high frame rate (up to 30 fps)
• fairly good spatial resolution (up to QCIF _at_
  176x144 pixels)
• fairly good field of view (up to 80x80 degrees)
• aliasing effects (non-ambiguity range up to 30
  meters)
• low depth measurement error (lt 1 in
  non-ambiguity range)
• direct Cartesian coordinate output (x, y, z) for
  3D reconstruction
• built-in band-pass optics for background light
  suppression
• illumination power less than 1W (LED array,
  Class 1 for eye-safe)

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

• Two critical parameters affect performances
• modulation frequency (it is a design parameter
  that mainly affects the non-ambiguity range)
• integration time (it could be adjusted and
  affects depth resolution and frame rate)

7
Comparison of the most important characteristics
of TOF cameras and stereo vision systems

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

8
Advantages in the use of TOF sensors in
surveillance contexts

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

9
Drawbacks in the use of TOF sensors in
surveillance contexts

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

10

• Datasets description
• Each sequence is acquired at QCIF resolution by
  a state-of-the-art TOF sensor (MESA SR-3000) and
  it is composed by 1800 frames captured at
  variable frame rate (by varying integration time)

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

• Sequences have been acquired in
  wall/ceiling-mounting configurations at different
  subject orientations, in presence/absence of
  occlusions, in order to cover a large amount of
  events
• Extrinsic parameters are available for
  calibration purpose (camera-floor distance,
  camera orientation, scene depth)
• Sequences present people (one/more persons in
  the scene) having different postures (stand, sit,
  lay down, bent, squat)

• In the sequences persons have different
  behaviours (walking, falling down, moving
  objects, picking objects, limping)
• Datasets are at http//siplab.le.imm.cnr.it

11

• Datasets description
• A generic frame (176x144 pixels) of each sequence
  presents the following structure
• Raw data
• Depth image (2Bytes/pixel unsigned integer)
• Intensity image (2Bytes/pixel unsigned
  integer)
• FPGA processed data
• Depth image with noise reduction (2Bytes/pixel
  unsigned integer)
• Intensity image with noise reduction
  (2Bytes/pixel unsigned integer)
• Cartesian x coordinate (4Bytes/pixel signed
  float)
• Cartesian y coordinate (4Bytes/pixel signed
  float)
• Cartesian z coordinate (4Bytes/pixel signed
  float)

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

For each frame a great amount of information is
defined (495KBytes)!
12
Noise is due to high-reflective objects (LCD TV)
Aliasing and multi-path effects

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

Intensity image
Depth image
Fluctuations are due to the continuously adjusted
emitted light power
13

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

Intensity image
Depth image
Could the depth information help the tracking in
the presence of total occlusions?
14

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

Intensity image
Depth image
15
TOF sensor-based fall detection

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

Gaussians mixture for background modelling
Bayesian approach for segmentation
16
In indoor surveillance applications, range images
provide a better perception of scenes in all
illumination conditions, deterring the use of
cheap stereo systems that fail in dark or
low-textured environments. If critical
parameters of TOF sensor are adjusted, reliable,
computationally low-cost and real-time
segmentation/tracking can be realized by only
using depth measure, since intensity images
present unwanted fluctuations. Depth
information overcomes projective ambiguity,
whereas intensity image provides appearance
information, so that the joined use of them
improves critical steps (object recognition,
behavior analysis, ) allowing a better
description of moving objects. The suggested
datasets provide common basis to investigate
vision algorithms they can be improved by
defining ground-truth data to quantify
performances.

• Introduction
• Outline of active range vision
• Active range vision Vs Passive vision
• Datasets for indoor surveillance
• Case study
• Conclusions

17
THANK YOU FOR YOUR ATTENTION