Driver

1
Drivers View and Vehicle Surround Estimation
using Omnidirectional Video Stream
Kohsia S. Huang, Mohan M. Trivedi, and Tarak
Gandhi khuang_at_ucsd.edu, mtrivedi_at_ucsd.edu,
tgandhi_at_ece.ucsd.edu Computer Vision Robotics
Research (CVRR) Laboratory University of
California at San Diego La Jolla, CA 92093-0434
Abstract Our research is focused on the
development of novel machine vision based
telematic systems, which provide non-intrusive
probing of the state of the driver and driving
conditions. In this paper we present a system
which allows simultaneous capture of the driver's
head pose, driving view, and surroundings of the
vehicle. The integrated machine vision system
utilizes a video stream of full 360 degree
panoramic field of view. The processing modules
include perspective transformation, feature
extraction, head detection, head pose estimation,
driving view synthesis, and motion segmentation.
 The paper presents a multi-state statistical
decision models with Kalman filtering based
tracking for head pose detection and face
orientation estimation. The basic feasibility
and robustness of the approach is demonstrated
with a series of systematic experimental studies.
Keywords Driver head tracking, face orientation
estimation, drivers view generation, surround
vehicle detection.
Research Objective Accurate and real-time
estimation of drivers face orientation, drivers
view, as well as vehicle surround for a driver
assistance system.
Results
Drivers View Generation
Driver Head Detection and Tracking
Surround Vehicle Detection
Surround Vehicle Detection
Estimation of Head Pose and Face Orientation
Scheme 1
Scheme 2 (Future)
Average Performance
Head Detection
Head Detection before KF (DFFS Bound 2500) Setup 1 (Side View) Setup 2 (Front View)
Rough RHT, 1 Epoch 32 50
Rough RHT, 2 Epochs 52 61
Extensive RHT, 10 Epochs 71 79
RHTFeedback, 10?1 Epoch 64 73
RHTFeedback, 10?2 Epochs 67 87
Head Detection after KF 100 Head Detection after KF 100 Head Detection after KF 100
Computation of Drivers Viewing Direction
DFFS Bound False Positive
2500 9
2000 7
Face Orientation

• Summary
• Simultaneous driver head detection, driver face
  pose estimation, driving view generation, and
  surround vehicle monitoring in real-time using a
  single omni-video stream.
• Suitable for novel televiewing interfaces, driver
  assistance systems, and driver distraction
  studies.

Clip Frames Error before KF Error before KF Error after KF Error after KF Note
Clip Frames Mean Std Mean Std Note
1 200 -1 8 -1 7
2 75 -19 27 18 24 Uneven illum.
3 70 1 7 0 8
4 30 16 28 -15 16 Face occlusion
5 15 0 19 4 7
6 15 -3 8 -2 3