Noninvasive Techniques for Human Fatigue Monitoring

1
Non-invasive Techniques for Human Fatigue
Monitoring
Qiang Ji Dept. of Electrical, Computer, and
Systems Engineering Rensselaer Polytechnic
Institute qji_at_ecse.rpi.edu http//www.ecse.rpi.edu
/homepages/qji
2
(No Transcript)
3
Visual Behaviors

• Visual behaviors that typically reflect a
• person's level of fatigue include
• Eyelid movement
• Head movement
• Gaze
• Facial expressions

4
Eyelid Movements

• Tracking Eyes
• Develop techniques that can robustly track eyes
  under different face orientations, illuminations,
  and large head movements.
• Compute Eye movement parameters
• PERCLOS
• Average Eye Closure/Open Speed (AECS)

5
Eyes tracking demo
6
PERCLOS measurement over time
7
Average Eye Closure Speed Over time
8
(No Transcript)
9
(No Transcript)
10
Gaze (Pupil Movements)

• Real time gaze tracking
• No calibration is needed and allows natural head
  movements !.
• Gaze parameters
• Spatial gaze distribution overtime
• Ratio of fixation time to saccade time.

11
Gaze distribution over time while alert
12
Gaze distribution over time under fatigue
13
(No Transcript)
14
Head Movement

• Real time head pose tracking
• Perform 3D face pose estimation from a single
  uncalibrated camera.
• Head movement parameters
• Head tilt frequency over time
• Percentage of side views (PerSideV)

15
(No Transcript)
16
(No Transcript)
17
Facial Expressions

• Tracking facial features
• Recognize certain facial expressions related to
  fatigue like yawning.
• Building a database of fatigue expressions.

18
Facial expression demo
19
Fatigue Modeling

• Knowledge of fatigue is uncertain and from
  different levels of abstraction.
• Fatigue represents the affective state of an
  individual, is not observable, and can only be
  inferred.

20
Overview of Our Approach

• Propose a probabilistic framework based on
  Bayesian Networks (BN) to
• model fatigue.
• systematically integrate various sources of
  information related to fatigue.
• infer and predict fatigue from the available
  observations and the relevant contextual
  information.

21
Bayesian Networks Construction

• A BN model consists of target hypothesis
  variables (hidden nodes) and information
  variables (information nodes).
• Fatigue is the target hypothesis variable that we
  intend to infer.
• Other contextual factors and visual cues are the
  information nodes.

22
Causes for Fatigue

• Major factors to cause fatigue include
• Sleep quality.
• Circadian rhythm (time of day).
• Physical conditions.
• Working environment.

23
Bayesian Network Model for Monitoring Human
Fatigue
24
Interface with Vision Module

• An interface has been developed to connect the
  output of the computer vision system with the
  information fusion engine.
• The interface instantiates the evidences of the
  fatigue network, which then performs fatigue
  inference and displays the fatigue index in real
  time.

25
Conclusions

• Developed non-intrusive real-time computer vision
  techniques to extract multiple fatigue parameters
  related to eyelid movements, gaze, head movement,
  and facial expressions.
• Develop a probabilistic framework based on
  Bayesian networks to model and integrate
  contextual and visual cues information for
  fatigue monitoring.