Senior Capstone Project Gaze Tracking System

1
Senior Capstone ProjectGaze Tracking System

• Progress Presentation
• IRALAR
• Breanna Heidenburg -- Michael Lenisa --
  Daniel Wentzel
• Advisor Dr. Malinowski

2
Presentation Outline

• Project Overview
• Application Layout Progress
• Image Processing
• Calibration
• Clicking
• Communication
• User Interface
• Overall Progress

3
Brief overview

• What is our project?

4
Brief Overview

• Track a users eye
• Small form factor camera
• Use input to control cursor
• Where you look is where the cursor is
• Need to calibrate for every user
• Create user interface

5
Application Layout

• How will we achieve our goal?

6
Application Layout
7
Application LayoutImage Processing

• Hardware
• Camera
• QuickCam Pro for Notebooks
• Polarizer
• Tiffen 25mm polarizing filter
• Removes glare from eye reflections
• Lighting
• Diffuse LEDs
• Not sure if they are necessary, will find out
  when we have the display

8
Application LayoutImage Processing

• New Image processing methods
• Blob Recognition
• Classic Image Processing Techniques
• Contrast stretching
• Thresholding
• Adaptive Windowing

9
Application LayoutImage Processing

• Sample Recognitions
• Difficult Low Light False Positive

10
Application LayoutCalibration

• Why do we need to calibrate?
• Map the users eye to a point on the screen
• Everyones eyes are different
• Account for various positions of the camera
• What does our data look like?
• Somewhat unreliable
• Problem
• Movements while taking data

11
Application LayoutCalibration

• X-Axis values

12
Application LayoutCalibration

• Y-Axis values

13
Application LayoutCalibration

• What does the data tell us?
• A 2D mapping is not good enough
• Horizontal eye movement is closer to linear than
  vertical eye movement
• Vertical eye movement depends on both coordinates
  of data

14
Application LayoutCalibration

• How do we solve the problem?
• Multiple Variable Linear Regression Least
  Squares
• Y B0 B1x1 Bkxk
• Uses matrix algebra to obtain a coefficient
  matrix
• B (XX)-1XY

15
Application LayoutCalibration
B (XX)-1XY

• Results
• pixelX 224.9 4.8eyeX 2.9eyeY
• pixelY 1612.5 2.0eyeX 5.6eyeY
• Error very high (need more than 4 random data
  points)

16
Application LayoutClicking

• How do we click?
• Monitor eye movements
• Identify pauses
• When eye position is within a small percentage
  for a certain amount of time
• Set to 5 frames (200 mS)
• Generally, it takes 230 mS for a hand to click a
  mouse.

17
Application LayoutCommunication
18
Application LayoutCommunication

• Why is it important?
• Allows multiple processes to communicate
• Allows relay of time sensitive information
• UDP vs. TCP
• UDP -gt Connectionless
• TCP -gt Connection oriented
• Multi-threading
• Necessary for running multiple pieces of code in
  a single process

19
Application LayoutCommunication

• Multi-threading
• Issues with public variable usage
• Solutions we are using
• Critical Section
• Raises thread priority (does not allow for
  interrupts)
• Mutual Exclusion
• Creates a variable lock
• Using variable first requires unlocking it

20
Application LayoutUser Interface

• Achieved via a combination of Hardware and
  Software
• Software
• WPF C
• XAML
• (eXtensible Application Markup Language)
• Very similar to XML
• Uses code-behind in a similar style to
  javascript
• code-behind is in C

21
Application LayoutUser Interface

• Framework is in place for all pages
• Navigation between pages is complete
• Core operating portions are complete

22
Application LayoutUser Interface

• Hardware
• LitEye LE500 has been ordered
• (will arrive early-mid March)
• SVGA resolution (800x600)
• Transparent

23
Overall Progress

• How far along are we?

24
ProgressHardware

• HMD has arrived (today)
• Have not had much time to work with it
• We arent 100 sure how we will physically mount
  the camera to the display
• Arent sure if extra lighting will be necessary

25
ProgressSoftware

• Image Processing
• Still working on some recognition improvements
• Calibration
• Minor adjustments left to be made
• Communication
• Minor adjustments left to be made
• User Interface
• Only essential parts are included at the moment
• Still ironing out details on what will be
  included in final version

60
90
90
40
26
ProgressOverall
27
Questions Demo