Proposal for an Eye-Gazed

1
Proposal for an Eye-Gazed Reading Remediation
System
S. Ali Ahmadi
The George Washington University Department of
Physiology Department of Computer Science
2
Outline
I. Introduction II. Solution III. Feasibility
Tests
3
Introduction
Motivation Background Requirements Primary
Objective
4
Motivation National Institute of Health 10
million children with Dyslexia in 50,000 school
district in US Large number of adults with
learning disabilities Reading disabilities
caused by neurological trauma (such as head
injury) Mouse-activation prompting system for
reading improvement text is displayed on a
computer screen selected by a mouse and
sequentially highlighted text is spoken by
subject Mouse-activation prompting system has
disadvantages requires a precise hand-eye
coordination adds a significant delay in the
cognitive process of reading and understanding.
Problem Statement Current computer-based reading
remediation systems are not very effective and/or
they significantly delay the cognitive process of
reading and comprehension.
INTRODUCTION
5
Background Reading Disabilities Dyslexia Most
common form of learning disability Reading
difficulties Difficulty with sounding words
and letters Difficulty spelling or writing
numbers Words or letters may appear
reversed For example b for d or saw for
was Source Brain (Positron Emission
Topography PET) Dyslexic A person diagnosed
with dyslexia Above average intelligence Ful
l understanding of spoken words They may never
learn to read effectively. Alexia A reading
disability similar to Dyslexia Caused by a
damage to the brain (such as a head injury)
INTRODUCTION
6
Background Eye Movements in Reading Eye
movements in reading Have been studied for
many years Ranges of values have been
established for normal readers. Can be used to
infer moment-to-moment cognitive processing in
the brain. (Rayen 1998) Can be easily tracked
with the current Eye Tracking technology Fixatio
n Eye movement which stabilizes the retina over
a stationary object (or point) of Interest.
Saccade The rapid intermittent movement of
eyes between fixations during
reading 1- Progressive (rightward) Saccad
es can be 2- Regressive
(leftward) Largely controlled by the frontal
eye field of the cerebral cortex
INTRODUCTION
7
Background Eye Movements in Reading
Example of Fixations and Saccades
Now is the time for all quick brown foxes to
come to the aid of their countryside. He thought
usually...
Text
Fixations
Saccades
Typical gaze point graphs in a normal reading
task (from left to right, top to bottom)
INTRODUCTION
8
Background Eye Movements
Statistical Data for Normal Readers Eye Movements
Approximate mean fixation time 250 milliseconds
Range of fixation time 100-500 millisecond
Average progressive saccades 7-9 characters 2 visual degrees
Average regressive saccades 3-4 characters less than 1 visual degree
Average Percentage of regressive saccades 10-20 of all saccades

• For dyslexic and poor readers
• Longer fixation durations
• Shorter and larger number of saccades
• More regressive saccades
• Difficulties in processing written words are not
  because of poor control of eye movements the
  source is the brain.

INTRODUCTION
9
Background Reading Theory and Eye
Movements Immediacy Assumption Content of a
word is processed as soon as it is
encountered. Eye-Mind Assumption As long
as a word is being processed by the brain,
eyes are fixed on the word. Alternative
Assumption Data from several successive eye
fixations are internally buffered before
being semantically processed. Most of
experiments support the Eye-Mind
assumption. Eye Movement study in reading
shows words with lower frequency and
unfamiliar words need longer fixations words
with higher frequency and familiar words need
shorter fixations The length of fixations on a
word and the number of fixations around a word
are proportional to the level of difficulty of
that word.
INTRODUCTION
10
Background Reading Theory and Eye
Movements Reading Modes A readers behavior
during a reading task can be classified
into 1- Reading 2- Scanning 3-
Dormant Gazing 4- Encounter
Difficulties While in Reading mode, gaze
duration reflects the time needed to execute the
cognitive process. Increase in duration of
fixations or number of fixations around a word
can be interpreted as Encounter Difficulties
(mode 4).
INTRODUCTION
11

• Requirements
• The proposed Reading Remediation system
• Must use the Eye Gaze system from LC technologies
  for eye tracking
• Must be implemented with Visual C IDE under
  Windows 2000
• API provided by the LC technologies uses Visual
  C IDE
• Must be able to receive any text file as input
• Must be installed on Matrox 4II (to be used by
  potential disabled users)

INTRODUCTION
12
Primary Objective
To evaluate the effectiveness of an
eye-gazed-trigged visual and auditory prompting
system based on the eye tracking technology and
used in the context of reading remediation.
INTRODUCTION
13
Solution
Proposed Solution Goals and Hypothesis
14
Proposed Solution

• Mechanism
• A software application which,
• renders the input text file as a sequence of
  word objects on a computer screen
• receives the readers gaze-point coordinates
  from the Eye Gaze system
• determines the difficult words for the reader by
  a specific detection algorithm
• assists the reader by highlighting and
  pronouncing the difficult words in real time

SOLUTION
15
Solution Detection of Difficult Words

• Defining text as word objects
• Xi distance between two word objects
• Yi distance between lines of text
• Xw boundary box width
• Yw boundary box height
• Xw , Yw depend on font type and font size
• Xi , Yi can be set for each reader

Fixations and Dwell on each Word

• Single fixation duration is insufficient for the
  cognitive process for word recognition.
• Usually more than one fixation per word
• Dwell Series of one or more fixations
  separated by very short saccades, necessary to
  process a single word
• Dwell Time Total time for all of the fixations
  in a dwell

SOLUTION
16
Solution Detection of Difficult Words

• Dwell Time Thresholds
• Dwell Time Threshold1
• Signifies that a word is the focus of attention
• Word will be highlighted by the program
• Dwell Time Threshold2
• Subject has difficulty recognizing the word
• Word will be pronounced by the program

SOLUTION
17
Primary Goal
To measure the improvement of the reading ability
of test patients who have used the Eye-Gazed
prompting system to assist them in reading
Primary Hypothesis
The unobtrusive assistance rendered by the
Eye-Gazed auditory prompting system is an
effective remediation for Dyslexia and Alexia.
Primary Goal and Hypothesis
18
Research Goals
GOAL TREE
19
Feasibility Test
Pilot Tests Preliminary Results
20
Feasibility Tests and Preliminary Results

• Experiment
• Subject were given a text on a computer screen
  to read aloud with RR system.
• Text presented twice in immediate succession to
  observe the effects of familiarization and
  practice.
• Trials were audio and video recorded to help
  detecting reading errors.
• Obtained data
• Reading speed, errors, fixations per second,
  fixation duration, fixations per word, number of
  prompts

• Test subjects
• 8 children, aged between 10 and 14, all
  diagnosed with Dyslexia
• Four of them were receiving Learning
  Disabilities services
• Reading Text
• Two passages from standard Gray Oral Test
  Series, 3rd Edition based on their age and grade

PILOT TESTS
21
Feasibility Tests and Preliminary Results
Parameters and Conditions
Font type Times New Roman Font size 42
point Line Separation 70 pixels Word
Distance 14 pixels Horizontal Margin 30
pixels Vertical Margin 60 pixels Dwell
Threshold 1 240 milliseconds Dwell Threshold
2 360 milliseconds Sampling Frequency 60
Hz Light Conditions no sun light, fluorescent
lamp
PILOT TESTS
22
Feasibility Tests and Preliminary Results
Average Results for subjects 1 4 5th Grade Text 5th Grade Text 10th Grade Text 10th Grade Text
Average Results for subjects 1 4 Try I Try II Try I Try II
time (seconds) 37.0 30.0 84.8 68.6
words per seconds 2.9 3.6 1.3 2.2
errors 0.4 0.0 4.8 3.0
fixations 122.6 112.4 254.8 240.4
fixations per seconds 3.3 3.7 3.0 3.5
fixations per word 1.2 1.1 1.7 1.6
fixation duration (ms) 231.4 228.4 232.0 241.0
prompts 1.2 0.4 9.0 3.6
RESULTS
23
Feasibility Tests and Preliminary Results
Average Results for subjects 5 8 3rd Grade Text 3rd Grade Text 5th Grade Text 5th Grade Text
Average Results for subjects 5 8 Try I Try II Try I Try II
time (seconds) 67.5 67.0 87.75 76.5
words per seconds 1.23 1.24 1.20 1.37
errors 4.75 2.25 9.5 4.75
fixations 177 158 196 191
fixations per seconds 2.62 2.36 2.23 2.50
fixations per word 2.13 1.90 1.87 1.82
fixation duration (ms) 434.25 310.12 378.85 349.93
Prompts 7 3.75 14.5 9.5
RESULTS
24
Feasibility Tests and Preliminary Results

• From trial 1 to trial 2, results showed
• increased reading speed
• more fixations per second
• shorter fixation duration
• less number of fixations
• less number of errors

RESULTS
25
Project Task Tree
Primary Task 1
To Measure the effectiveness of an Eye-Gazed Reading Remediation system
Task 1.1 (A)
Investigate Related Work
Task 1.2 (H)
Specify Solution
Task 1.5 (V)
Validate Project Objective
Task 1.4 (S)
Acquire Remediation Measurements
Task 1.3 (H)
Set Research Goals
Task 1.2.1 (S)
Design Reading Remediation Software
Task 1.3.1 (S)
Define Factors and Performance Metrics
Task 1.1.1 (S)
Determine Problems with Current Computer Based Reading Remediation Systems
Task 1.4.1 (S)
Implement Reading Remediation Software
Task 1.5.1 (A)
Compute Performance Statistics
Task 1.2.1.1 (S)
Design a Detection Algorithm for Difficult Words
Task 1.5.2 (H)
Draw Conclusions
Task 1.4.2 (S)
Mount Remediation Experiments
Task 1.1.2 (S)
Ascertain Eye Movements in Reading
Task 1.2.1.1.1 (V)
Run a Pilot Test on Detection Algorithm with Eye Gaze system
Task 1.5.3 (S)
Document Project
Task 1.4.2.1 (A)
Select Appropriate Text for Reader
Task 1.1.3 (S)
Ascertain Appropriate Eye Tracking Methods
Task 1.5.4 (V)
Submit For Peer Review
Task 1.4.2.2 (H)
Design Remediation Experiment
Task 1.1.4 (S)
Ascertain Role of Eye Movements in Reading Theory
Task 1.4.2.3 (S)
Conduct Remediation Experiments
A Analysis HHypothesis SSynthesis
VValidation
26
The Initial Milestone chart for the Reading
Remediation Project
Milestone Chart
27
Governing Propositions

• Dyslexia The most common from of Learning
  Disability
• Dyslexia is usually for a person with normal
  intelligence and full understanding of spoken
  words
• Dyslexic A person diagnosed with dyslexia
• Dyslexic patients tend to show above average
  intelligence, but they may never learn to read
  effectively.
• To a dyslexic person words or letters may appear
  reversed. For example b for d or saw for was
• Recent studies with Positron Emission Topography
  (PET) strongly suggest that the source of
  dyslexia is located in the brain.
• Although Dyslexic patients tend to show above
  average intelligence, they may never learn to
  read effectively.
• Alexia - A reading disability similar to Dyslexia
  caused by a damage to the brain (such as a head
  injury)
• Eye movements can be used to infer
  moment-to-moment cognitive processing in the
  brain during a reading task. (Raven 1998)
• Saccade The rapid interment movement of eyes
  between fixations during reading

28
Governing Propositions

• Fixation - Eye movement which stabilizes the
  retina over a stationary object (or point) of
  Interest.
• Saccadic eye movements are largely controlled by
  the frontal eye field of the cerebral cortex.
• In a reading task, eye movements reflect the
  cognitive burden during the process of the text.
• Poor readers and dyslexic readers tend to have
  longer fixation durations, shorter and larger
  number of saccades and more regressive saccades
  than normal readers.
• Differences in eye movements for most dyslexic
  readers reflect difficulties in processing
  written words rather than poor control of eye
  movements.
• The duration of fixation on a word reflects the
  difficulty level of that word.
• During normal reading, if adults are given a
  text appropriate for their age and education,
  they have an average of one to two words per
  fixation.
• A reader tries to interpret the content of a
  word as soon as it is encountered. ( Immediacy
  Assumption)
• The eye remains fixated on a word as long at the
  word is being processed by the brain. (Eye-Mind
  assumption)

29
Governing Propositions

• Data from several successive eye fixations are
  internally buffered before being semantically
  processed.
• Words with lower frequency and unfamiliar words
  need longer fixations and words with higher
  frequency and familiar words need shorter
  fixations.
• The length of fixations on a word and the number
  of fixations around a word are proportional to
  the level of difficulty of that word.
• Single fixation duration is insufficient for the
  cognitive process for word recognition and
  usually more than one fixation per word is
  required.
• Dwell - Series of one or more fixations
  separated by very short saccades that are
  necessary to process a single word.
• Dwell Time is the total time for all of the
  fixations in a dwell.

30

• References
• Columbia University Medical Encyclopedia
• American Heritage Dictionary
• National Institute of Health
• National Institute of Health
• Columbia University Medical Encyclopedia
• American Heritage Dictionary
• A. Duchowski. Eye Tracking Methodology