Assistive Technology

1
Assistive Technology
CS 6750 Human Computer Interaction
MMJ 2008
2
Disability in the USA

• 53 million (18) Americans have one or more
  physical or cognitive disabilities (recent
  census data) about 1 in 5 people
• Graying of America number growing
  significantly as baby boomers age
• Life expectancy 75 and climbing
• Better medical care actually increases numbers of
  people with disabilities

3
The ADA

• The Americans with Disabilities Act (1990)
• Prohibits discrimination based on disability
• Employment (qualified individuals)
• Public Services (section 504 of 1973 Rehab Act)
• Public Accommodations
• Telecommunications
• Miscellaneous Provisions
• No cognitive section!

4
Assistive Technology

• Includes assistive, adaptive, and rehabilitative
  devices and the process used in selecting,
  locating, and using them.
• AT promotes greater independence for people with
  disabilities by enabling them to perform tasks
  that they were formerly unable to accomplish

5
Mainstream AT
Jacuzzi
Typewriter
Real-time captioning
Voice Recognition
6
Assistive Technology Areas

• Communication disability
• Visual disability
• Hearing disability
• Cognitive disability
• Motor disability

7
Communication

• Augmentative and Alternative Communication (AAC)
  devices
• For individuals with severe communication
  disorders and for whom gestural, speech, and/or
  written communication is temporarily or
  permanently inadequate to meet all of their
  communication needs

8
GT ResearchAwareComm

• DBI low-bandwidth, high error
• rate control interface
• How to increase performance?
• Reduce selection space for AAC
• by context
• location
• conversational partner
• time of day
• conversational history

9
GT AAC Research HealthTalker
10
GT Research NeilComm
11
Visual Disability
Talking Products
Braille Printer
Haptic Displays
12
GT Research SWAN

• System for Wearable Audio Navigation
• Orientation tracking technologies including GPS,
  inertial sensors, pedometer, RFID tags, RF
  sensors, and compass
• Sophisticated sensor fusion is used to determine
  the best estimate of the user's location and
  which way she is facing
• An audio-only interface ("beacons") guide the
  listener along a path, while at the same time
  indicating the location of important features in
  the environment

Profs. Frank Dellaert and Bruce Walker
13
Hearing Disability
Hearing Aid (amplifier)
Cochlear implant
14
GT Research American Sign Language Recognition
Multi-sensor calibration
CopyCat game for deaf children to learn ASL
Hat-mounted camera
Prof Thad Starner and the Contextual Computing
Group
15
BrainSign Detecting ASL in Brain Signals

• Detect Imagined Gestures in motor cortex
• Correlations with visual, Brocas
• fMRI initial study
• Jackson and Starner

16
Cognitive Disability

• Task tracking and reminder systems
• Aids for learning disabilities
• Crisis intervention support

Isaac project PDA with GPS, cell phone, camera
in mid-90s (Univ of Colorado Boulder)
17
Cognitive Disability Research at GT
Cooks Collage intelligent kitchens Provides a
visual summary of recent cooking activity along a
kitchen countertop. Emphasizes the temporal order
of cooking events. (Prof Beth Mynatt)
Abaris for children with autism (Prof Gregory
Abowd)
18
Motor Disability
Switches for scanning interfaces (WiVik)
Sip and Puff switchMouth joystick
Muscle mouse EMG switch
19
Motor Disability Research at GT Direct Brain
Interfaces

• a system that detects minute changes in signals
  generated by a human brain, providing a channel
  to control computers and other devices.
• The BrainLab Mission
• is to
• pioneer real-world applications research for
  DBI and other biometric technologies
• to improve the quality of life for people with
  severe disabilities, and
• to explore possibilities for mainstream
  applications.

20
Who uses DBIs Current Subjects - Locked-In
Syndrome

• Disease, stroke, accident survivors
• Completely physically paralyzed
• Unable to speak
• Cognitively intact and alert
• 500,000 people worldwide

21
Is DBI Mind Reading?

• Aspects of Brain Signals that can used for
  control
• Frequency-domain features (EEG mu or beta
  rhythms) increase signal voltage over
  different brain areas Wolpaw Birbaumer
  Millan
• Movement-Related Potentials real or imagined
  movements create signals in the motor cortex
  Mason Levine
• Evoked Responses (P300 potentials, SSVEP) -
  recognition or oddball response, frequency
  detection Donchin Farwell Bayliss
• Action potentials of single unit cortical
  neuronal firings implanted electrodes
  Kennedy Donoghue Nicolelis, Chapin

22
Outputs and Control

• Discrete
• Switch binary value
• Selection - from a set of choices
• 1-D device control logical navigation
• Continuous
• Pointing / Drawing - 2-dimensional movement of
  a cursor
• 2-D and 3-D Device control - Control of an
  external device (wheelchair) or virtual prosthetic

23
Brain Signal Detection Techniques
Surgically implanted electrodes
Non-surgical techniques scalp electrodes (EEG),
Functional Infrared Imaging
24
fMRI for Electrode Placement
25
System Architecture
Receiver
Amplifier
Amplifier
FM Transmitter
Power Induction Coil
DSP Computer
Electrode
Brain
Subjects Computer
Scalp
Skull
26
First Human Implantations
Marjorie
Johnny
Tim
27
Virtual Keyboard
Task move cursor in two dimensions to select
letters using dwell
28
Learning Curves
5th KENEDY GQLDXWAIJTF 4th PHILXKIM
NMFELODY 3rd JOIH.N 2nd HIJJROHLN
1st JOHLQQQ.GYUVWABD N
29
Emergence of Cursor Cortex
4th NOTHING What were you thinking when
moving the cursor? 3rd THE WIND 2nd GONE
WITH What is your favorite movie? 1st BRANDO
N Spell the new guy's name JRDAY423 5/24/99
30
DBI Approaches Mu Control

• Mu is a brain wave that occurs over motor cortex
• Mu is increased by movement or thinking about
  movement
• Subjects can be trained to increase and decrease
  mu

31
A M
N - Z
32
(No Transcript)
33
DBI Approaches P300

• P300 is an evoked response that occurs over the
  parietal region of the brain
• 300 ms after a stimulus, a change can be seen
• Has been used to control a speller Donchin
  Allison and interactions in a virtual world
  Bayliss
• We have controlled a robot arm Vora, Allison,
  Moore

34
M
A B C D E F G H I H K L M N O P Q R S T U V W X Y
Z . ? ! ltbackgt
35
DBI Approaches SSVEP

• Steady State Visual Evoked Potentials
• Flash images at different frequencies
• Can see from activity in the visual cortex which
  image subject is attending to
• Does not require gaze shift
• Currently fastest DBI

36
YES
NO
37
DBI Approaches Invasive techniques Kennedys
Neurotrophic Electrode

• Hollow glass cone with two gold wires and
  neurotrophic factor inside
• Implanted in motor cortex of locked-in subjects
• Intercept single-unit firings
• Control based on frequency of firings

38
Other Biometric Interfaces fNIR

• Functional Near Infrared imaging
• Portable OTIS system
• No messy gels very easy to set up
• Requires no training
• Works for locked-in subjects
• New NSF grant Creative Expression

39
Other Biometric InterfacesGalvanic Skin Response

• Measure skin conductance with COTS polygraph
  machine
• Train user to control GSR with imagery and
  visualization
• Currently studying 4 completely locked-in
  patients
• Average 80 accuracy on binary task

40
Neural InternetP3 BrainBrowser

• Low Bandwidth Internet Access
• Access to email and web via
• specialized browser and email program
• Uses
• communication
• shopping
• education
• handling of personal finances
• employment

41
The Aware Chair

• Integrated communication and environmental
  control
• Intelligent, neurally controlled wheelchair
• Conversation and environmental control
  prediction
• Learns users habits and context

42
Restoring Motion - Neural Prosthetics

• Brain re-learns how to move limbs via an
  artificial nervous system
• Simulation
• Virtual reality hand
• Restoring Physical Motion
• Functional Electrical
• Stimulation (FES)

43
Disability from Stroke

• Leading cause of long-term disability
• 1 in 6 people over age 55 are at risk for stroke
• 1.1 million in U.S. disabled by stroke at
  present
• Baby boomer population increasing these numbers

44
DBI for Stroke Rehabilitation

• Can using a DBI Influence Neural Plasticity?

New collaboration with GT Applied Physiology and
BioEngineering Emory Rehab Medicine KINARM
roboticarm
45
Visit the BrainLab!

• TSRB 2nd floor
• Participate as a subject - learn brain control!
• Independent research projects for credit