Using Handhelds to Help People with Motor Impairments

1
Using Handhelds toHelp People withMotor
Impairments
Human Computer Interaction Institute School of
Computer Science Carnegie Mellon University

• Brad A. Myers, Jacob O. Wobbrock, Sunny Yang,
  Brian Yeung,Jeffrey Nichols, and Robert
  Millerhttp//www.cs.cmu.edu/pebbles/assistive/

Assets2002
2
Muscular Dystrophy

• About 250,000 people in the United States have
  Muscular Dystrophy (MD)
• One in every 4,000 newborn boys has Duchenne
  Muscular Dystrophy
• Use computers for recreation, education, etc.
• Often home-bound
• increased susceptibility to infections
• Internet provides opportunities for at-home
  employment

3
Muscular Dystrophy, cont.

• People with MD often have fine-motor control but
  limited coarse-motor control
• Tiresome to press keys on keyboard
• Difficult to transfer hand between mouse and
  keyboard
• Hard to move mouse precisely
• We investigate using Palm handhelds as assistive
  device for PCs
• May be easier to use stylus for both typing and
  pointing tasks

4
Pebbles Project

• This research is part of the Pebbles project
• Overall goal investigate use of handheldsat the
  same time as PCs and other computerized devices
• Multi-Machine User Interfaces
• Assumption handhelds will frequently be in close
  interactive communication with each other and
  other computers
• Wireless and wired technologies

5
Original Remote Commander

• Allow PDAs to control a PC
• Can be used with any application
• Perform all mouse andkeyboard functions
• Use PDA like touchpad
• Graffiti or our own pop-up keyboard
• Original motivation audience in ameeting take
  turns controllingapplication

6
History

• Drew Rossman saw RemoteCommander on the web
• Found it helped his daughterJennifer who has a
  form ofMuscular Dystrophy known asSpinal
  Muscular Atrophy, Type II.
• Wrote articles and created a webpage to help
  promote its use
• We wanted to improve the software for this use

7
Methodology

• Recruited local (Pittsburgh-area)subjects with
  MD using thePittsburgh-area MD newsletter
• Visited them in their homes
• Loaned each one a Palm IIIx (or equivalent IBM
  Workpad) and installed our software on their
  computer
• Donated by Palm, Inc. and IBM

8
Methodology, cont.

• Initial evaluation
• Measured typing and mouse speed with PC-based
  tests
• Directly observed subjects
• Follow-up visit later
• Repeated typing and mouse tests
• Modified Remote Commander based on observations

9
Observations

• Physical Environment
• Needed lighter longer stylus
• Donated by Handango
• Needed better lighting to see Palm
• Cradle not sufficiently stable needed long cord
• Donated by Synergy Solutionshttp//www.slideshowc
  ommander.com

10
Adaptations to Remote Commander

• Made on-screen buttons for left, right mouse
  buttons and keyboard modifiers
• Physical Palm buttons toodifficult to press
• Made on-screen buttons toggleso stay on

11
Adaptations, cont.

• Turn off key repeat
• So not repeating by accident
• Faster Acceleration
• Smaller movements on Palm
• More flexible tapping
• Click mouse button bytapping on mouse pad area
• Tap anywhere in Graffiti area gets keyboard
• Long time before power-off
• Palm power button too hard to press

12
Adaptations, cont.

• Various Keyboard Layouts
• Bigger buttons
• Not popular liked having mouse region at top
• Alphabetic layout
• Most common keys grouped together to minimize
  travel distance
• Not particularly helpful

13
Observed Problems

• Battery life is a real barrier
• Need replacement after 3 days
• Solution use rechargeable model
• One subject got sore neck from repeatedly
  switching from looking down at Palm and up at
  screen
• Learned Graffiti for text entry and then didnt
  need to look at Palm as much

14
Problems, cont.

• Still often slower than regular keyboard mouse
• Will try word prediction

15
New! Word Prediction

• Predicts word from first letters
• Also, predicts next word from previous word
• Layout to minimize distance from keys
• Most likely choice in center and highlighted
• Available with soft keyboard and Graffiti

16
Observed Advantages

• Fatigues less with RemoteCommander than with
  regular keyboard, so can use PC for longer time
• Inexpensive!
• Keyboard and mouse in one place
• Ability to control PC from across room
• Difficulties in moving to computer from desk
• Ability to work with regular keyboard mouse
• Computer shared by multiple people

17
Another Pebbles Application

• Shortcutter allows users to create panels of
  controls for any PC application
• Can provide accelerators
• E.g. special buttons to make it easier to play a
  game
• Can start applications
• Can control X-10 devices (lights, etc.)
• One subject created a panel to start his favorite
  applications

18
Future work

• Evaluation of word prediction
• Handhelds to help people with other
  disabilities/illnesses
• ALS, cerebral palsy, some kinds of stroke
• Handhelds as assistive technologies in other
  areas
• Handhelds as interface to other devices
• Control room lights, telephone, wheelchair, etc.?
• Personal Universal Controller
• Ideas welcome!

19
Thanks to Our Sponsors!

• Supported by grants from
• DARPA
• Microsoft
• Pittsburgh Digital Greenhouse
• NSF
• General Motors
• And equipment grants from
• Hewlett Packard
• Lucent Technologies
• Palm Computing
• Symbol Technologies

• IBM
• SMART Technologies, Inc.
• Synergy Solutions, Inc.
• Handango

20
Using Handhelds to Help People with Motor
Impairments
Human Computer Interaction Institute School of
Computer Science Carnegie Mellon University
Thank you!

• Brad A. Myers, Jacob O. Wobbrock, Sunny Yang,
  Brian Yeung, Jeffrey Nichols, and Robert Miller
  Download our software!!http//www.cs.cmu.edu/pe
  bbles/assistive/

Assets2002
21
Shameless Plug

• This talk being given usingour SlideShow
  Commander
• Developed as part ofPebbles research project
• Licensed as a commercialproduct
• For Palm or PocketPC
• http//www.slideshowcommander.com

22
Related Work

• Adaptations to OS and software
• Microsoft accessibility modes
• Adaptations to hardware devices
• E.g., alternative keyboardshttp//www.microsoft.
  com/enable/
• Handhelds for other communities
• Communication aides Enkidu http//www.enkidu.net
  
• Research as cognitive aide Archimedes 16
• Predictive keyboards and other input techniques
  for handhelds

23
Alternative Input Devices

• Less than 60 of people who need assistive
  devices use them
• Not known
• High cost
• Decreasing utility over time
• Easier for users to develop their own
  alternatives!

24
Pebbles is
DAs for ntry of oth ytes and ocations
from xternal ources.

• P
• E
• B
• B
• L
• E
• S

http//www.pebbles.hcii.cmu.edu/
25
Mouse Test

• Modification of classic pointing device test
• Click on red target, which moves from left to
  center to right
• 3 sizes, 2 distances
• Measure time anderrors

26
Typing Test

• Typing test from Soukoreff MacKenzie 17
• Type target phrase exactly
• Capital andlower-caseletters, punctuation