Dialog Design Gesture

1
Dialog Design - Gesture Pen Interfaces
This material has been developed by Georgia Tech
HCI faculty, and continues to evolve.
Contributors include Gregory Abowd, Jim Foley,
Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris
Shaw, John Stasko, and Bruce Walker. Comments
directed to foley_at_cc.gatech.edu are encouraged.
Permission is granted to use with acknowledgement
for non-profit purposes. Last revision January
2004.
2
Dialog Styles

• 1. Command languages
• 2. WIMP - Window, Icon, Menu, Pointer
• 3. Direct manipulation
• 4. Speech/natural language
• 5. Gesture pen

3
Agenda

• PDA overview
• Pen input styles
• Issues

4
How to use a PDA
5
Personal Digital Asst. (PDA)
Palm VII
Palm IIIc
Handspring Visor
HP Jornada
Apple Newton (1993)
6
PDAs

• Becoming more common and widely used
• Smaller display (160x160), (320x240)
• Few buttons, interact through pen
• Estimate 14 million shipped by 2004
• Improvements
• Wireless, color, more memory, better CPU, better
  OS
• Palmtop versus Handheld

7
No Shredder
8
Input

• Pen is dominant form
• Main techniques
• Free-form ink
• Soft keyboard
• Numeric keyboard gt text
• Stroke recognition - strokes not in the shape of
  characters
• Hand printing / writing recognition
• Sometimes can connect keyboard

9
Free-form Ink

• Ink is the data, take as is
• Human is responsible forunderstanding
  andinterpretation
• Like a sketch pad

10
Examples

• Digital Ink - CMU
• Video, CHI 98
• View it at www. ..
• Flatland - Xerox PARC
• Video, CHI 99
• View it at www. .

11
Soft Keyboards

• Common on PDAs and mobile devices

12
Soft Keyboard

• Presents a small diagram of keyboard
• You click on buttons/keys with pen
• QWERTY vs. alphabetical
• Tradeoffs?
• Alternatives?

13
Numeric Keypad -T9

• Tegic Communications developed
• You press out letters of your word, it matches
  the most likely word, then gives optional choices
• Faster than multiple presses per key
• Used in mobile phones
• www.tegic.com/t9

14
T9 Demo

• See video at www.

15
Cirrin - Stroke Recognition

• Developed by Jen Mankoff (GT -gt Berkeley CS
  Faculty)
• Word-level unistroke technique
• UIST 98 paper
• Use stylus to go from one letterto the next
  -gt

16
Quikwriting - Stroke Recogntion

• Developed by Ken Perlin
• UIST 98 paper

17
Quikwriting Example
p
l
e
Said to be as fast as graffiti, but have to learn
more
http//mrl.nyu.edu/projects/quikwriting/
18
Hand Printing / Writing Recognition

• Recognizing letters and numbers and special
  symbols
• Lots of systems (commercial too)
• English, kanji, etc.
• Not perfect, but people arent either!
• People - 96 handprinted single characters
• Computer - gt97 is really good
• OCR (Optical Character Recognition)

19
Recognition Issues

• Off-line vs. On-line
• Off-line After all writing is done, speed not an
  issue, only quality.
• Work with either a bit map or vector sequence
• On-line Must respond in real-time - but have
  richer set of features - acceleration, velocity,
  pressure

20
More Issues

• Boxed vs. Free-Form input
• Sometimes encounter boxes on forms
• Printed vs. Cursive
• Cursive is much more difficult to impossible
• Letters vs. Words
• Cursive is easier to do in words vs individual
  letters, as words create more context

21
More Issues

• Using context words can help
• Usually requires existence of a dictionary
• Check to see if word exists
• Consider 1 vs. I vs. l
• Training - Many systems improve a lot with
  training data

22
Special Alphabets

• Graffiti - Unistroke alphabet on Palm PDA
• What are yourexperienceswith Graffiti?
• Other alphabets or purposes
• Gestures for commands

23
Graffiti Demo

• See www. .

24
Pen Gesture Commands

• Might mean delete
• Insert
• Paragraph

Define a series of (hopefully) simple drawing
gesturesthat mean different commands in a system
25
Pen Use Modes

• Often, want a mix of free-form drawing and
  special commands
• How does user switch modes?
• Mode icon on screen
• Button on pen
• Button on device

26
Error Correction

• Having to correct errors can slow input
  tremendously
• Strategies
• Erase and try again
• When uncertain system shows list of best guesses
• ...

27
A Different Application

• Signature verification
• But not with a mouse )

28
The End