Dialog Design Gesture

1
Dialog Design - Gesture Pen Interfaces, Mobile
Devices
This material has been developed by Georgia Tech
HCI faculty, and continues to evolve.
Contributors include Gregory Abowd, Jim Foley,
Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris
Shaw, John Stasko, Bruce Walker, and Melody Moore
Jackson. Comments directed to foley_at_cc.gatech.edu
are encouraged. Permission is granted to use with
acknowledgement for non-profit purposes. Last
revision October 2007.
2
Dialog Styles

• 1. Command languages
• 2. WIMP - Window, Icon, Menu, Pointer
• 3. Direct manipulation
• 4. Speech/natural language
• 5. Gesture pen

3
Agenda

• PDA overview
• Pen input styles

4
How to use a PDA
5
Personal Digital Asst. (PDA)
Apple iPhone
Palm Treo
Dell / Asus A639 GPS PDA
Apple Newton (1993)
Blackberry Curve
6
PDAs

• Now ubiquitous
• Small displays
• Often touch and pen interfaces
• Small thumb-based keyboards
• Recent Improvements
• Wi-Fi, GPS, more memory, better CPU, better OS,
  BlueTooth

7
Is it a PDA? Phone? GPS? Camera? Computer?

• Line between devices is blurred today
• Apple iPhone phone, MP3 player, PDA, camera
• Palm Treo 700w phone, Windows computer, PDA,
  camera
• Asus MyPal Windows computer, GPS device

8
No Shredder
9
Input Options

• Pen / Stylus is dominant form
• Main techniques
• Free-form ink
• Soft keyboard
• Numeric keyboard text
• Stroke recognition - strokes not in the shape of
  characters
• Hand printing / writing recognition
• Sometimes have or can connect keyboard

10
Free-form Ink

• Ink is the data, take as is
• Human is responsible forunderstanding
  andinterpretation
• Like a sketch pad

11
Soft Keyboards

• Common on PDAs and mobile devices

12
Soft Keyboard

• Presents a small diagram of keyboard
• You click on buttons/keys with pen or finger
• QWERTY vs. alphabetical
• Tradeoffs?
• Alternatives?

Apple iPhone soft keyboard
13
Numeric Keypad -T9

• Tegic Communications developed
• You press out letters of your word, it matches
  the most likely word, then gives optional choices
• Faster than multiple presses per key
• Used in mobile phones
• www.tegic.com/t9

14
Stroke Recognition - Graffiti

• Graffiti - Unistroke alphabet on Palm PDA
• What are yourexperienceswith Graffiti?
• Graffiti demo
• mms//199.77.128.107/pub/hcirep/demos/DEMO-palm.wm
  v

15
Stroke Recognition - Cirrin

• Developed by Jen Mankoff (GT - Berkeley CS
  Faculty)
• Word-level unistroke technique
• UIST 98 paper
• Use stylus to go from one letterto the next
  -

16
Hand Printing Hand Writing Recognition

• Recognizing letters and numbers and special
  symbols
• Lots of commercial systems
• English, kanji, etc.
• Not perfect, but people arent either!
• People - 96 handprinted single characters
• Computer - 97 is really good
• OCR (Optical Character Recognition)

17
Recognition Issues

• Off-line vs. On-line
• Off-line After all writing is done, speed not an
  issue, only quality.
• Work with either a bit map or vector sequence
• On-line Must respond in real-time - but have
  richer set of features - acceleration, velocity,
  pressure
• Use best-guess pattern matching, including
  digram, trigram probabilities and word lists to
  remove ambiguity
• 1 I l

18
More Issues

• Boxed vs. Free-Form input
• Sometimes encounter boxes on forms
• Printed vs. Cursive
• Cursive is much more difficult to impossible
• Letters vs. Words
• Cursive is easier to do in words vs individual
  letters, as words create more context

19
Pen Gesture Commands

• Might mean delete
• Insert
• Paragraph

Define a series of (hopefully) simple drawing
gesturesthat mean different commands in a system
20
Pen Use Modes

• Often, want a mix of free-form drawing and
  special commands
• How does user switch modes?
• Mode icon on screen
• Button on pen
• Button on device

21
Error Correction

• Having to correct errors can slow input
  tremendously
• Strategies
• Erase and try again
• When uncertain system shows list of best guesses
• ...

22
A Different Application

• Signature verification
• But not with a mouse )

23
Multi-touch interfaces

• Apple iPhone

Capacitive touchscreen http//www.apple.com/iphon
e/features/index.htmltouch Gestures flick,
tap, pinch, un-pinch http//www.apple.com/iphone/
features/index.htmlmap