PenCentric Shorthand Handwriting Recognition Interfaces - PowerPoint PPT Presentation

Title: PenCentric Shorthand Handwriting Recognition Interfaces


1
Pen-Centric Shorthand Handwriting Recognition
Interfaces
Charles C. Tappert1 and Jean R. Ward2
1 School of CSIS, Pace University, New York,
USA 2 Pen Computing Consultant, Massachusetts, USA
2
Background

• Famous writings throughout history were
  effectively written in a style of shorthand
• Ciceros orations
• Martin Luthers sermons
• Shakespeares and George Bernard Shaws plays

3
Thesis Pen-Centric Shorthand Interfaces

• Can provide critical infrastructure for natural
  pen-centric interactions, enhancing many
  pen-centric learning applications
• Can provide faster text input for pen-centric
  teaching, studying, and learning applications
• Will have greatest impact on the utility of
  applications running on small mobile devices

4
Agenda

• Fundamental Property of Writing
• Handwriting Recognition Difficulties
• Online (Pen-Centric) Handwriting Recognition
• Online more accurate than Offline Recognition
• Online Info Can Complicate Recognition Process
• Design Tradeoffs/Decisions
• Historical Shorthand Alphabets
• Pen-Centric Shorthand Alphabets
• Pen-Centric Word/Phrase Shorthand
• Allegro/Chatroom Shorthand System

5
Fundamental Property of Writing

• Differences between different characters are more
  significant than differences between different
  drawings of the same character
• This makes handwritten communication possible

6
Fundamental Property of Writing

• Property holds within subalphabets of uppercase,
  lowercase, and digits, but not across them
• I, l, and 1 written with single vertical
  stroke
• O and 0 written similarly with an oval

7
Handwriting Recognition Difficulties

• Shape, size, and slant variation
• Similarly shaped characters U and V
• Careless writing
• in the extreme, almost illegible writing
• Resolving difficult ambiguities requires
  sophisticated recognition algorithms,
  syntax/semantics

8
Online (Pen-Centric) Handwriting Recognition

• Machine recognizes the writing while the user
  writes
• Digitizer equipment captures the dynamic
  information of the writing
• Stroke number, order, direction, speed
• A stroke is the writing from pen down to pen up

9
Online (Pen-Centric) more accurate than Offline
(Static) Recognition

• Can use both dynamic and static information
• Can often distinguish between similarly shaped
  characters
• E.g., 5 versus S where the 5 is usually written
  with two strokes and the S with one stroke

10
Online Information Can Complicate Recognition
Process

• Large number of possible variations
• E can be written with one, two, three, or four
  strokes, and with various stroke orders and
  directions
• Four-stroke E has 384 variations (4! stroke
  orders x 24 stroke directions)

11
Online Information Can Complicate Recognition
Process

• Segmentation ambiguities
• character-within-character problem
• lowercase d might be recognized as a cl if drawn
  with two strokes that are somewhat separated from
  one another

12
Design Tradeoffs/Decisions

• No constraints on the user
• Machine recognizes user's normal writing
• User severely constrained
• Must write in particular style such as handprint
• Must write strokes in particular order,
  direction, and graphical specification

13
Simplified Design Tradeoffs/Decisions for
Graffiti and Allegro PDA Alphabets

• Small alphabet
• one case rather than both upper and lowercase
• Small number of writing variations per letter
• preferably only one
• One stroke per character (character stroke)
• allows machine to recognize each character upon
  pen lift
• Separate writing areas for letters and digits
• avoids confusion of similarly shaped letters and
  digits

14
Historical Shorthand Alphabets

• We first review the history of shorthand systems
  prior to pen computing
• Shorthand is a method of writing rapidly by
  substituting characters, abbreviations, or
  symbols for letters, words, or phrases
• Shorthand can be traced back to the Greeks

15
Historical Shorthand Alphabets

• We focus on shorthand alphabets that might be
  appropriate for PDAs
• We review two types of shorthand
• Geometric shorthand
• Small number of basic shapes
• Shapes reused in multiple orientations
• Non-geometric shorthand

16
Tironian Alphabet, 63 B.C.
17
Stenographie Alphabet, 1602
18
Stenographie Alphabet, 1602

• Geometric shorthand basic shapes/orientations

19
Moon Alphabet, 1894

• Geometric shorthand basic shapes/orientations

20
Other Historical Shorthand Systems

• Phonetic alphabets
• Pitman (1837)
• Gregg (1885)
• Systems for the blind
• Braille (1824)
• Cursive shorthands
• Gabelsberger (1834)

21
Pen-Centric Shorthand Alphabets

• Some of the earliest were for CAD/CAM
• symbols represent graphical items and commands
• Others developed for text input on small consumer
  devices like PDAs that have limited computing
  power
• We review geometric and non-geometric shorthands
  appropriate for small devices

22
Pen-Centric Shorthand Alphabets

• Historical alphabets presented above could be
  used for machine recognition
• symbols drawn with a single stroke (except K in
  Tironian and in Stenographie)
• In addition to shape and orientation, online
  systems can use stroke direction to differentiate
  among symbols

23
Allen Alphabet
24
Allen AlphabetBasic Shapes and Orientations
25
Goldberg Alphabet
26
Goldberg AlphabetBasic Shapes and Orientations
27
Graffiti and Allegro Alphabets

• Not geometric alphabets
• High correspondence to Roman alphabet
• Might not qualify as shorthand but included here

28
Graffiti Alphabet
29
Allegro Alphabet
30
Commercially Successful Shorthands

• Similar to the Roman alphabet
• Easier to learn
• Graffiti used in Palm OS devices
• notably the Palm Pilot and Handspring models
• Allegro used in Microsoft Windows devices
• Geometric alphabets not successful

31
Pen-Centric Word/Phrase Shorthande.g., Chatroom
Shorthand

• Further increase speed of text entry
• Potential applications
• Where input speed important
• Where word/phrase abbreviations occur frequently
  e.g., email

32
Allegro/Chatroom Shorthand System

• Developed for M.S. dissertation
• Student was hearing impaired
• Developed as output component of communication
  system
• Handwriting to text to speech
• Two input writing areas
• One for Allegro (all-purpose)
• One for chatroom-like words/phrases

33
Allegro/Chatroom Shorthand System
34
Allegro/Chatroom Shorthand System

35
Conclusions

• Pen-centric learning interfaces should use
  shorthand, and word/phrase shorthand where
  appropriate, for fast text input
• Benefit of shorthand interfaces
• Provides critical infrastructure for many
  pen-centric learning applications
• Enhances natural pen-centric interactions for
  teaching, studying, and learning applications
• Has greatest impact on the utility of
  applications running on small mobile devices