Delimiters for Selection-Action Pen Gesture Phrases in Scriboli

1
Delimiters for Selection-Action Pen Gesture
Phrases in Scriboli
Ken Hinckley Patrick Baudisch Gonzalo
Ramos Francois Guimbretiere Microsoft Research
2
The Traveling (pen) SalesmanProblem

• Copy Paste 1. Select Lasso Mode

3
The Traveling (pen) SalesmanProblem

• 2. Circle objects

4
The Traveling (pen) SalesmanProblem

• 3. Invoke Edit Menu

5
The Traveling (pen) SalesmanProblem

• 4. Choose Copy Command

6
The Traveling (pen) SalesmanProblem

• 5. Invoke Edit Menu again

7
The Traveling (pen) SalesmanProblem

• 6. Choose Paste

8
The Traveling (pen) SalesmanProblem

• 7. Acquire the copy

9
The Traveling (pen) SalesmanProblem

• 8. Drag copy to desired position.

10
The Traveling (pen) SalesmanProblem

• The well traveled pen.

11
The Traveling (pen) SalesmanProblem

• Waste time moving the pen
• Focus attention on small targets
• Same steps every time
• But what did I forget?

12
The Traveling (pen) SalesmanProblem

• 9) Forgot to reselect PEN mode. Oops.

13
Pen Computing Now?
14
Recognize Pen Gestures?

• Classic problem ink or gesture ?
• Hard problem any gesture could be ink
• All we need to solve this is ONE BIT of info
• Scriboli Make it a physical skill instead of
  guessing game
• physical button on bezel
• nonpreferred hand, no trip to lasso mode
• Next paper Li, Hinckley, Guan Landay
• works really well

15
Gestures for Nouns Verbs

• Lasso selection Marking menus
• Crossing selection? Abandon lasso selection?
• How to combine scope command efficiently?

16
Delimiters for scope selection marking

• A delimiter is something different
• lexical structure of stroke
• e.g. self-crossing gesture stroke
• VIDEO delimiter techniques

17
Experiment Design

• Pigtail, Handle, Timeout, Button X
• Practice 32 trials
• Main Block
• All 8 Marking Directions (N, NE, ) X
• 2 Selection Types (single, or row/column) X
• 5 trials
• 80 main block trials
• Repeated Invocation (RI) Block
• Lasso one item, select E from menu 24x

18
Results Completion Time
Button Handle Pigtail Pigtail2
Timeout

• Left bar main block right bar repeated block
• Dotted bars are Pigtail2 design iteration

19
Results Learning Effects

• For the 24 trials of Repeated Invocation Block

20
Results Error Rate
Button Handle Pigtail Pigtail2
Timeout

• Error rate of selecting wrong marking direction

21
Improving Pigtail

• Problems
• 1. Pigtail Size
• No upper bound

22
Pigtail Design Iteration

• Problems
• 1. Pigtail Size
• No upper bound
• 2. Pretzels

23
Pigtail Design Iteration

• Problems
• 1. Size Limit
• Alt5600 no good
• 2. Pretzels
• 1st intersection? No.

24
Pigtail Design Iteration

• Problems
• 1. Size Limit
• Alt5600 no good
• 2. Pretzels
• 1st intersection? No.
• 2nd intersection? Yep.

25
Pigtail2 Preliminary Results
Button Handle Pigtail Pigtail2
Timeout

• No errors at all in Repeated Invocation block!

26
Scriboli DEMO

• Direct manipulation phase
• Scope, command, manip in one stroke
• Self-revelation techniques
• Disjoint Selections
• Lassos, Crossing, Tapping
• Circles of exclusion

27
Conclusion

• Performance data on delimiter techniques
• Data suggest Handle has slight advantage
• But handle was large at 48x48 pixels
• pen up/ pen down spatial mux
• Cant be used e.g. in paper interface
• Pigtail offers design alternative
• Performs almost the same, lasso tough case?
• Seems to offer some unique design advantages

28
Future Work

• grammar of building blocks for pen UI
• What are alternative designs for each part?
• More complex phrases possible?

29
Questions?
30
Handle vs.Pigtail

• Handle can get in the way
• Adds some visual clutter
• Pigtail handles more than one scope elegantly

31
Handwriting has a Deep Structure

• Learning cursive
• fast, brief, repetitive, pre-programmed
• variations on oscillatory movements
• The scribbling of Scriboli

32
Qualitative Results

• Favorite 7 Handle, 4 Pigtail, 4 Timeout
• Handle
• two strokes disruptive, natural
• Clutter box makes part of screen inaccessible
• Timeout
• good for beginners but I hate waiting for it.
• I felt like I was almost too accurate with it
• Pigtail
• Felt just like handwriting once I learned it, I
  could open up fly!
• Pigtail was trouble. I got a little mad at it.
• Takes a little bit of practice.

33
(No Transcript)
34
Marking Menus

• Blind Prompted
• Scale-independent in blind mode (333ms)
• Visually-guided in prompted mode

35
Preserves scale-independence

• Scale-ind. only matters for blind marking
• Dragging naturally lasts longer

marking starts
visual menu pops up
cross outer boundary
check for crossing
dragging
blind phase
333 ms
36
GEdit Kurtenbach Buxton
Delete one obj.
Copy add final C to move
Delete group end stroke inside lasso

• One of the first works considering pen phrases
• Every stroke is gesture
• Only 3 cmds
• Group vs. single obj. inconsistent

Move end stroke outside. Objects move on PenUp.
37
FlowMenu Francois Guimbretiere

• Integrates command selection with continuous
  parameter manipulation
• But no blind marking, size of gesture matters

Crossing in?out chooses Item submenu
Crossing out?in chooses Move command
Dragging continues
38
Design Philosophy of Scriboli

• RAPID
• Fast motions, repeatable, no waiting / dwelling,
  minimal demands on visual attention
• UNAMBIGUOUS
• No guessing games
• No reco unless I tell the system otherwise
• (There is almost no reco in Scriboli)
• EXPRESSIVE
• Supports many commands, easy to grow design
• Mechanisms for a wide span of pen apps, not
  restricted to specific niche application

39
Other Delimiter Techniques

• Handle
• 2 strokes
• Box makes part of screen inaccessible
• Timeout
• 500 ms dwell(stroke-and-hold)
• Button
• Hit when want to bring up menu
• too early / too late

40
Questions?