Leveraging Human Capabilities in Advanced User Interfaces

1
Leveraging Human Capabilitiesin Advanced User
Interfaces

• Mary Czerwinski
• Sr. Researcher
• Microsoft Research

2
Thanks to my Colleagues

• Desney Tan
• George Robertson
• Greg Smith
• Patrick Baudisch
• Brian Meyers

3
Introduction

• Overview of user-centered design and process
• Exploration of 3D UI designs that leverage human
  capabilities
• Examination of animation in UI
• Exploration of large displays and gender
  differences in 3D navigation
• Conclusion

4
User-Centered Design

• Start from existing user problem
• Ethnographic work, lab studies or literature
• Design innovation and brainstorming
• Psychological and human-computer interaction
  (hci) principles-driven!
• Prototype -gt show to users -gt iterate
• Extend principles, publish findings to hci
  community

5
Progress Chart

• Overview of user-centered design and process
• Exploration of 3D UI designs that leverage human
  capabilities
• Examination of animation in UI
• Exploration of large displays and gender
  differences in 3D navigation
• Conclusion

6
3D UI Projects

• Data Mountain (UIST 98)
• Task Gallery (CHI 2000)
• Polyarchy visualization (shipped June 2003)
• Scalable Fabric
• Large displays, optical flow and wider fields of
  view (CHI 2001, 2002, 2003)

7
Data Mountain

• User problemcant use IE Favorites
• Leverages spatial memory and visual recognition
• Strongest cue-
• relative size

Subject Layout of 100 Pages
8
Data Mountain Usability

• Study 1 (Compare with IE4 Favorites)
• Reliably faster (26)
• Study 2 (Longevity and Thumbnails)
• After 6 months, no performance change
• Images help, but are not required
• Studies 3 4 (Implicit Query)
• Faster retrieval if similar pages highlighted

9
Task Gallery

• User problem task switching
• Task management
• Simple, forward-back navigation
• Tasks laid out spatially on floor, ceiling, walls
• Simple task switch
• Leverages spatial memory, visual attention and
  recognition

10
Task Gallery

• Simultaneous viewing of multiple windows
• Simple shift select
• Smart arrangement
• Use 3D to provide uniform scaling
• Saves user from having to manage layouts

11
Progress Chart

• Overview of user-centered design and process
• Exploration of 3D UI designs that leverage human
  capabilities
• Examination of animation in UI
• Exploration of large displays and gender
  differences in 3D navigation
• Conclusion

12
Animation Effectiveness

• Tversky et al. (2001)
• Animation not always useful
• Info Vis Community
• Robertson, Card Mackinlary (91) Cone Tree
  node transitions involved rotations for
  maintaining context
• Bartram (98) animation evoked an emergent
  property of grouping when multiple, similar
  motions occur
• Bederson Boltman (98) 1 sec. animation
  significantly reduced errors and task times

13
Polyarchy Visualization

• Multiple Intersecting Hierarchies
• Solves user problems of
• Shows multiple foci at once
• Shows item relationships in context
• Manages viewing multiple hierarchies
• Key concept visual pivot
• Shipped June 2003

14
Two Styles of Visual Pivot

• Rotating
• Sliding

15
Rotation around Horizontal Axis
16
Sliding Animation
17
Proffitt and Kaiser (93)

• Users analyze animations into relative (rotation)
  and common (translation) motion components
• Secondly, rotation and translation motions have
  different perceptual significance
• Rotations define 3D form, while translations
  define observer-relative displacements
• Suggests sliding pivot perceived as
  observer-relative and rotating perceived as
  defining 3D form (less useful for our tasks?)

18
Polyarchy VisualizationUser Studies

• Study 3 Animation Styles and Speeds
• Six animation styles Picked two best
• Twice as fast as study 2 Still too slow
• Study 4 Prototype 2D vs 3D
• Identified most effective animation style
• Identified best speed range0.5 sec.
• Study 5 Examined complexity of query and sliding
  v. stacked animations

19
Study 4 Animation StylesSliding versus Rotating
20
Study 4 Animation Styles Learning Effects
21
Study 4 Animation Timing
22
Progress Chart

• Overview of user-centered design and process
• Exploration of 3D UI designs that leverage human
  capabilities
• Examination of animation in UI
• Exploration of large displays and gender
  differences in 3D navigation
• Conclusion

23
Ignore Science Fiction at Our Peril
Workstation in the world of the Matrix
24
Large Display Surfaces are Here
Workstation in the real world
25
Why A Larger Display Surface?

• Productivity benefits 15-30 (despite OS issues)
• Users prefer more display surface
• Prices dropping fast
• Footprints getting smaller

26
Multimon Trend is Growing

• (Jon Peddie Research
• Dec, 2002 N6652)

27
2004 Large Monitor ASP Projections
169 x 22 Diagonal
20Diagonal
2 x 17 (30 Diagonal)
1000
Relative Pricing
2 x 15 (26 Diagonal)
17Diagonal
15Diagonal
Single
Multiple
Wide
Note All Prices are for Liquid Crystal
Displays Source for Single Panel Pricing IDC and
Display Search
28
Large Display User Experience, MSR

• Large display surfaces fundamentally change user
  interaction
• Focus on input, visualization and windows
  management
• Large display surfaces provide non-linear
  productivity increases
• Additional space has different utility
• E.g. Focal/peripheral displays provide different
  cues

29
Windows and Task Management Issues Emerge

• Larger displays more open windows
• Multimon users arrange windows spatially
• TaskBar does not scale
• Aggregation model not task-based
• Users cant operate on groups of related windows

30
INPUT Drag n Pop

• Problems
• Large displays create long distance mouse
  movement
• Drag n Pop brings proxies of targets to the user
  from across display surfaces

31
Scalable Fabric (for Large Displays)

• Beyond Minimization
• Large display users keep more windows open
• With so much screen real estate, why minimize?
• Manage tasks using visual recognition and spatial
  memory
• Central focus area
• Periphery windows scaled
• Cluster of windows task

32
Women Take a Wider View (CHI 2002)

• Grew from work designing and evaluating 3D
  virtual navigation techniques
• On regular desktop display
• Men performed significantly better than women
• On exploratory widescreen display
• Overall improvement for all users
• Surprising finding
• Gender gap disappeared - Males and females
  performed equally on widescreen display

33
Related Work

• Formation of cognitive maps while navigating 3D
  virtual worlds
• Spatial abilities
• Artifacts (maps, landmarks,)
• Gender differences in spatial ability and
  navigation strategies
• Most report male advantages, especially in
  virtual environments

34
Related Work Optical Flow

• Changing retinal image as we move through the
  environment
• Aids perception of environmental structure

35
Related Work Optical Flow

• Changing retinal image as we move through the
  environment
• Aids perception of environmental structure

36
What we know about Optical Flow

• Optical flow benefits heading perception in
  active navigation
• Shown for fields of view up to 90 degrees
• Hypothesized that effectiveness of optical flow
  depends on spatial ability
• Cutmore et al. 2000
• Gender unexplored

37
Our Hypotheses

• Optical flow cues help all users form better
  cognitive maps when navigating 3D virtual
  environments
• Better optical flow cues help women more than men
  in cognitive map formation
• Wider displays offer even better optical flow cues

38
Dsharp Display
43"
11"
39
Task General Description

• Learning User controls movement along path
  through virtual 3D maze
• Testing Remember path traveled

40
Virtual Maps

• 14 rooms (6 straight ahead, 8 turns)
• Some paths go through same room twice
• For example

41
Cognitive Map Learning

• Use arrow keys to go through green door
• Determine if path crosses itself
• Remember full path

42
Cognitive Map Memory Test

• Tested on memory for maze
• Forward test and backward test
• Measured task time number of correct doors
  opened on first attempt
• Same controls as in learning phase, but without
  green door guides
• Given feedback

43
Experimental Design
Female
Male
Large FOV 120 degrees
Small FOV 100 degrees
Optical Flow Absent
Optical Flow Present
44
Experimental Procedure

• Paper folding test of spatial ability
• 1 practice trial 4 test trials
• Satisfaction questionnaire

45
Benefits of Optical Flow
46
Optical Flow Helps All Users in Forward Test
Forward
Backward
47
Optical Flow Benefits Females More in the Forward
Test
Females
Males
48
Other Results

• No effects for field of view
• No effects for spatial ability measure
• Satisfaction ratings matched performance results

49
Conclusion

• Optical flow cues help all users form better
  cognitive maps when navigating 3D virtual
  environments
• Better optical flow cues help women more than men
  in cognitive map formation
• Unexplained by biases in spatial ability
• Wider displays offer even better optical flow
  cues
• 100 degree field of view seems sufficient

50
Information Voyeurism Social Impact of Large
Displays

• Exploit social cues induced by physical size
• Help people communicate
• Increase productivity on individual tasks
• Must quantify in order to exploit
• Information on large displays more public
• Ask user? Cannot guarantee accuracy
• Video? Cannot disambiguate glance from reading

51
Measuring Peeking

• Implicit memory priming paradigm
• Expose user to stimulus
• Test user implicitly on how much theyve
  processed stimulus
• Word stem completion
• Eg. Mon_____
• Priming measured by faster response or higher
  frequency of stimulus
• Monkey, Money, Monster, Monday, Monopoly,

52
Experiment Materials

• Stimulus 30 words embedded in
• 7 e-mail subject lines
• 2 e-mail messages
• Place e-mail where it can be seen by user
• Priming test to see if theyve read it
• Word stem completion

53
Experimental Setup
54
Implicit Memory Results
N12
Average of Target Words
N12
Small Display
Large Display
55
Other Converging Data

• More users admitted reading text on
• Large Screen (7/12) vs.
• Small Screen (3/12)
• Comments indicated reading someone elses e-mail
  more acceptable on large screen
• Video shows users glanced more at
• Large Screen (M19 seconds) vs.
• Small Screen (M14 seconds)

56
Design Implications

• Protect private information from prying eyes
• Private information never placed on public
  screens
• Interface conventions that convey level of
  privacy
• Facilitate ad hoc collaboration
• Display systems that make people interact more

57
Progress Chart

• Overview of user-centered design and process
• Exploration of 3D UI designs that leverage human
  capabilities
• Examination of animation in UI
• Exploration of large displays and gender
  differences in 3D navigation
• Conclusion

58
Conclusion

• Successful, advanced user interface design
  requires knowing the user problem to be solved
• As opposed to being technology-driven
• Leveraging human capabilities ensures
  improvements over existing techniques
• Usable designs
• Contributions to science
• Principles derived for human-computer interaction
  discipline

59
Thank you
60
Large Display Surfaces are Here
Workstation in the real world (InfoCockpit--CMU)
61
User Views of Maze
Narrow field of view (100 degrees)
Wide field of view (120 degrees)
62
Visual Pivot (Rotation around Vertical Axis)
63
Schematic of Visual Pivot (horizontal rotation)