Interactivity

1
Interactivity Design

• David Kirsh
• Dept of Cognitive Science
• UCSD

2
Question
Designing for experience
is it similar to Designing for efficiency
3
Topics

• Physical interactivity - an example
• Efficiency
• Fundamental Design Principle
• Metrics
• Experience
• Conclusion

4
Physical Interactivity an example

• Complex folding sequence
• Easy to forget
• Continuous interaction
• Difficult for one person
• How would you make this process easier?
• Think context aware, digital elements in around
  cardboard.

5
Video
6
Design it better!

• Embed instructions (digital or physical)
• Where, when, how
• Improve manipulability
• Where, when, how

7
Fundamental Design Principle
8
Design Principle for Efficiency
For a given task, design an environment so that
it provides the
to support efficient work activity.
9
Right for what?

• Pragmatic efficiency (getting job done quickly,
  few errors)
• Cognitive Efficiency
• Optimizes certain performance metrics
• Enhances experience of working/acting?

10
Right(Info, Tools, Cues, Constraints, Affordances)
For cardboard task
All revealed and concealed to manage attention
11
Right Info

• What is the right information to spatialize?
• Recipe study
• problems with modularizing info
• Origami study
• What activities should you support?

12
Spatially Distributed Recipe
13
Spatially distributed
RECIPE STIMULI Spatially Distributed - improve
locality
After learning from pilot
14
Spatially Distributed Activity Map Preparation
1/2 OF RECIPE
COOKING TOOLS
Participant
POTS PANS
INGREDIENTS
2/2 OF RECIPE
Laying out ingredients, chopping, washing,
measuring
15
Results

• Participants made better use of space
• Used more surface, stabilized better, prepared
  better
• BUT
• took much longer,
• looked at recipe more
• Wanted to look ahead!
• Implication we dont always know what
  information is needed by users

16
What do subjects do?Origami
To support activity we need to know what users
do - their routines etc. .. the task structure.

• More than fold
• flip over, inflate, rotate, register, point, try
  out in gesture form
• Pragmatic actions needed to complete structure
• on sub-goal trajectory
• fold and some non-folding actions - flatten, flip
  over, inflate
• But we saw other actions that were not pragmatic
  but which seemed important for the subject
• Only a fraction of the actions performed are
  represented explicitly in origami instructions

17
Non-pragmatic Hand Actions
Registration Verification Gestural
Thought Focusing Attention Trying out -
exploratory
18
Upshot

• Choosing the right information to spatialize is
  hard
• Even when workflow is known
• What should be shown, when and where?
• But often we havelittle idea what thereal
  workflow involves

19
Right Form
Given some information content, cue, constraint
or tool how should it be displayed to support

• Cognitive efficiency
• Faster processing

20
That is something up with which I will not put.
21
I wont put up with that.
22
Police police police police police.
23
Police whom policemen police also police
other police.
24
Topology vs. algebra

• Topological constraints are more natural

25
Right Form Modality
Given some information content, cue, constraint
or tool how should it be displayed to be the

• Right modality
• Visual decisions are visual
• Audio statements free visual search

26
Interface 2
27
Interface 3
Rely on recognition rather than memory. Show,
dont just tell
28
Right Form
Given some information content, cue, constraint
or tool how should it be displayed to have
Visibility - cue stands out
29
Right Form visibility
Is there a non-R?
30
Right Form visibility
31
Upshot

• Each step or phase in a routine or activity
  requires information or cues to be in the right
  form
• The right form may vary with step and task
• There are some general principles

32
Right Place

• The information or cue should be placed where you
  need it - given your resources and workflow

Starbucks cup
33
Right Place
Volume, Channel Buttons
On steering wheel - workflow
On radio - logically
34
Right Place
35
Interface 2
Options are where they should be
Goes together semantically ? goes together
visually
36
Right place for cue when there are distracters
Better place
reduce descriptive complexity reduce visual
complexity
37
Upshot

• Spatializing information correctly depends not
  just on workflow and resources but on
• Showing semantic or work significant relations
  between information or cues - what goes with what
• A theory of attention
• Should we use P or RRR P?
• Visibility or Place

38
Right Time

• See what I need to at this stage of activity
• Eg. Jigsaw puzzle might show perimeter pieces
  first
• Show instructions, cues, tools just when needed
  in workflow
• Hide tools in Illustrator that cannot be used in
  current context
• Encourage right sequence - soft constraints
• Travel If I choose departure time first then
  calendar for return time can be autoset
• Show horizon of relevant options

39
Right Time relevant options
Given structure of task - order of sub-goals -
show only relevant actions
D
H or M
P
A B C
P
40
Upshot

• Time, place and form interact
• Get them right and users have what they need to
  make an informed choice right at their fingertips

41
Right Pace

• Game coming at you too fast
• Activity has a natural frequency
• Slides in a presentation - comprehension rate

42
Right Pace blind to change if slow
43
Change blindness
Fast enough and you see it.
Before
After
44
Upshot

• Pace is the overall speed users find comfortable
  when performing their tasks
• Pace can change with mood and other user states

45
Bottom Line

• To give the right information at the right time
  is equivalent to creating a
• Dynamic keyhole - cognitive sweet spot

46
Metrics
47
Speed Accuracy Design
How fast can you fold the cardboard house?
48
Learnability Design
How long did it take to master the folding
routine?
49
Complexity Design
How much more complex structures can you fold?
50
Error Recovery Design
If you make an error how long does it take to
recover?
51
Variance Design
How much variance is there in your performance?
52
Bean Counters Design
How often do you damage or ruin the cardboard?
53
Interim Summary

• Designing for efficiency is hard
• Requires developing a science of
• Workflow analysis
• Cue effectiveness, task informativeness
• Cognitive complexity of forms
• E.g. visual simplicity, representational
  efficiency
• Impact of timing on routine efficiency
• Impact of pace on attention, routine efficiency
  and comprehension

54
Experience
55
Feeling versus Efficiency
56
What feeling is there in state space environments?

• Timeless
• do things in sequence but state transitions
  have no function beside moving to next state
• Qualitative feel of transition is irrelevant

57
States and Actions

• At each node there are
• Possible actions
• Goal is to reach end state
• by selecting actions

58
Reflection One

• Experience is essentially in time
• People are still experiencing even when system is
  stopped
• Experience is continuous never discrete
• Its about the time between the state
  transitions
• About the process more than the outcomes
• Longer beautiful route rather than shortest uglier

59
Coverflow
What matters is the feel of the transition
Smooth, nice reflections - non-pragmatic
properties
60
Design Principle

• For state space environments
• E.g. Forms, wisards, most websites, database
  queries, dialogue based interfaces, most tool
  based interfaces (photoshop)
• Design principles needs additionRight() right
  (.) that makes user feel good

61
Reflection Two

• What matters is
• How agent engages an object or environment
• He was hungry and attacked his food with gusto

• How agent feels when acting or watching
• While patting his dog he felt love and devotion

62
Continuous Control environments
Actions are continuous so not like state
space operators - everything is a transition
63
Continuous control
Highly interactive gesture based
Theremin the first musical instrument designed
to be played without being touched.
64
Engagement and Feeling

• Depends on inner state hormones, appetite,
  conditioning
• Agent can learn to engage or to feel

65
Reflection Three

• Two forms of experience
• Consciousness - qualia
• Nature of sensation
• Cut a tomato - sharp knife
• Visceral feeling
• Content - interpreting as, seeing as,
  categorizing

Ecstasy of Fruit Loops
66
Qualia

• Varies between individuals
• Sensitive to inner state hormones, emotions,
• Some qualia - visceral feeling are the result of
  mirror neurons - physical empathy
• If affected by stage setting, anticipation,
  scripting then not purely qualia

67
Interpretation needs anchoring

• Cue effectiveness
• Image completion
• Word fragment completion
• Intentional framework causation vs.
  self-propelling, desires
• Rational gestures
• Behaving like a friend
• Criteria for making an inference-rich
  attribution

wrd frgmnt
68
Upshot

• Study of projection of meaning - what is
  recognized and why - is necessary to determine
• Right (cue) right (Form, time, place, pace) to
  engender an interpretation

69
Further reflections

• Pleasure and delight is also derived from
  self-expression
• Thought has an experiential component -
  intellectual delight is content related
• Absorption in environments often requires
  commitment (this work is important) or the
  willing suspension of disbelief - that avatar is
  Josh
• Flow states are desirable but mostly
  ill-understood

70
Conclusion

• Designing for experience in state space
  environments is about the time between
• Designing for experience in continuous
  environments is about delight, absorption,
  engagement, hormones, emotions
• Both efficiency and experience design requires
  developing a science of

71
The End