Cognition

1
Cognition Perception

• It's not about making computers more human, but
  about removing the barriers between humans and
  computers (B.J. Fogg)

2
Overview

• Motivation for perception and cognition
• Design possibilites
• Perception
• Visual, auditiv, haptic, taste, smell
• Multimodal systems
• Cognition
• Attention
• Communication
• Emotions

3
What motivates such knowledge?

• Knowledge of perception and cognition (already
  has and) will affect areas like AI, Ambient
  Intelligence, HCI, design, architecture and more
  

4
Intelligent products

• An intelligent product is an everyday artefact,
  where computation is used to invisibly support or
  enhance its intended use
• (Lars Erik Holmquist)

5
Ambient Intelligence

• Ubiquitous computing (system integration, ad
  hoc-, wireless networking)
• Context awareness (sensors, tracking, position)
• Intelligence (learning algorithms, user
  profiling, recommenders, autonomous intelligence)
• Natural user -system interaction, ambient
  technologies, multimodal interaction, interaction
  styles
• European Symposium on Ambient
• Intelligence Nov 3-4, 2003

6
User perspective

• The users goal is not to interact with an
  intelligent product but to
• create, communicate, explore, plan, draw, design,
  learn
• (Keyson)

7
Interaction Design is about

• Defining the behavior of artifacts, environments,
  and systems (i.e., products) and therefore
  concerned with
• Defining the form of products as they relate to
  their behavior and use
• Anticipating how the use of products will mediate
  human relationships and affect human
  understanding
• Exploring the dialogue between products, people,
  and contexts (physical, cultural, historical)
• (R. Reimann)

8
Cogntion Perception

• Relates to how humans percieve their world,
  reason and act in it.
• Any design of products, intelligent or not, will
  be percieved and interacted with, based on
  peoples perception, cognition in a specific
  situation.

9
Cognition

• Processes that have a content (Lundh,
  Montgomery, Waern, 2001)
• Perception
• Memory
• Language
• Thought

10
Perception

• The acqusition and processing of sensory
  information in order to see, hear , taste, smell,
  or feel objects in the world also guides an
  organisms actions with respect to those objects.
  
• Sekuler Blake (1994)

11
The desktop computer

• How are our senses used in the design?
• Acting by touch, getting visual feedback, ..
• sound feedback
• touch feedback?
• taste, smell?

12
Consider the things we've given up in the
physical world which might be nice to have back

• but augmented with computation and connectivity.
  Paintbrush and pencils and musical instruments a
  single personal key that lets you into home, car
  and work and has a distinct feel as you insert it
  in a lock depending on whether your spouse, a
  friend or a stranger has been by in your absence
  a bank card that feels as heavy as your account
  balance when you swipe it in the ATM.

13
Two perspectives of how to use knowledge of
human perception

• for systems/artefacts that have attentive and
  reactive users (HCI)
• for designing attentive and reactive
  objects/systems (AI or not)

14
First perspective Using perception in design

• Information visualisation (Tufte, 1990)
• General HCI design guidelines, like
• Group similar information
• Give proper feedback..
• Minimalistic design

15
Using perception in design

• Affordances of objects (Norman, 1990)
• Constraints that guides use and can be physical,
  logical, semantic or cultural.

16
Affordances
17
Artefacts that imitate perception

• Furby, A/Barney, Aibo, Spookies
• Sensing capabilites similar to humans
• Constrained sensing capabilites vs. capabilites
  beyond humans

18
Back to our perception

• The acqusition and processing of sensory
  information in order to see, hear , taste, smell,
  or feel objects in the world also guides an
  organisms actions with respect to those objects.

19
Visual perception

• Ocularcentrism
• We live in a visual society

20
(No Transcript)
21
(electron micrograph by Scott Mittman)
22
Visual perception

• The brain assume that we live in a
  three-dimensional world
• The brain tries to find depth in everything that
  it sees.
• The brain assume that everything is constant in
  shape, color and size.

23
Visual perception example
24
Visual perception

• Organisation of objects
• Movement perception
• Spatial perception
• Perception of objects

(e.g. Gestalt Principles)
25
Gestalt Principles

• The principle of
• Proximity
• Similarity
• Good continuity
• Closure
• Movement

26
Gestalt Principles

• The principle of Proximity
• features which are close together are associated

27
Gestalt Principles

• The principle of similarity

28
Gestalt Principles

• The principle of good continuity

29
Gestalt Principles

• The principle of closure
• Close unfinished shapes
• When the information is diffuse or indistinct,
  our expectations will create the impression for
  us..
• Group information

30
Closure
31
Perception of objects

• How do we recognize something e.g. A chair as a
  chair?
• Figure/ground organization
• Learned Stereotypes or prototypes
• a A a A a A
• Useful when designing icons

32
Spatial perception

• We perceive relative sizes and distance by using
• Perspective (e.g. Linear)
• Overlap
• Known sizes
• Texture gradients
• Shading
• Absolute distance- Relative distance

33
(No Transcript)
34
Movement perception

• The principle of Movement
• React fast on movement
• The eye is drawn towards movement
• Movement towards background

35
Hearing

• Recognize identity of soundsource
• Give information on the nature of the
  environment via echoes, reverberation (normal
  room, cathedral, open field)

36
Hearing example

• Multimodal system presented
• at the Interact Conference03
• Non-visual exploration of digital pictures
  (Root, 2003)
• Active auditory representation
• Passive auditory representation (Verbal summary)
• Haptic representation (contour, surface

37
Smell

• Olfactory
• Can distinguish different 10.000 smells
• A chemical sense (A substance has to be
  volatile)
• Recognition
• influence mood, memory, emotions
• We can actually communicate by smell

38
Smell example

• Fire fighting training, remote surgery,
  entertainment..
• Joseph Kaye (MIT)
• The olfactory display of Abstract Information
• Remember dinner at five (smelling curry)

39
Taste

• Also a chemical sense
• Gustatory sensation
• Substance must be soluable
• Sour, salty sweet, bitter by tastebuds are also
  located in cheeks, in the throat
• Recognition, Influence memory

40
Taste example

• Food simulator, Siggraph Conference 2003
• Haptic displays displays with biting force
• Auditory, chemical sensations of eating

41
Tactile perception

• Haptics
• Thermoreceptors (temperature)
• Mechanoreceptors (pressure, vibration)
• Painreceptors
• Kinesthetics
• Vision/audition usually dominate haptic
  perception
• Muscle memory

42
Haptics example

• Tangible interfaces
• Brygg Ullmer (MIT and ZIB)
• Tangible bits
• MetaDESK
• MediaBlocks
• Tangible Query Interfaces

43
Why use tactile information?

• Clarify ambiquity in dominant modality
• Some parameters not availible in dominant
  modality
• Continous control vs Discrete control
• (Karon Maclean)

44
Multimodal distinctions

• Crossmodal, intermodal. One modality
  subconsciously influences perception in another
  modality
• Multimodal an event is perceived and integrated
  by multiple senses
• Supramodal phenomenon that applies to all
  senses.
• Grounded vs ungrounded interfaces

45
A multimodal example
46
Why use multimodal interaction?

• Avoid overkill find most efficient path to
  desired result
• Exploit illusions work around hardware
  limitations to clever compensation
• Design rules control net percept in user
• Ecological verity respect perceptual latency
  thresholds for perceived synchrony

47
But how many things can we attend to in the same
time?
48
Attention

• Selective ability
• Our brain compensates

49
Change blindness
50
Attention

• Guidance of attention
• High level control
• Low level salience (movement)
• Can send the attention of the user to an
  appropriate location at an appropriate time.
• Change blindness transitions can become
  effectively invisible if attention is not drawn
  to them.
• Soft warning User automatically sees what they
  should see!

51
Cognition

• The world as an external memory
• Distributed cogntion (Hollan, Hutchins, Kirsh,
  2000)
• Culture, history context affects distr
  cognition
• Situated cognition
• Embodied cognition

52
Cognition is also about language. and thus
communcation
53
Communication

• Verbal, Nonverbal
• Sender Reciever
• Language (speaking) are means for communication
• Technology can act as an amplifyer of our
  ability to communicate. Makes communication
  possible despite the distance e.g. Phones, sms
  Constraints/shapes the communication.
• People have to communicate with a device in
  order to be able to create, explore, plan, draw,
  design, learn

54
An example

• Amigo (instant messaging) vs. SMS

55
One last thing that affect cogntion and that is
affected by our senses
56
Emotions

• If you want a golden rule that will fit
  everybody, this is it Have nothing in your
  houses that you do not know to be useful, or
  believe to be beautiful.
• William Morris The Beauty of Life, 1880

57
Emotions

• Moods vs emotions towards an object
• Affective computing
• computing that relates to, arises from, or
  deliberately influences emotions

58
Affective design

• Robots and artificial creatures on computer
  displays are being designed to show more emotion.
  The goal is better communication with people.
• Panasonics Tama
• Necs Robot R100

59
So, why try to understand cognition and
perception?

• It affects how we percieve and interact with
  devices around us, thus also what you intend to
  design.
• It can be used as inspiration to think beyond
  current interfaces

60
Thank you for listening