Geen diatitel

1
(No Transcript)
2
De religieuze connectie
3
Cognitive engineering II
Engineering of Human Cognition.
Perception
cognition
action
The Arjan Veringa connection
Strategies in (spastic) communication
Abstract word use (Levelt, Clark Brennan)
Perseverence across multi media conditions
4
The project X
"when does a complex task become more then its
components limits and constraints in task
integration
Automaticity
Classical definition
Control
perceptual
unconscious
exogenous
But comp. Tselgov
5
e.g. Car driving tennis, piano, luit, text
editing, etc. etc.
Initially Controlled
training
Automatic ?
6
What makes automaticity possible ?
Task structure (design !)
Limits and Constraints?
7
Simpele switch taak 1
Switchen tussen concurrerende taken
Taak 1 tel voor alle stims het aantal karakters.
Taak2 noem het type character van elke stim.
Switch taak 1e rijtje taak 1, 2e rijtje taak 2,
derde rijtje taak1, etc.
dd fff gggg k ooo ww zzzz gg e pppp s
gg rr wwww q kkk mm a hh jjjj oo hh
sss ff h kk ooo bbb yyyy l dd
x
8
The project X
"when does a complex task become more then its
components limits and constraints in task
integration
Automaticity
Control
perceptual
unconscious
exogenous
Task 1 and task 2 automatic
Why not task 3?
Removal of switch costs?
What causes switch costs?
9
No Unique cuing
Variable Mapping Function on Symbol types
Task 1
fff
Three
fff
Ef
Task 2
Task 3
Verbal response changes for similar stimuli
Mental Reconfiguration of Task Set
Mental effort/control (limited, no parallel IP)
10
Unique cuing
Simple switch task that automates
Switch task 1st number, 2nd letter, third row
number, etc.

• ddd 444 f 33 jj 444 gg 3333
• 11 pppp 11 gg 222 wwww 1 kkk
• 22 a 222 jjjj 33 hh 3333 ff
• 111 kk 33 bbb 444 l 2222 x
  

11
Occasion Setting
Semantic disambiguation
the mouse
on the mat
John moves
The cat catches
colour
coding
12
Unigue Color Coding
Switch task 1st stim number, 2e stim type, etc.
dd fff gggg k ooo ww zzzz gg g
ll ss tttt h ccc hhhh jj mm a
hh jjjj oo hh sss ff e zz
jjj kkk rrrr u ll y
13
Unigue Color Coding with Stroop
Switch task 1st stim number, 2e stim type, etc.
Stroop interference

• dd fff gggg k ooo ww zzzz gg
• 5 11 44 2222 3 111
• mm a hh jjjj oo hh sss ff
• 2 33 444 222 3333 2 44 2

14
DynaKey
ATM
Variable Mapping functions on buttons and menus.
15
Task 1 categorise by colour Task 2 categorise
by shape
16
(No Transcript)
17
Matching to sample Learning of new associations
between
18
Stimulus set 1
Stimulus set 2

Colour 1
Colour 2
A1
A2
X1
X2
B1
B2
Y1
Y2
C1
C2
Z1
Z2
Task I Task II Uniquely cued. Until
stimulus equivalent.
19
Stimulus set 1
Stimulus set 2

Colour 2
Colour 1
X2
A1
C2
B1
Y2
Z1
Trial Task I Trial Task II
20
Switch kosten op elke 9e trial
21
Measuring Mental Effort
Unobtrusive Sensing the Subject
Sensible and affective computing
22
(No Transcript)
23
(No Transcript)
24

• Output van de analyse (Praat)
• Proefpersoon-code x blij
• N 746
• Median 159.388862017289
• Mean 182.432672604596
• Standard deviation 111.00171688755
• Quantile 90 397.418566040441
• Quantile 10 92.519650497537
• delta Quantile
  304.898915542904
• Proefpersoon-code x droevig
• N 887
• Median 104.337808816211
• Mean 135.257805012306
• Standard deviation
  109.78115403165
• Quantile 90 132.530147765804
• Quantile 10 87.3119374482499
• delta Quantile 45.2182103175541

25
Formalisation
Cognitive Engineering (CE) in the NICI
organisation.
CE and Applications
Motor function
Perception
Psycholinguistics
behavioural level
organ level
cellular level
molecular level
Neuronalisatio
26
De mens, omgeven door alom aanwezige intelligentie
27
Breeding and Taming
A cognitive Science Perspective
Sloterdijk Het Mensenpark
28
Developments in Biology and Informatics
Moral and Ethical aspects
Autonomous Agents (who makes the decisions)
What plug ins are alowed?
Who controls the Cyborg
What organisms should we grow (DNA)
Shall the Human spiecies disappear?
29
Cognitive Science AI and CE
The (accepted) Narrow vision
Artificial Means in artificial hard/software
AI Smart computers
CE userfriendliness
30
The broad vision
The Knowledge System
Compare Pylyshyn the Informafore
Independent of implementation and Hardware.
Are Woman Human?
What are the criteria for being a knowledge
system?
31
Criteria for being a KS
Compare Turing
Performance Does some body act as a knowledge
system?
Kan je even de deur dicht doen
The perennial Question What is knowledge,
rationality consciousness, a conscience, nature
vs. culture etc. etc.
32
Animals Poor emulators
Humans and Computers Very advanced emulators
Tools Language, Movies, Bill boards,
Environments.
Implicit and Explicit learning
Humans Implicit is the rule, explicit the
exeption
Computers Explicit (programming) the rule,
implicit the exeption
33
Computers and Humans develop towards oneanother
As computers become more and more intractable
informatics evolves towards computer
psychology comp. current mentalising of
computer performance
Humans have always been computers, but will
develop plug in calabilities Cyborgs. Psychology
becomes human informatics
34
What may cognitive science do to uncover the
engineering potential or evolution of knowledge
systems?
Cognitive engineering Science based design of
environments to influence knowledge.
To Know is to study.
35
De religieuze connectie
36
Cognitive Engineering
The taming of cognition
Endogenous Creation of a certain taskset
No taming possible the first neural levels
37
(No Transcript)
38
Cognitieve ergonomie als cognitive engineering.
Ab de Haan Cognitieve
ergonomie kan gezien worden als het op grond van
wetenschappelijke inzichten ontwerpen van
omgevingen, technologische en andere, zoals
communicatie methodieken en behandelmethoden, om
de menselijke cognitie te beïnvloeden en kan in
die zin gezien worden als een variant van
cognitive engineering. Hierin onderscheid zij
zich van bijvoorbeeld kunstmatige intelligentie,
die primair gericht is op de beïnvloeding van
machinale cognitie. Om iets te kunnen
beïnvloeden, moet je het wel kennen. Vandaar het
belang van fundamentele inzichten uit de cognitie
wetenschap. Tijdens de voordracht zal aan de hand
van een aantal voorbeelden dit nader worden
toegelicht. Hierbij staat met name het
perspectief van taken als uitdrukking van
(neurale) configuraties en herconfiguraties,
zoals die bijvoorbeeld in task switching,
voorkomt centraal.
39
(No Transcript)
40
trends in Cognitive Sciences red blue xxx green
green green mmmmmm blue yellow red hhhh yellow
red blue sssss green blue yellow hhhh red green
blue xxx blue blue green sssss yellow red red xxx
red yellow yellow mmmmmm green blue green sssss
red yellow yellow mmmmmm blue green red hhhh
yellow Time 1 234 Fig. I. The Stroop task.
41
De originele Stroop Taak
Rood Blauw Paars Geel Groen Blauw Graaf Groot
Bloot Prijs Broek Rand Groen Geel Blauw Paars
Bruin Rood Very much a what task, But automated
42
A la Yersild kleur, woord, kleur, etc.
Geel Groen Bruin Rood Paars Groen Geel Blauw
Paars Bruin Rood Geel Groen Bruin Rood Paars
Groen Geel Blauw Paars Bruin Rood Bruin
Paars Geel Groen Bruin Rood Paars Groen Geel
Blauw Paars Bruin Rood Geel Geel Groen Bruin Rood
Paars Groen Geel Blauw Paars Bruin Rood
Rood Very much a what task, But automated
43
Picture word interference

• bear

Semantic Inhibition (Roelofs)
44
Tasks
Levels of description
What
(requirements)
How
(functional specification)
45
What / where
Frontal control
46
(No Transcript)
47
A what task Introduction

• Optimising human-computer interaction
• Multimedia environments
• Non-speech audio
• Human-human communication as a starting point
• The Stroop/Simon paradigm how are primary info
  streams influenced by secondary info streams.
• A main task e.g. pointing in a direction,
  categorising a picture.
• A secondary (competing) task presence of a
  stimulus aspect.

48
EarconsMajorMinor
Auditory IconsCongruentIncongruent
49
Results CombinedWhat and Where in one View
50
Picture word interference

• bear

Semantic Inhibition (Roelofs)
51
Picture word interference II
Kaart
Fonological facilitation
52
Picture Sound categorisation paradigm

• Visual categorisation
• Categorisation similar to working with interface
• pictures (where)
• animals
• non-animals or musical instruments
• additional earcons (what) and auditory icons
  (where)
• trials in blocks
• congruent
• incongruent
• neutral

53
Blocks

• Auditory Icons.
• Congruent
• animal major
• non-animal minor
• Incongruent
• animal minor
• non-animal major
• Neutral
• animal major/minor
• non-animal major/minor

54
Earcons
Major chord Minor chord
55
Results Earcons
56
Auditory Icons

• Examples

57
Results Auditory Icons
58
Results Combined
59
Conclusions

• Different levels of processing
• feature based/examplar level
• animals do that (where)
• rule based/conceptual level
• animals hardly do that (what)
• Different effects of visual categorisation
• facilitation
• inhibition

60
Visuele en auditieve integratie
dier
positief
categorisatie
oordeel
niet-dier
200ms
negatief
interferentie
100ms
Ja/nee
concept
concept
100ms
perceptie
perceptie
plaatje
geluid
61

• The task set perspective
• "Adopting such a set is to select, link and
  configure the elements of a chain of processes
  that will accomplish a task . In everyday life,
  we constantly configure and reconfigure elements
  from our repertoire of cognitive operations and
  representational capacities to perform now one
  task, now another (Rogers and Monsell, 1995".
• Dual task Two configurations Active.
• Mutual interference measures (RT, errors, mental
  load) index joint use of resources and endogenous
  scheduling. General rule Avoid joint use of
  resources (e.g. experienced driving and talking)
• Stroop task interference and facilitation
  measures index relative activation of secondary
  task with respect to primary task the
  possibility to inhibit (perceptual) urges.
  General rule avoid competitive secondary
  information, and especially information that is
  strongly cued with respect to the same response
  channel.

62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
e.g. Car driving
Controlled
training
Automatic
66
Moral
Make automaticity possible
Through carefull design
Avoid controlled processing As much as possible
Limits and Constraints?
67
Cognitive Engineering Intuitive Design, Art,
Books, landscaping, etc. philosophical Plato,
Sloterdijk, Heidegger etc. Politics
etcetcetc. Scientific and problem oriented
engineering of cognition 1. Stimulation. 2.
Engineering stimulatability. Design of
environments Learning,Therapy,Task design, REMD,
VR, etcetc. Direct engineering of
stimulatability drugs,
extra cranial stimulation etc. (reversable) Irreve
rsible Genetic, Surgery ETC. ETC.
68
Endogene Controle Probleem als dezelfde
(perceptuele) situatie een andere interpretatie
moet krijgen Uw UMTS knopjes hebben
verschillende functies, afhankelijk van het doel
v.d. interactie etc. Swappen tussen perceptueel
dezelfde tekstverwerkers met verschillende
instellingen Persistente Switch Kosten, geen
automatisering
69

• Switching between tasks.
• Task Switch research studies the configuration
  and reconfiguration of task sets.
• For every digit string, repeatedly name the
  digits in the string,
• in the next string count them etc.
• 444 66 3333 8 99999 00 111 444 66 3333
  8 99999
• 00 111 444 66 3333 8 99999 00 111
• Repeatedly, for every digit string count the
  characters and name the digits.
• aaaa 7777 fff 9 ggg 44 j 2222 kkkk
  aaaa 7777 fff 9
• ggg 44 j 2222 kkkk 999 iii
• In dual tasks and stroop task no reconfiguration
  of task sets.
• Task switch costs index (we think) the
  deactivation of one task configuration and the
  activation of the other. It is the task set with
  the greatest activation that executes the
  response. Not you.

70
simple rules
The Neural Dynamics of Task Set Switching
between Conditional Visual Motor tasks
71
(No Transcript)
72
Simpele switch taak 1 Taak 1 tel voor alle
stims het aantal karakters. Taak2 noem het type
character van elke stim. Switch taak 1e rijtje
taak 1, 2e rijtje taak 2, derde rijtje taak1,
etc. dd fff gggg k ooo ww zzzz gg
e pppp s gg rr wwww q kkk mm
a hh jjjj oo hh sss ff h kk
ooo bbb yyyy l dd x
73
Simpele switch taak 2 Taak 1 tel voor alle
stims het aantal karakters. Taak2 noem het type
character van elke stim. Switch taak 1e stim
taak 1, 2e stim taak 2, derde stim taak1,
etc. dd fff gggg k ooo ww zzzz gg
4 4444 5 11 44 2222 3 111 mm
a hh jjjj oo hh sss ff 2 33
444 222 3333 2 44 2
74
Simpele switch taak 1 Taak 1 tel voor alle
stims het aantal karakters. Taak2 noem het type
character van elke stim. Switch taak 1e rijtje
taak 1, 2e rijtje taak 2, derde rijtje taak1,
etc. dd fff gggg k ooo ww zzzz gg
e pppp s gg rr wwww q kkk mm
a hh jjjj oo hh sss ff h kk
ooo bbb yyyy l dd x
75
Simpele switch taak 2 Taak 1 tel voor alle
stims het aantal karakters. Taak2 noem het type
character van elke stim. Switch taak 1e stim
taak 1, 2e stim taak 2, derde stim taak1,
etc. dd fff gggg k ooo ww zzzz gg
4 4444 5 11 44 2222 3 111 mm
a hh jjjj oo hh sss ff 2 33
444 222 3333 2 44 2
76
Simpele switch taak 2 Taak 1 tel voor alle
stims het aantal karakters. Taak2 noem het type
character van elke stim. Switch taak 1e stim
taak 1, 2e stim taak 2, derde stim taak1,
etc. dd fff gggg k ooo ww zzzz gg
4 4444 5 11 44 2222 3 111 mm
a hh jjjj oo hh sss ff 2 33
444 222 3333 2 44 2
77
Vorm vs. Kleur
78
(No Transcript)
79
Task I Task II Uniquely cued. Until
stimulus equivalent.
80
Switch kosten op ruwe scores
81
Switch kosten op ruwe scores
82
Na verwijderen within ppn variabiliteit
83
Na verwijderen within trial variabiliteit
84
Na verwijderen lineaire bijdrage switch kosten
85
(No Transcript)
86
A non categorical what task A monkey or a pigeon
could do it Measuring the pressure on a Fitts
Task.
87

• HRV-data not as expected.
• Single task Press button on Screen, be accurate
  and fast.
• Double task Button has a different letters on
  top. Count the occurrences of the different
  letters while pressing the button, be accurate
  and fast.
• Results performance data (RT/ERRORS) as Expected
• Results op HRV not as expected
• Expected single task generates relatively low
  workload and thus a high HRV while the double
  task has a high mental load with low HRV.
• Data single task low variability ( high mental
  load), doubbll task high variability ( low
  mental load)

88

• Neuromotor Noise
• Mental, emotional and physical load generate more
  activity in the nervous system.
• Part of that activity is noise, random
  fluctuations in activity that are superimposed on
  the signal.
• Symptoms
• Noise may and disturb the execution of motor
  tasks.
• Symptoms less accuracy, trembling etc.
• Adaptation
• Filtering of the noise to preserve the signal
  noise ration and to execute the movement within
  parameters. Filtering mechanisms
• Speed up performance.
• Co-contraction of muscles, introducing stiffens
  in the limb and thus more accurate movement.
• Lean on something.

89
De wet van Fitts RT a b ID Waar ID
log2 ((Afstand Breedte)/Breedte)) f
(A/B) Breedte Signal Afstand Noise (Shannon,
1949)
90
A non categorical what task A monkey or a pigeon
could do it Measuring the pressure on an unlikely
Fitts Task.
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94

• HRV-data not as expected.
• Single task Press button on Screen, be accurate
  and fast.
• Double task Button has a different letters on
  top. Count the occurrences of the different
  letters while pressing the button, be accurate
  and fast.
• Results performance data (RT/ERRORS) as Expected
• Results op HRV not as expected
• Expected single task generates relatively low
  workload and thus a high HRV while the double
  task has a high mental load with low HRV.
• Data single task low variability ( high mental
  load), double task high variability ( low mental
  load)

95
(No Transcript)
96

• Neuromotor Noise
• Mental, emotional and physical load generate
  activity in the nervous system.
• Part of that activity is noise, random
  fluctuations in activity that are superimposed on
  the signal.
• Symptoms
• Noise may disturb the execution of motor tasks.
• Symptoms less accuracy, trembling etc.
• Adaptation
• Filtering of the noise to preserve the signal
  noise ration and to execute the movement within
  parameters. Filtering mechanisms
• Speed up performance.
• Co-contraction of muscles, introducing stiffens
  in the limb and thus more accurate movement.
• Lean on something/someone.

97
Squeeze mouse

• In use no difference with normal mouse
• Direct measure
• Unobtrusive
• Enables registration of mental stress during the
  execution of natural tasks.
• Co-contraction may play a role in the aetiology
  of stress related motor problems such as RSI.
• No indirect measure such as number of movement or
  time at the interface.
• The actual pressure on the mouse is measured.

98
Results Grip Force on Squeeze Mouse

• More grip force in single tasks.
• No effect of pacing.
• No interaction effect effect.
• Dual task seems more relaxing.
• Confirms HRV data
• Task speed different for single task and double
  task.
• Post hoc
• Find regression of speed on pressure
• Detail analysis of segments in movement
• , plt .000

99
Project Automated Performance Direct stress
measures and feedback change of
interface. Input MOUS / PEN / TOUCH
Pressure. Speech signal in speech oriented
interaction F0 variability etc.
100
Squeeze mouse

• Real life, ecologically valid
• In use no difference with normal mouse.
• Enables registration of mental stress during the
  execution of natural tasks.
• Co-contraction may play a role in the aetiology
  of stress related motor problems such as RSI.
• Direct measure conceptual valid.
• No indirect measure such as number of movement or
  time at the interface.
• The actual pressure on the mouse is measured.
• Prototype problems
• Hard to calibrate.
• Stable within a session, not over longer periods
  of time.

101
Results Performance Data
Where and What
Where
102
Results HRV

• Lower HRV for single task
• no effect of pacing and interaction
• similar results as previous experiments

103
Results Grip Force on Squeeze Mouse

• More grip force in single tasks.
• No effect of pacing.
• No interaction effect effect.
• Dual task seems more relaxing.
• Confirms HRV data
• Task speed different for single task and double
  task.
• Post hoc
• Find regression of speed on pressure
• Detail analysis of segments in movement
• , plt .000

104
Results Post Hoc Regression analysis

• R (and R²) for double tasks higher
• Equal speed gain costs more force in double
  tasks

____ Single task, self-paced, r²
0.02 ------- Single task, machine-paced r²
0.02 ____ Dual task, self-paced, r²
0.15 _ . _ . Dual task, machine paced, r²
0.13
105
Conclusion

• HRV data and Grip Force data measure the same
  construct with respect to the tasks
• HRV is useful for non motor tasks but is not easy
  to apply
• Grip force as measured by the squeeze mouse seems
  a useful device for safe mousing.
• Where and what types of action (counting and
  moving) can influence one another in a double
  task
• Measurements with Mouse feed back into the neuro
  motor noise theorie modulates the relevance with
  respect to practical applications.

106
(No Transcript)
107
The Role of Neuromotor Stress in Understanding
User Performance on Kiosks for the Retail
Shopping Environments
108
(No Transcript)
109
(No Transcript)
110
Visuele en auditieve integratie
dier
positief
categorisatie
oordeel
niet-dier
200ms
negatief
interferentie
100ms
Ja/nee
concept
concept
100ms
perceptie
perceptie
plaatje
geluid
111

• The task set perspective
• "Adopting such a set is to select, link and
  configure the elements of a chain of processes
  that will accomplish a task . In everyday life,
  we constantly configure and reconfigure elements
  from our repertoire of cognitive operations and
  representational capacities to perform now one
  task, now another (Rogers and Monsell, 1995".
• Dual task Two configurations Active.
• Mutual interference measures (RT, errors, mental
  load) index joint use of resources and endogenous
  scheduling. General rule Avoid joint use of
  resources (e.g. experienced driving and talking)
• Stroop task interference and facilitation
  measures index relative activation of secondary
  task with respect to primary task the
  possibility to inhibit (perceptual) urges.
  General rule avoid competitive secondary
  information, and especially information that is
  strongly cued with respect to the same response
  channel.

112

• Switching between tasks.
• Task Switch research studies the configuration
  and reconfiguration of task sets.
• For every digit string, repeatedly name the
  digits in the string,
• in the next string count them etc.
• 444 66 3333 8 99999 00 111 444 66 3333
  8 99999
• 00 111 444 66 3333 8 99999 00 111
• Repeatedly, for every digit string count the
  characters and name the digits.
• aaaa 7777 fff 9 ggg 44 j 2222 kkkk
  aaaa 7777 fff 9
• ggg 44 j 2222 kkkk 999 iii
• In dual tasks and stroop task no reconfiguration
  of task sets.
• Task switch costs index (we think) the
  deactivation of one task configuration and the
  activation of the other. It is the task set with
  the greatest activation that executes the
  response. Not you.

113

• Trauma Taal (PTST).
• Kun.
• Klinische ?, N.I.C.I - HRF, Fysiology
• Method integration Through Exposure.Compare to
  emdr
• Measuring instrumentsSud scales (Subjective
  measures).Narrative structure (Fragmentation,
  Tone, Cohesion, dissociation etc.)Speech signal
  F0 variabilitye etc.
• Pilot

114
Wolf with sound.
115
F0 somber modulation
116

• Pitch happy mood.

117

• Output van de analyse (Praat)
• Proefpersoon-code x blij
• N 746
• Median 159.388862017289
• Mean 182.432672604596
• Standard deviation 111.00171688755
• Quantile 90 397.418566040441
• Quantile 10 92.519650497537
• delta Quantile
  304.898915542904
• Proefpersoon-code x droevig
• N 887
• Median 104.337808816211
• Mean 135.257805012306
• Standard deviation
  109.78115403165
• Quantile 90 132.530147765804
• Quantile 10 87.3119374482499
• delta Quantile 45.2182103175541

118

• Research.
• The alternating runs task switch paradigm.
• AABBAABBAABBAABB etc.
• Switch Cost
• MEAN (RT(AB)RT(BA)) - Mean(RT (AA)RT(BB)).
• Switch Costs index exogenous and endogenous
  costs.
• If preparation longer than e.g. 1 sec. no further
  switch cost reduction is observed. What remains
  is residual exogenous switch cost.
• switch costs are not observed "if the
  to-be-performed tasks are uniquely cued by
  stimulus type within the experimental context
  (Arbuthnott and Frank, 2000)".
• General rule try to avoid non unique cueing.
  E.g., if switching between editors, situate them
  at distinct locations on your screen.

119
(No Transcript)
120
(No Transcript)
121
(No Transcript)
122
(No Transcript)
123
Near infrared spectroscopy from single channel
towards multi-channel optical topographyWilly
N.J.M. ColierUniversity Medical Center
Nijmegen, The Netherlandsin co-operation with
Marco van der Sluijs, Marco Ferrari, Valentina
Quaresima, Jan Menssen, Jannet Mehagnoul, Bas van
der Kallen, Rüdiger Wenzel, Arno Villringer, Jaap
de Vries, Berend Oeseburg
124
Functional Brain Mapping

• Methods employed
• PET
• (Vascular response)
• fMRI
• (Vascular response)
• EEG and MEG
• (Neuronal response)
• Optical Imaging
• (Vascular response)

• Advantages fNIRS
• High temporal resolution
• Measures deoxyHb and oxyHb
• Bedside applicable
• Disadvantages fNIRS
• Low spatial resolution
• Superficial cortical measurements

125
Neonate
Adult
Optodes
126
NIRS Instrumentation (1)
Analog channels
Light sources
Fiber-optic cables
775
HEAD
845
Detector
904
127
NIRS Instrumentation (2)

• Continuous wave NIRS (CW)
• Stable and reliable
• Commercially available
• Omron, Japan Hamamatsu Photonics, Japan
  Shimadzu, Japan Runman, USA OXYMON,
  Netherlands

128
NIRS Principles (1)

• Optical
• Non-invasive
• Continuous
• Direct feedback
• Regional
• Relatively simple
• Based on Lambert-Beer law

129
NIRS Principles (2)
Lambert-Beer law ?OD ????c L
B OD Optical density ? Absorption
coefficient c Concentration L Physical
pathlength B Factor accounting for scattering
Scattering medium
IN
OUT
Detector
L
130
12 Channel Brain Imaging (2)
The nine optical fibers attached over Broca area
131
Example NIR tracing
Simultaneous NIRS and trans cranial Doppler
registration of a patient during an elective
cardiac arrest. Optodes were placed on the
frontal side of the head, with a distance of 5.5
cm.
132
Dual Channel Applications (1)
Computer screen
Full field , one subject, left occ. cortex
Full field
133
Dual Channel Applications (2)
Computer screen
Hemifield , one subject, leftright cortex
Right hemifield
or
Left hemifield
134
Dual Channel Applications (3)

• Grand average of all
• 6 subjects
• Simultaneous mea- surements
• Left right occ. cortex
• 10 Hz, 4 cm IO

Colier et al. Vision Research. 2000 (in press)
135
Multi-Channel NIRS Optical Topography

• Advantages
• Localize active areas
• (brain or muscle )
• Investigate heterogeneity in oxygenation
• Investigate heterogeneity of flow

Two examples of japanese optical topograhy
prototype systems
136
Comparison fMRI-fNIRS (2)

• - Finger tapping
• - Simultaneous NIRS
• and MRI
• 11 subjects
• Averaged data

137

• fNIRS met name geschikt voor corticaal onderzoek.
• De Pre frontale gebieden.
• De Executive Function
• De zetel van de Willekeurige Sturing, coordinatie
  etc.
• Het inhiberen van where responsen die what gedrag
  bedreigen.
• De zetel van de cultuur, het bedwingen van
  impulsen, het beheersen van emoties etc.
• Bij fysiek trauma Perseverence, Slecht switchen,
  Veel fouten bij dual tasks en stroop task.

138
Wat en Hoe in MMI. Taak controle en
automatisering. De toegankelijkheid van
internet-inhoud 1 Gecontroleerde
handelingen het wat van de interactie
doelgericht, probleemoplossend zoeken van
routes. 2 Automatische handelingen het
waar van de handelingen opportunistisch
navigeren, e.g. a Virtual Mall, Muizen,
etc.
139
Project X "when does a complex task become more
then its components limits and constraints in
task integration Automation From What
(conceptual, slow, thinking) to Where (
perceptual, fast, direct) E.g. driving a car
dissociation
140
On Task Performance (mainly Monsell, 1997) Task
set e.g. driving a car, reading a text. A mental
configuration of less or more simple elements
steering, breaking, clutching, looking, gsming
etc etc. Can be activated and deactivated. You
decide to drive, not to clutch, steer, look etc.
Automatic contention scheduling. Intention is
involved in task setting, not in execution
(monsell, 1997). The strongest activated task set
wins. The intentional task setting may be
overridden The weakness of the flesh
141
Who is in CHARGE SAS (supervisory attentional
system)
modulation
signalling
Contention scheduling ?? Performance e.g.
driving a car
142
On Task performance
Task set
A mental configuration of more simple elements

Can be activated
The strongest activated task set wins
Intention is involved in task setting Not in
execution (monsell, 1997).
Simple tasks vs. Complex tasks.
Single step vs. Multi step.
143
The project X
"when does a complex task become more then its
components limits and constraints in task
integration
Automation
From intentional to habitual
Removal of switch costs
From conceptual to perceptual control
From conscious to unconscious
From endogenous to exogenous
Van denken naar doen
144
Simpel vs. complex tasks. Free vs. forced task
execution. A text fragment is "natural"
order selection text, selection operation?
Can the reverse be taught. Cognitive
constraints on learnability. Is it a 2nd
nature, an automated habit?. Correlations
between multiple stimuli dimensions.
Simultaneous Stroop. In time task switching,
changing association strength. Frontal lobe
development and potential task competence.
145
Endogene Controle Probleem als dezelfde
(perceptuele) situatie een andere interpretatie
moet krijgen Uw UMTS knopjes hebben
verschillende functies, afhankelijk van het doel
v.d. interactie etc. Swappen tussen perceptueel
dezelfde tekstverwerkers met verschillende
instellingen Persistente Switch Kosten, geen
automatisering
146
Endogenous scheduling
What
Passive
Individual
social
Active
Exogenous scheduling
Where
Ungerleider Mishkin, 1982 Kosslyn, 1994
Neisser, 1992. Twee cognitieve
systemen.
147
Ungerleider Mishkin, 1982 Kosslyn, 1994
Neisser, 1992 Two cognitive
systems
148
Task dimensions Automation
Endogenous scheduling
What
Top down
The What Bottleneck
E.g. Stroop Autom. reading.
Where/how
Bottom up
Exogenous scheduling
149
Endogenous scheduling
What
Passive
Individual
social
Active
Exogenous scheduling
Where
How do we get from where to what (and back) - A
warped where e.g. Cultural Warping of perceptual
space from sound to phonemes the scientist
in the crib (Gopnik et. al). - Naming of
perceptual categories 1st use of symbols if
I say /aap/ I mean the thing over there. -
Creating a what for the where rule based
institutions, a woman is not someone with
certain perceptual features but somebody that
falls into a rule based category, with rights and
obligations. And the actual woman is of
course both. That is the Stroop task for the
man (or woman).
150
The Frontal Task Dimension
What
Top down
Endogenous scheduling
passive
individual
social
active
Exogenous scheduling
Where/how
Bottom up
151
The Frontal Task Dimension
Endogenous scheduling
What
Top down
Passive
Individual
social
Active
Exogenous scheduling
Where/how
Bottom up
152
Endogenous scheduling
What
Passive
Individual
social
Active
Exogenous scheduling
Where
How do we get from where to what - A warped
where e.g. Cultural Warping of perceptual space
from sound to phonemes the scientist in the
crib. - Naming of perceptual categories 1st
use of symbols if I say /aap/ I mean the
thing over there. - Creating a what for the
where rule based institutions, a woman is
not someone with certain perceptual features but
somebody that falls into a rule based
category, with rights and obligations. And
the actual woman is of course both. That is the
Stroop task for the man (or woman).
153
Endogenous scheduling
What
Passive
Individual
social
Active
Exogenous scheduling
Where
The growth of social institutional competence or
The emergence of social competence or The
enforcement of social competence
154
Endogenous scheduling
What
Passive
Individual
social
Active
Exogenous scheduling
Where
Taming / socialising or Inactivation
155
Endogenous scheduling
What
Passive
interpassive
Individual
social
Active
Exogenous scheduling
Where
TV / Plays etc. Inter-passivity See the
experiencing others.
156
Task dimensions Automation (learning)
The What Bottleneck Controlled navigation
Automatic Navigation Acquired 2nd Nature
157
Schrijfspoor Origineel (MvH)
158
Pendruk origineel (MvH)
159
Spanning in MMI

• Worden wij Borgs?

Nieuwe vormen van Mens-Machine Interactie Louis
Vuurpijl http//hwr.nici.kun.nl/vuurpijl
160
Kunstmatige Inteligentie
Een 5-jarige opleiding aan de KUN

• Cognitiewetenschap
• Psychologie
• Informatica

161
Cognitiewetenschap en
Psychologie
De mens
162
Cognitiewetenschap en
Informatica
De mens
De machine
163
Kunstmatige Intelligentie
De mens in de machine
De machine in de mens?
De machine
De mens
164
Human-Computer Interaction
Hoe kun je als mens met de computer communiceren?
Welke informatie kan er worden overgedragen?
Hoe kun je dit optimaliseren?
165
De Homunculus
166
Human-Computer Interaction
Hoe interacteer je met de computer?
Eerst een aantal bekende modaliteiten
167
Human-Computer Interaction
De mens
De machine
168
Human-Computer Interaction
De mens
De machine
169
Human-Computer Interaction
geluid
De mens
De machine
170
Human-Computer Interaction
geluid
Gevoel?
VR
De mens
De machine
171
Human-Computer Interaction
gaze
spraak
toetsenbord
Facial expressions
handschrift
Body Movements
172
The virtual workbench
173
Augmented Reality
174
Situational Awareness
175
Van GSM naar handhelds
176
Wireless Fingering
177
Steve Mann from MITstarring asCyberman
178
Is dit alles..
Of worden wij Borgs?
179
Cortical electrode-arrays (Normann, Utah 1992)
Electrodes
Neuron on silicium (Fromherz Stett, 1995)
180
Wordt het al spannender?
181
Doven kunnen weer horen
Meer dan 20000 cochleaire implantaten sinds 1979
Spierverlamming oplosbaar?
182
Blinden kunnen weer zien!
183
De geheugen-chip
184
Trends for the future
185
Ubiquitous Computing
Mark Weiser
186
Our current home environment
187
Our home tomorrow
188
Back to HCI right nowHandwriting recognition
Dit mag je zo direct zelf proberen!
189
Back to HCI right now Image Retrieval
Ik zoek plaatjes
Vraag
met daarop een hond
Dit mag je zo direct zelf proberen!
190
Vind je dit leuk?
Wij doen onderzoek aan mens en machine met het
oog op inteligente vormen van mens-machine
interactie..
http//www.cogsci.kun.nl
191
Groep1Handschriftherkenner dScript
192
Groep 2Image retrieval