Title: Lecture Overview
1Lecture Overview
- Regier System Limitations
- Image Schemas Recap
- Force Dynamic Schemas Recap
- Sensory-Motor Schemas
- Evidence in Primates
- Evidence in Humans
- Do motor schemas play a role in language?
- A Computational Model of Motor Schemas
- Learning Hand Action terms (Bailey)
- Cultural Schemas and frames
2Limitations
- Scale
- Uniqueness/Plausibility
- Grammar
- Abstract Concepts
- Inference
- Representation
3Force Dynamics, modals and causatives
- A gust of wind made the pages of my book turn.
- The appearance of the headmaster made the pupils
calm down. - The breaking of the dam let the water flow from
the storage lake. - The abating of the wind let the sailboat slow
down.
4Talmys force dynamic schemas
- Two entities in a forceful interaction
- Agonist (foregrounded)
- Antagonist (has an effect on the Agonist)
- Each entity exerts a force on the other
- An entity has an tendency
- towards rest (inaction)
- toward motion
- Opposed forces have different relative strength
- Depending on the relative strength the result
of the interaction could make the Agonist move
or be at rest.
5Schematic Representation(Talmy)
6FD Patterns
- A gust of wind made the pages of my book turn.
- The appearance of the headmaster made the pupils
calm down. - The breaking of the dam let the water flow from
the storage lake. - The abating of the wind let the sailboat slow
down.
7Semantic field Force-dynamics representation
Physical The ball kept rolling along the green
Physical/psychological John can't go out of the house
Intra-psychological He refrained from closing the door
Intra-psychological (lexicalized) She's civil to himÂ
Socio-psychological She gets to go to the park
8An Experiment with Force Dynamics
9Language Specific Bias
10Closed Class vs. Open Class terms
- Image Schematic and Force Dynamic Patterns are
expressed by closed class terms in language - Prepositions (in, on, into, out)
- Modals and causatives (make, let, might, prevent)
- How about open class terms?
- Verbs and Event descriptions
- Motor Schemas - Embodied
- Is there evidence for motor schemas and if so are
they used in language? - Frames Composed from Image and motor schemas
-Cultural
11Coordination
- PATTERN GENERATORS, separate neural networks that
control each limb, can interact in different ways
to produce various gaits. - In ambling (top) the animal must move the fore
and hind leg of one flank in parallel. - Trotting (middle) requires movement of diagonal
limbs (front right and back left, or front left
and back right) in unison. - Galloping (bottom) involves the forelegs, and
then the hind legs, acting together
12Sensory-Motor Schemas
- A sensory (perceptual) schema determines whether
a given situation is present in the environment.
- Object Detection
- Spatial relation recognition
- Execution of current plans is made up of motor
schemas which are akin to control systems but
distinguished by the fact that they can be
combined to form coordinated control programs - Sensory and Motor Schemas are closely coupled
circuits sensory-motor schemas.
13The neural theory
- Human concepts are embodied. Many concepts make
direct use of the sensory-motor capacities of our
body-brain system. - Many of these capacities are also present in
non-human primates. - Let us look at concepts that make use of our
sensory-motor capacities, ex. Grasp.
14Area F5
General Purpose Neurons General Grasping General
Holding General Manipulating
15General Purpose Neurons in Area F5
A Grasping with the mouth B Grasping with the
cl. hand C Grasping with the ipsil. hand
(Rizzolatti et al. 1988)
16General Purpose Neurons Achieve Partial
Universality Their firing correlates with a
goal-oriented action of a general type,
regardless of effector or manner.
17Area F5c
Convexity region of F5 Mirror neurons
18F5c-PF
Rizzolatti et al. 1998
19F5 Mirror Neurons
tools
Gallese and Goldman, TICS 1998
20Strictly congruent mirror neurons (30)
Observed Action
Executed Action
Executed Action
(Rizzolatti et al. Cog Brain Res 1996)
21Category Loosening in Mirror Neurons (60)
(Gallese et al. Brain 1996)
A C is Observe (Execute) Precision Grip
(Prototype) B D is Observe (Execute) Whole
Hand Pre-hension
22The F5c-PF circuit Links premotor area F5c and
parietal area PF (or 7b). Contains mirror
neurons. Mirror neurons discharge when Subject
(a monkey) performs various types of goal-related
hand actions and when Subject observes another
individual performing similar kinds of actions
23Phases Area F5 contains clusters of neurons that
control distinct phases of grasping opening
fingers, closing fingers. Jeannerod, et al.,
1995 Rizzolatti, et al., 2001.
24 Mirror Neurons Achieve Partial
Universality, since they code an action
regardless of agent, patient, modality
(action/observation/hearing), manner,
location. Partial Role Structure, since they
code an agent role and a purpose role. The
Agent Role In acting, the Subject is an agent
of that action. In observing, the Subject
identifies the agent of the action as having the
same role as he has when he is acting namely,
the agent role. The Purpose Role Mirror neurons
fire only for purposeful actions.
25Mirror Neurons Achieve Category tightening and
loosening
26The F4-VIP circuit
27The F4-VIP Circuit Links premotor area F4 and
parietal area VIP. Transforms the spatial
position of objects in peri-personal space into
motor programs for interacting with those
objects. Examples Reaching for the objects, or
moving away from them with various parts of your
body such as the arm or head.
28Area F4
Arm reaching Head turning
29Somato-Centered Bimodal RFs in area F4
(Fogassi et al. 1996)
30Somato-Centered Bimodal RFs in area VIP
(Colby and Goldberg 1999)
31Somato-Centered Bimodal RFs in area F4
(Fogassi et al. J Neurophysiol 1996)
32AIP and F5 (Grasping) in Monkey
AIP - grasp affordances in parietal cortex Hideo
Sakata
F5 - grasp commands in premotor cortex Giacomo
Rizzolatti
33Size Specificity in a Single AIP Cell
- This cell is selective toward small objects,
somewhat independent of object type ( Hideo
Sakata) - Note Some cells show size specificity others
do not.
34Summary of Fronto-Parietal Circuits
- Motor-Premotor/Parietal Circuits
- PMv (F5ab) AIP Circuit
- grasp neurons fire in relation to movements
of hand prehension necessary to grasp object - F4 (PMC) (behind arcuate) VIP Circuit
- transforming peri-personal space coordinates so
can move toward objects - PMv (F5c) PF Circuit F5c
- different mirror circuits for grasping, placing
or manipulating object - Together suggest cognitive representation of the
grasp, active in action imitation and action
recognition
35- MULTI-MODAL INTEGRATION
- The premotor and parietal areas, rather than
having - separate and independent functions, are neurally
integrated - not only to control action, but also to serve the
function of - constructing an integrated representation of
- Actions, together with
- objects acted on, and
- locations toward which actions are directed.
- In these circuits sensory inputs are transformed
in order to - accomplish not only motor but also cognitive
tasks, such as - space perception and action understanding.
36Modeling Motor Schemas
- Relevant requirements (Stromberg, Latash, Kandel,
Arbib, Jeannerod, Rizzolatti) - Should model coordinated, distributed,
parameterized control programs required for motor
action and perception. - Should be an active structure.
- Should be able to model concurrent actions and
interrupts. - Should model hierarchical control (higher level
motor centers to muscle extensor/flexors. - Computational model called x-schemas
(http//www.icsi.berkeley.edu/NTL)