Embodiment

1
Embodiment Compositionality

• Two fundamental, but unreconciled, aspects of
  human language embodiment and compositionality
• Embodiment the realization that scientific
  understanding of mind and language entails
  detailed modeling of the human brain
• Compositionality - only people can express and
  understand an essentially unbounded range of
  messages

2
Compositionality

• Anything that deserves to be called a language
  must contain meaningful expressions built up from
  other meaningful expressions 4
• strong compositionality states that the meaning
  of a expression is totally determined by its form
  and is independent of context
• In its strict version, this claim is clearly
  wrong 5.

3
Truth-conditional Semantics

• The existence of these forms of
  context-dependence would thus appear to spell the
  doom for truth-conditional semantics, understood
  as the project of assigning properties to
  syntactic items (partly by finding principles for
  determining properties of complex syntactic items
  on the basis of their parts) such that we await
  only the specification of something like an index
  to know what the truth-conditions of a given
  utterance of a given assertive sentence would
  be.- Jason Bridges
• Wittgensteins Philosophical Investigations A
  Critical Guide, A. M. Ahmed (ed.), Cambridge
  University Press, 2009.

4
Simulation Semantics

• Lectures 1 and 3 gt embodied
• But what about public language, text?
• Skeletal shared meaning gt schemas
• Language Community (LC)
• Shares grammar, schemas, analysis rules

5
Compositionality

• Put the parts together to create the meaning of
  the whole.
• Questions
• what is the nature of the parts?
• How and why do they combine with one another?
• What meaning is associated with this composition?

6
Short NTL answers

• Parts constructions, schemas
• Combination binding, unification
• Meaning of the whole
• Public (LC) Semantic Specification SemSpec
• Private Enactment/Simulation of SemSpec

7
ECG Lattices

• Schemas Image, X-schemas, Frames
• Constructions Lexical, Grammatical
• Language Communities
• Situations Mental Spaces
• Maps Metaphor, etc.

8
Lectures I. Overview 2. Simulation
Semantics 3. ECG and Best-fit Analysis 4.
Compositionality 5. Simulation, Counterfactuals,
and Inference
Constructions
Utterance
incremental, competition-based, psychologically
plausibleA
Simulation
Semantic Specification image schemas, bindings,
action schemas
9
Conceptual Structure

• Embodied
• Schematic
• (Partially) language-independent
• Conceptual Composition
• Highly interconnected
• Primitives
• Conceptual composition
• Metaphor

10
Some Conceptual Primitives
Grammaticalized concepts person, gender, age,
agent, speaker possession, mass/count,
reflexives, Primal scenes/ event types
transitive, tense, aspect   General
Logic connectives, numbers similarity, inference,
uncertainty part/ whole, scales,
magnitude binding, variables, indefinites,
generalization   Mental Operations Learning,
matching, binding mental spaces,
mappings simulation, displacement
Image Schemas parameters of spatial
cognition action schemas - controller goals,
force-dynamics (causation) parameters of parts
boundaries Time Social World young/ mature/
old authority, approval, help value , exchange,
obligation theory of mind, perception and
intention,   Communication speaker/ hearer,
direct/ indirect true/ false question, command,
etc.                  
11
Conceptual Composition Ellen Dodge Thesis 1.
Conjunction horse with stripes 2. Modification
camel without hump 3. Abstraction - vehicle 4.
Mapping a. Metonymy London thinks b.
Metaphor causes are forces 5. Blending apple
bus 6. Relational horses eat grass
12
First example

• He bit the apple.

13
Schemas
schema MotorControl subcase of Process
roles Actor ? Protagonist Effector
Effort Routine constraints Actor ?
animate
14
schema Contact subcase of SpatialRelation
roles Entity1 entity Entity2
entity
schema MotorControl subcase of Process
roles Actor ? Protagonist Effector
Effort Routine constraints Actor ?
animate
schema ForceTransfer evokes Conact as C
roles Supplier ? C.entity1 Recipient ?
C.entity2 Force
schema ForceApplication subcase of MotorControl
evokes ForceTransfer as FT roles
Actor ? FT.Supplier ? Protagonist Acted
Upon? FT.Recipient Effector Routine
Effort ? FT.Force.amount
15
Schema Lattice
 
Contact
MotorControl
ForceTransfer
Motion
Effector Motion
SelfMotion
ForceApplication
CauseEffect
MotionPath
Effector MotionPath
SelfMotion Path
SPG
SpatiallyDirectedAction
Agentive Impact
Contact
16
Verb Constructions
Construction BITE1 subcase of Verb form
bite meaning ForceApplication
constraints Effector ? teeth
Routine ? bite // close mouth  
schema ForceApplication subcase of MotorControl
evokes ForceTransfer as FT roles
Actor ? FT.Supplier ? Protagonist Acted
Upon ? FT.Recipient Effector
Routine Effort ? FT.Force.amount
17
Verb Constructions
cxn BITE meaning ForceApplication
schema MotorControl
cxn GRASP meaning ForceApplication
schema ForceApplication subcase of
MotorControl
cxn PUSH meaning ForceApplication
cxn SLAP meaning AgentiveImpact
schema Agentive Impact subcase of
ForceApplication
cxn KICK meaning AgentiveImpact
cxn HIT meaning AgentiveImpact
18
Argument Structure Construction
construction ActiveTransitiveAction2 subcase
of VP constituents V verb NP NP
form constraints VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED ForceApplication as FA
constraints Selfm ? ED.EventType
Vm ? ED.ProfiledProcess Causer ?
ED.ProfiledParticipant FA ? Vm
Causer ? FA.Actor Affected ? FA.ActedUpon
Affected ? NPm

19
Argument Structure Construction

construction ActiveTransitiveAction2 subcase
of VP constituents V verb NP NP
form constraints VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED ForceApplication as FA
constraints Selfm ? ED.EventType
Vm ? ED.ProfiledProcess Causer ?
ED.ProfiledParticipant FA ? Vm
Causer ? FA.Actor Affected ? FA.ActedUpon
Affected ? NPm


20
CauseEffect schema
schema CauseEffect subcase of
ForceApplication Process roles Causer ?
Actor Affected ? ActedUpon ?
Process.Protagonist Instrument ? Effector
21
Schema Lattice
Contact
MotorControl
ForceTransfer
Process
Motion
Effector Motion
SelfMotion
ForceApplication
CauseEffect
MotionPath
Effector MotionPath
SelfMotion Path
SPG
SpatiallyDirectedAction
Agentive Impact
Contact
22
Argument Structure Construction

construction ActiveTransitiveAction2 subcase
of VP constituents V verb NP NP
form constraints VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED ForceApplication as FA
constraints Selfm ? ED.EventType
Vm ? ED.ProfiledProcess Causer ?
ED.ProfiledParticipant FA ? Vm
Causer ? FA.Actor Affected ? FA.ActedUpon
Affected ? NPm


23
Important points

• Compositionality does not require that each
  component contain different information.
• Shared semantic structure is not viewed as an
  undesirable redundancy

24
Argument Structure Construction

construction ActiveTransitiveAction2 subcase
of VP constituents V verb NP NP
form constraints VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED ForceApplication as FA
constraints Selfm ? ED.EventType
Vm ? ED.ProfiledProcess Causer ?
ED.ProfiledParticipant FA ? Vm
Causer ? FA.Actor Affected ? FA.ActedUpon
Affected ? NPm


25
Event Descriptor schema
  schema EventDescriptor roles
EventType Process ProfiledProcess
Process ProfiledParticipant Entity
ProfiledState(s) State SpatialSetting
TemporalSetting    
 
26
Preconditions, resources, fine control structure
are important aspects of events
27
Argument Structure Construction
Construction ActiveTransitiveAction2 subcase
of VP constituents V verb NP NP
form constraints VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED ForceApplication as FA
constraints Selfm ? ED.EventType
Vm ? ED.ProfiledProcess Causer ?
ED.ProfiledParticipant FA ? Vm
Causer ? FA.Actor Affected ? FA.ActedUpon
Affected ? NPm


28
Bindings with other cxns
construction NPVP1 constituents Subj NP
VP VP form Constraints Subj f
before VPf meaning EventDescriptor
ProfiledParticipant ? Subjm    
construction ActiveTransitiveAction2 subcase
of VP constituents V NP form VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED constraints
Selfm ? ED.EventType Vm ?
ED.ProfiledProcess Causer ?
ED.ProfiledParticipant Affected ? NPm


29
Bindings with other cxns
Construction NPVP1 constituents Subj NP
VP VP form constraints Subj f
before VPf meaning EventDescriptor
ProfiledParticipant ? Subjm    
construction ActiveTransitiveAction2 subcase
of VP constituents V NP form VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED constraints
Selfm ? ED.EventType Vm ?
ED.ProfiledProcess Causer ?
ED.ProfiledParticipant Affected ? NPm


schema EventDescriptor roles
EventType ProfiledProcess
ProfiledParticipant ProfiledState(s)
SpatialSetting TemporalSetting
30
Bindings with other cxns
construction NPVP1 constituents Subj NP
VP VP form Constraints Subj f
before VPf meaning EventDescriptor
ProfiledParticipant ? Subjm    
construction ActiveTransitiveAction2 subcase
of VP constituents V NP form VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED constraints
Selfm ? ED.EventType Vm ?
ED.ProfiledProcess Causer ?
ED.ProfiledParticipant Affected ? NPm


schema EventDescriptor roles
EventType ProfiledProcess
ProfiledParticipant ProfiledState(s)
SpatialSetting TemporalSetting
31
Unification
Meaning
Constructions
EventDescriptor EventType ProfiledProcess
ProfiledParticipant
NPVP1
CauseEffect causer affected
TransitiveAction2
ForceApplication actor actedupon
BITE
NP2
ReferentDescriptor
THE
APPLE
NP1
ReferentDescriptor
HE
32
Unification
Meaning
Constructions
EventDescriptor EventType ProfiledProcess
ProfiledParticipant
NPVP1
CauseEffect causer affected
TransitiveAction2
ForceApplication actor actedupon
BITE
NP2
ReferentDescriptor
THE
APPLE
NP1
ReferentDescriptor resolved referent
HE
33
Unification
Meaning
Constructions
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
NPVP1
CauseEffect causer affected
TransitiveAction2 Verb
ForceApplication actor actedupon
BITE
NP2
ReferentDescriptor
THE
APPLE
NP1
ReferentDescriptor resolved referent
HE
34
Unification
Meaning
Constructions
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
NPVP1 subj
CauseEffect causer affected
TransitiveAction2
ForceApplication actor actedupon
BITE
NP2
ReferentDescriptor
THE
APPLE
NP1
ReferentDescriptor
HE
35
Unification
Meaning
Constructions
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
NPVP1
CauseEffect causer affected
TransitiveAction2 NP
ForceApplication actor actedupon
BITE
NP2
ReferentDescriptor
THE
APPLE
NP1
ReferentDescriptor
HE
36
Semantic SpecificationHe bit the apple
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
CauseEffect causer affected
ForceApplication actor actedupon routine ?
bite effector ? teeth
RD27 category
Person
RD55 category
Apple
37
Argument Structure ConstructionHe was bitten (by
a toddler)

construction PassiveTransitiveAction2 subcase
of VP constituents V PassiveVerb
(PP agentivePP) form constraints
VF before PPF meaning
CauseEffectAction evokes EventDescriptor as
ED ForceApplication as FA constraints
Selfm ? ED.EventType Vm ?
ED.ProfiledProcess Affected ?
ED.ProfiledParticipant FA ? Vm
Causer ? FA.Actor Affected ? FA.ActedUpon
Causer ? PP.NPm


38
Semantic SpecificationHe was bitten (by a
toddler)
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
CauseEffect causer affected
ForceApplication actor actedupon routine ?
bite effector ? teeth
RD27 category
Person
RD48 category
Person
39
Variations on a theme

• He shattered the window
• The window was shattered
• The window shattered

40
Verb Construction -- shatter
Construction SHATTER1 subcase of Verb
form shatter meaning StateChange
constraints Initial Undergoer.state
? whole Final Undergoer.state ?
shards  
schema StateChange subcase of Process
roles Undergoer ? Protagonist
41
Argument Structure ConstructionHe shattered the
window

construction ActiveTransitiveAction3 subcase
of VP constituents V verb NP NP
form constraints VF
before NPF meaning CauseEffect evokes
EventDescriptor as ED StateChange as SC
constraints Selfm ? ED.EventType
Vm ? ED.ProfiledProcess Causer ?
ED.ProfiledParticipant SC ? Vm
Affected ? SC.Undergoer Affected ? NPm


42
Semantic SpecificationHe shattered the window
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
CauseEffect causer affected
StateChange Undergoer state ? wholeness
RD27 category
Person
RD189 category
Window
43
Argument Structure ConstructionThe window was
shattered

construction PassiveTransitiveAction3 subcase
of VP constituents V PassiveVerb
(PP agentivePP) form constraints
VF before NPF meaning CauseEffect
evokes EventDescriptor as ED StateChange as
SC constraints Selfm ?
ED.EventType Vm ? ED.ProfiledProcess
Affected ? ED.ProfiledParticipant
SC ? Vm Affected ? SC.Undergoer
Causer ? PP.NPm


44
Semantic SpecificationThe window was shattered
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
CauseEffect causer affected
StateChange Undergoer state ? wholeness
RD175 category
Window
45
Argument Structure ConstructionThe window
shattered

construction ActiveIntransitiveAction1
subcase of VP constituents V verb
form meaning Process evokes
EventDescriptor as ED StateChange as SC
constraints Selfm ? ED.EventType
Vm ? ED.ProfiledProcess
Protagonist ? ED.ProfiledParticipant SC ?
Vm Protagonist ? SC.Undergoer


46
Semantic SpecificationThe window shattered
EventDescriptor eventtype ProfiledProcess
ProfiledParticipant
Process protagonist
StateChange Undergoer state ? wholeness
RD177 category
Window
47
Summary

• Small set of constructions and schemas
• Composed in different ways
• Unification produces specification of simulation
• Sentence understanding is simulation
• Different meanings gt different simulations

48
NTL Compositionality

• Language understanding is simulation
• Simulation activates conceptual structures
• Conceptual Compositionality is basic
• Grammatical Compositionality is inherently
  constructional, not surface
• SemSpec Semantic Specification
• Skeletal Meaning
• Captures shared understanding of an LC
• Site of Compositionality

49
(No Transcript)
50
Argument Structure ConstructionHis white teeth
bit into the apple

construction ActiveEffectorMotionPath3
subcase of VP constituents
V verb PP Spatial-PP
form constraints VF before PPF meaning
EffectorMotionPath evokes EventDescriptor
as ED ForceApplication as FA
constraints Selfm ? ED.EventType Vm ?
ED.ProfiledProcess Effector ?
ED.ProfiledParticipant FA ? Vm Actor ?
FA.Actor // INI Effector ? FA.Effector Target
? FA.ActedUpon SPG ? PPm Target ? PPm
.Prep.LM  
51
Simulation His white teeth bit into the apple
Action
Protagonist Actor
Effector Motion
Protagonist Effector
Source
Path
Goal
52
Non-agentive biting

• He landed on his feet, hitting the narrow
  pavement outside the yard with such jarring
  impact that his teeth bit into the edge of his
  tongue. BNC
• The studs bit into Trent's hand. BNC
• His chest burned savagely as the ropes bit into
  his skin. BNC

53
Schema Network
Contact
MotorControl
ForceTransfer
Process
Motion
Effector Motion
SelfMotion
ForceApplication
CauseEffect
MotionPath
Effector MotionPath
SelfMotion Path
SPG
SpatiallyDirectedAction
Agentive Impact
Contact
54
Simulation His teeth bit his tongue
Motion
Protagonist Mover
Source
Path
Goal
55
Key assumptions of NTL

• Language understanding is simulation
• Simulation involves activation of neural
  structures

56
Constructions

• Construction Grammar
• Constructions are form-meaning pairings
• A given utterance instantiates many different
  constructions
• Embodied Construction Grammar
• Construction meaning is represented using schemas
• Meaning is embodied

57
Lectures I. Overview 2. Simulation
Semantics 3. ECG and Best-fit Analysis 4.
Compositionality 5. Simulation, Counterfactuals,
and Inference
Constructions
Utterance
incremental, competition-based, psychologically
plausible
Simulation
Semantic Specification image schemas, bindings,
action schemas
58
Constructions
Utterance
incremental, competition-based, psychologically
plausible
Simulation
Semantic Specification image schemas, bindings,
action schemas
59
Some Conceptual Primitives
Grammaticalized concepts person, gender, age,
agent, speaker possession, mass/count,
reflexives, Primal scenes/ event types
transitive, tense, aspect   General
Logic connectives, numbers similarity, inference,
uncertainty part/ whole, scales,
magnitude binding, variables, indefinites,
generalization   Mental Operations Learning,
matching, binding mental spaces,
mappings simulation, displacement
Image Schemas parameters of spatial
cognition action schemas - controller goals,
force-dynamics (causation) parameters of parts
boundaries Time Social World young/ mature/
old authority, approval, help value , exchange,
obligation theory of mind, perception and
intention,   Communication speaker/ hearer,
direct/ indirect true/ false question, command,
etc.                  
60
Conceptual Composition 1. Conjunction horse
with stripes 2. Modification camel without
hump 3. Abstraction - vehicle 4. Mapping a.
Metonymy London thinks b. Metaphor causes
are forces 5. Blending trash can basketball
61
Putting the parts together

• Bindings
• Unification

62
Pre-existing structure
schema
Cxn
schema
schema
Cxn
schema
Cxn
schema
63
Unification
schema
Cxn
schema
schema
Cxn
schema
Cxn
schema
64
Simulation - He bit the apple
CauseEffect
Protagonist Causer ? Actor
ForceApplication
Process
Protagonist Affected ? ActedUpon
65
Simulation - He bit the apple
CauseEffect
Protagonist Causer ? Actor
ForceApplication
Process
Protagonist Affected ? ActedUpon
66
Schema Lattice
Contact
MotorControl
ForceTransfer
Process
Motion
Effector Motion
SelfMotion
ForceApplication
CauseEffect
MotionPath
Effector MotionPath
SelfMotion Path
SPG
SpatiallyDirectedAction
Agentive Impact
Contact
67
Prototypes and extensions?

• CauseMotion Path
• He threw the ball across the room
• He kicked the ball over the table
• He sneezed the napkin off the table
• He coughed the water out of his lungs

68
Key points

• In prototypical verb-argument structure
  construction combinations, verb meaning is very
  similar to argument structure meaning.
• Verbs whose meaning partially overlaps that of a
  given argument structure constructions may also
  co-occur with that argument structure
  construction
• These less prototypical combinations may motivate
  extensions to the central argument structure
  constructions

69
Some more variations on a theme

• He bit the apple
• He bit into the apple
• His white teeth bit into the apple.

70
Argument Structure ConstructionHe bit into the
apple

construction ActiveEffectorMotionPath2
subcase of VP constituents
V verb PP Spatial-PP
form constraints VF before PPF meaning
EffectorMotionPath evokes EventDescriptor
as ED ForceApplication as FA
constraints Selfm ? ED.EventType Vm ?
ED.ProfiledProcess Actor ? ED.ProfiledParticipa
nt FA ? Vm Actor ? FA.Actor Effector ?
FA.Effector // INI Target ? FA.ActedUpon SPG
? PPm Target ? PPm .Prep.LM  
71
Schema
schema EffectorMotionPath subcase of
EffectorMotion subcase of SPG // or evokes SPG
roles Actor ? MotorControl.protagoni
st Effector ? SPG.Tr ? M.Mover ?
Motion.protagonist Target ? SPG.Lm

72
Schema Lattice
Contact
MotorControl
ForceTransfer
Process
Motion
Effector Motion
SelfMotion
ForceApplication
CauseEffect
MotionPath
Effector MotionPath
SelfMotion Path
SPG
SpatiallyDirectedAction
Agentive Impact
Contact
73
Argument Structure Construction He bit into the
apple

construction ActiveEffectorMotionPath2
subcase of VP constituents
V verb PP Spatial-PP
form constraints VF before PPF meaning
EffectorMotionPath evokes EventDescriptor
as ED ForceApplication as FA
constraints Selfm ? ED.EventType Vm ?
ED.ProfiledProcess Actor ? ED.ProfiledParticipa
nt FA ? Vm Actor ? FA.Actor Effector ?
FA.Effector // INI Target ? FA.ActedUpon SPG
? PPm Target ? PPm .Prep.LM  
74
EffectorMotionPath
Action
Protagonist Actor
Effector Motion
Protagonist Effector
Source
Path
Goal
75

• The subject NP (She) is bound to these roles
• profiledParticipant of EventDescriptor
• causer of CauseMotionAction
• actor of ForceApplication
• supplier of ForceTransfer
• actor of ForcefulMotionAction
• actor of EffectorMotionAlongAPath
• protagonist of CauseMotionAction and
  ForceApplication
• protagonist of ForcefulMotionAction and
  EffectorMotionAlongAPath

76

• The direct object NP (his hand) is bound to
• affected of CauseMotionAction
• protagonist2 of CauseMotionAction
• actedUpon of ForceApplication
• recipient of ForceTransfer
• mover of MotionAlongAPath
• trajector of SPG
• actedUpon lt -- gt Target of ForcefulMotionAction
• target of EffectorMotionAlongAPath
• (landmark of a different SPG schema)

77

• construction CauseMotion4 subcase of
  CauseMotion1
• constructional constituents
• v Verb
• np NP
• pp Path-PP
• form constraints
• v.f before np.f
• np.f before pp.f
• meaning CauseMotionAction
• evokes EventDescriptor as ed
• evokes ForcefulMotionAction as fma
• constraints
• ignore self.m lt--gt v.m
• self.m.process1 lt--gt v.m.process1
  
• v.m lt--gt fma
• self.mlt--gt ed.eventType
• v.mlt--gt ed.profiledPorcess
• self.m.affected lt--gt np.m
• self.m.causer lt--gt
  ed.profiledParticipant