Language

1
Language

• Sci271 Cognitive Neuroscience 2
• Frank Wijnen, 15-11-2005

2
Language outline

• Mental lexicon
• Sentence processing
• structure and meaning
• Language production
• meaning/grammar before form?
• The neural substrate
• aphasias
• imaging evidence

3
The language faculty (Jackendoff)
phonological rules
syntactic rules
semantic/con-ceptual rules
phonological structures
syntactic structures
sem/concept structures
speech
thought
4
1. Mental lexicon
5
mental lexicon
form
meaning
grammar
6
mental lexicon
form
meaning
grammar
lemma
lexeme
7
mental lexicon
concept
speech
form
meaning
grammar
lemma
lexeme
8
mental lexicon
concept
speech
form
meaning
grammar
lemma
lexeme
9
Lemmas

• e.g. He gives her flowers to his mother.
• thematic features
• arguments bijv. give (giver, given, receiver)
• syntactic features
• give is a verb with three associated syntactic
  functions subject, direct object indirect
  object

10
Meaning semantic networks
11
Lexemes

• information about the morfological and
  phonological properties of a word
• number and type of constituting morphemes
• number of syllables
• stress
• phonemes

12
Malapropisms

• Coined for Mrs. Malaprop, a character in the play
  The Rivals by R. Sheridan (1775).A malapropism
  is an accidentally inappropriate (usually that
  means 'humorous') alteration of words.
• Technically a form-based word substitution
• Suggestive of an organization of the lexicon in
  terms of phonological neighborhoods

13
Malapropisms

• Abraham Lincoln became America's greatest
  Precedent.
• Arabs wear turbines on their heads.
• Having one wife is called monotony.
• The Pyramids are a range of mountains between
  France and Spain.

• She's as headstrong as an allegory on the banks
  of the Nile
• Let me refurbish your memories
• Let me synthesize the discussion.

14
The divide between meaning and form
tip-of-the-tongue

• You are perfectly aware that you want to say word
  or name you know
• but its somehow impossible to pronounce that
  word.
• you may be able to give a rough indication of its
  form the initial phoneme, the number of
  syllables, the rhythm.? Tip of the tongue!

15
Tip of the tongue
Suppose we try to recall a forgotten name. The
state of our Consciousness is peculiar. There is
a gap therein but no mere gap. It is a gap that
is intensely active. A sort of wraith of the name
is in it, beckoning us in a given direction,
making us at moments tingle with the sense of
closeness, and then letting us sink back without
the longed-for term William James (1890)
16
Tip of the tongue

• subjects in a TOT state are left with a
  disembodied presence, like the grin without the
  Cheshire cat Roger Brown, 1970 234

17
TOT experiment

• An absolute ruler tyrant.
• A place where bees are raised for their honey.
• To give up the throne.
• A navigational instrument used in measuring
  angular distances, especially the altitude of
  sun, moon, and stars at sea.
• A stone with a space in it, lined with crystals.
• Favoritism shown or patronage granted to
  relatives, as in business.
• A flat-bottomed Asian skiff usually propelled by
  two oars.

after Brown McNeill 1966
18
2. Language comprehension
19
word meaning grammatical structure

• Consider this
• Man bites dog. vs. Dog bites man.

20
Structure determines meaning

• Police kill man with TV tuner.
• Life means caring for hospital director.
• Retired priest may marry Springsteen.
• Kicking baby considered to be healthy.

21
upshot

• the intended meaning and the funny meaning do not
  result from different word meanings or sth.
• rather, they derive from different arrangements
  of words into word groups (phrases)
• so, structure determines meaning
• so, structure building (parsing) is a necessary
  component of language comprehension

22
ambiguity attachment decisions
S
NP
VP
V
NP
PP
hit
He
the man
with the binoculars
23
ambiguity attachment decisions
S
NP
VP
V
NP
PP
NP
hit
He
the man
with the binoculars
24
Language processing ERPs

• Building sentence structure (syntactic
  analysis) and integrating meaning (thematic
  processing) each have their own ERP signatures.

25
ERP language N400
Kutas Hillyard 1980
26
N400

• negative(-going) component
• peak latency around 400ms
• bi-lateral slightly posterior distribution
• N400 effect (amplitude modulation)
• (mis)match of word meaning with preceding context
• also affected by semantic priming

27
P600/SPS
Osterhout Holcomb 1993 Hagoort, Brown
Groothusen 1993
28
P600

• positive(-going) deflection
• peak latency around 600ms
• bilateral, centro-parietal distribution
• grammatical anomalies
• ambiguities that are resolved in a dispreferred
  way
• long-distance dependencies

29
ERP Early Left-Anterior Negativity
add picture
30
ELAN

• negative(-going) deflection
• peak latency around 200ms
• left anterior distribution
• grammatical violations, e.g.
• phrase structure
• inflection, function words

31
Is syntactic processing modular?

• classic theory (Frazier) syntax first
• first step building structure on the basis of
  grammatical properties of words only (syntactic
  analysis)
• second step interpretation (thematic processing)
• third step (if necessary) reanalysis
• Issue
• are decisions made by the syntactic processor
  affected by meaning and discourse properties?
  (against syntax first)

32
sentence disambiguation in context
One-Referent context David had told the boy and
the girl to clean up their room before lunch
time. But the boy had stayed in bed all morning,
and the girl had been on the phone all the
time. (1) complement target David vertelde het
meisje dat er visite kwam. David told the girlNEU
thatCOMPL there would be some visitors. (2)
relative target David vertelde het meisje dat had
zitten bellen op te hangen. David told the
girlNEU thatRELPR(NEU) had been talking on the
phone to hang up.
van Berkum, Brown, Hagoort 1999
33
sentence disambiguation in context
Two-Referent context David had told the two girls
to clean up their room before lunch time. But one
of the girls had stayed in bed all morning, and
the other had been on the phone all the time. (1)
complement target David vertelde het meisje dat
er visite kwam. David told the girlNEU thatCOMPL
there would be some visitors. (2) relative
target David vertelde het meisje dat had zitten
bellen op te hangen. David told the girlNEU
thatRELPR(NEU) had been talking on the phone to
hang up.
34
het meisje dat er visite kwam (complement clause)
_

solid one referent (meisje) in contextdotted
two referents in context P600 effect
35
het meisje dat had zitten bellen (relative
clause)
_

solid one referent (meisje) in contextdotted
two referents in context P600 effect
36
3. Language production
37
A production model
38
electrofysiologisch onderzoek
Van Turennout 1997
39
electrophysiological study

• picture naming
• decision A
• animal ? left hand button press
• no animal ? right hand button press
• decision B
• word ends in r ? GO
• different final sound ? NO GO

40
examples stimuli
41
expectations

• concept- lemma-activation precede phonological
  planning
• so semantic word information is available
  earlier than phonological information
• therefore, decision A which hand? can be taken
  earlier than decision B go or no-go

42
lateralized readiness potential (LRP)

• a motor-related brain potential that precedes a
  specific hand movementKornhuber Deecke 1965
• a couple of hundreds of milliseconds before the
  onset of a hand movement, the EEG-signal becomes
  more negative over the contralateral scalp
• ? preparation of the hand movement in the motor
  cortex

43
predictions

• as soon as information necessary for decision A
  (L or R) is available, movement preparation
  begins
• as soon as information necessary for decision B
  (go/nogo) is available, motor preparation is is
  either discontinued or not
• so
• animal or not ? onset motor prep L of R
• final sound ? interruption of of motor prep or not

44
procedure

• 0 fixation cross
• 750 blank screen
• 1500 picture
• 1650 frame signalling judgment task in 50 of
  trials
• 3150 frame disappears overt naming
• 4000 picture disappears

45
results experiment 1
Both in Go and NoGo trials the signal becomes
more negative. NoGo trials decrease of
negativity after 120 ms
370
490
46
results experiment 1

• it takes about 120 ms. before the LRP starts to
  decrease in NoGo trials
• keep in mind the Go/NoGo decision depends on
  phonology (final sound of the word)
• conclusion (given assumptions) 120 ms passes
  between activation of semantic information and
  retrieval of the words final sound

47
experiment 2

• just as experiment 1, maar
• Decision B (Go/NoGo) is made dependent on the
  initial sound of the target word

48
experiment 2 result
LRP begins to level off after 40 ms. in NoGo
trials.
49
experiment 2 result

• word-initial sounds are retrieved earlier than
  word-final sounds
• ? sequential activation (L ? R) of a words
  phonological segments

50
control experiment

• reversal of decisions
• phonological form determines hand selection (L -
  R)
• semantics determines Go/NoGo
• New prediction
• NoGo trials no LRP, since the Go/NoGo decision
  can be made earlier than the L-R decision
• given the assumptions spelled out earlier

51
control experiment result
No LRP in NoGo trials!
52
experiment 3

• decision A (L or R)
• depends on grammatical gender(a syntactic
  lemma property)
• decision B (Go/NoGo)
• depends on phonology (initial sound)
• expectation?
• analogous to experiment 1

53
experiment 3 resultaat
370
410
54
Tip of the tongue

• correct naming of grammatical gender in a TOT
  exceeds chance expectations
• ? lemma is activated
• Miozzo, M, and A, Caramazza (1997). Retrieval of
  lexical-syntactic features in Tip-of-the-tongue
  states. Journal of Experimental Psychology
  Learning, Memory and Cognition. Vol. 19, No 4,
  841-850.

55
Conclusie (Van Turennout 1997)
the speaker has access... to the semantic and
grammatical properties of a word before the
phonological word form is specified. it is
impossible to specify a word phonologically
before having retrieved the semantic and
grammatical properties of that word from memory
... also the phonological form of a word is
constructed sequentially the initial sound is
available earlier than the final sound (p. 150)
56
a puzzle

• If grammatical encoding (lemmas) and
  phonological encoding (lexemes) are strictly
  serialized and independent, how can we explain
  accidental word substitutions (slips of the
  tongue) in which target and intrusion are both
  form- and meaning-related.

57
4. The neural substrate
58
Lateralisation of language (I)

• righthanded people
• 98 left hemisphere is language dominant
• lefthanded people
• clear LH dominance in only 60-80
• righthanded people with lefthanded relatives
• LH dominance is weaker

59
Split brain
60
Neurolinguistics

• The study of brain language relationships
  through neurological deficits
• Prime example aphasia
• A deficit in producing and understanding spoken
  and written language due to focal brain damage in
  persons who have gone through normal language
  development. Prins Bastiaanse 1997
• Incidence
• approx. 20.000 patients in NL
• approx. 6000 new cases each year in NL
• 95 of all aphasias after LH lesions

61
Paul Broca (1824-1880)

• 1861 Broca discovers in a post mortem study
  (Monsieur Tan) that speech/language (production)
  is associated with the foot of the 3rd
  convolution of the frontal lobe
• Brodmanns areas 44 45
• today, we call this Brocas area

62
Mr. Tans brain
63
Carl Wernicke (1848-1904)

• 1874 Wernicke discovers a second cortical area
  connected to language the posterior part of the
  uppermost temporal gyrus (STG), right behind the
  primary auditory cortex
• Brodmanns area 22
• today Wernickes area

64
Brodmanns areas
Korbinian Brodmann (1868-1918)
65
Brocas aphasia

• disfluent and effortful speech
• poor intonation
• relatively unaffected comprehension
• restriction of grammar in production
  telegraphic language (omission of functional
  elements, closed class morphemes)

agrammatism
66
agrammatic language production

• omission of TENSE markers (? infinitives)
• maar eh even morgen bellen
• aux mod zijn are always finite
• omission of grammatical subject (pro drop)
• __ belt de nummer op
• omdat ___ de nummer opbelt
• omission of determiners (det drop)
• __ meisje van school
• dat is meisje van school

67
afasie van Wernicke

• fluent, well-articulated speech
• normal intonation
• poor comprehension
• grammatical errors
• substitutions of function words closed class
  morphology

paragrammatism
68
Wernicke-Lichtheim-Geschwindt
69
Brocas vs. Wernicke areas

• Classical view (Wernicke/Lichtheim/Geschwindt)
• Broca production
• Wernicke comprehension
• Nota bene
• language is considered as a collection of skills
  (speaking, understanding, )
• this is very different from the modern modular
  view!

70
But

• Broca patients also have problems understanding
  non-canonical sentence structures involving
  constituent movement and long-distance dependency
  relations.(Zurif 1998 184)
• The ball that the boy is kicking ___ is red.
• The boy that the girl is pushing ___ is fat.
• 1 normal performance
• 2 chance level performance

71
And

• Wernickes patients have problems with semantic
  processes, e.g. aspectual coercion and complement
  coercion.(Piñango Zurif 2001)
• complement coercion the boy began the book
• aspectual coercion the girl jumped until dawn

72
Brocas vs. Wernickes aphasia(1970-s)

• Broca
• grammatical (syntactic) problems in production
  (agrammatism) comprehension
• Brocas area amodal syntax center
• Wernicke
• semantic problems (neologistic production and
  deficient comprehension)
• Wernickes area amodal semantics center

modern linguistic modularity
73
Some problems

• Lesion in Brocas area neither sufficient nor
  necessary to induce syntactic deficits
• Brocas area is not always lesioned in a
  clinically significant Brocas aphasia
• Brocas area can be affected in patients who do
  not display a Broca syndrome most of these
  patients are mildly anomic.
• Severity of morphosyntactic problems in aphasia
  is correlated with the extent of damage in BA 22.
• Also semantic deficits in Brocas aphasia.

74
Disadvantages of patient studies

• damage to neural tissue not well delineated (in
  functional terms)
• rather, depends on histological properties or on
  structure of vascular system
• possibility of compensation/adaptation
• diaschisis
• aphasic symptoms evolve over time (post onset)
• ? unclear which symptoms (and hence modules) are
  linked to which neural networks

75
Imaging studies

• PET
• fMRI

76
Imaging the grammar center

• Indefrey et al
• subjects describe computer animations of colored
  geometrical shapes moving around, e.g. a red
  square bumping into a blue ellipse
• three types of description (originally German)
• complete sentences, like the red square bumps
  the blue ellipse away
• sentence fragments, red square, blue ellipse,
  bump away
• separate words, square, red, ellipse, etc.
• difference in the amount of grammatical work
• more work more activation

77
Imaging the grammar center
Indefrey et al. (in Gazzaniga 2000)
78
But

• Brocas area is an excitable piece of
  tissue!(David Poeppel, p.c.)
• activated by (i.a.)
• memorizing word lists
• semantic tasks
• phonological tasks
• music perception? most likely not domain
  specific!

79
Grammar or working memory?

• Middle/superior temporal lobe
• lexical processing (activating semantic/syntactic,
  phonological features of words)
• Anterior temporal lobe
• combining activated information
• Brocas area
• storing non-integrated materials
• Right hemisphere
• prosody
• ambiguity
• discourse
• error detection

Kaan Swaab, 2002, TICS
80
Brocas area as a working memory for language
processing

• Serving
• Computation manipulation of processing elements
• Attachment operations
• Detachment and reanalysis
• Storage
• Unattached words
• Partially analyzed (incomplete) structure
  (fragments)Welk boek heeft hij voor haar
  verjaardag gekocht?

81
Utrecht Institute of Linguistics OTS
Faculteit Geesteswetenschappen, Subfaculteit
Letteren