Methods:%20A%20Brief%20Survey

1
Methods A Brief Survey
S
NP
VP

• Eva M. Fernández
• eva_fernandez_at_qc.edu
• ABRALIN 24FEB05

2
Tree-Building is Really Happening?

• Is syntactic structure psychologically real?
• even though its abstract
• and not at all present in the signal?
• A simple test RSVP task
• (see slides from 22FEB05)

3
RSVP Paradigm

• Center-screen, word-by-word display
• Timing N ms per word (here N 500)
• Sentence-recall task

4
?
The
beautiful
black
cat
chased
the
colorful
ball
.
The beautiful black cat chased the colorful ball.
5
?
Black
colorful
the
ball
chased
cat
beautiful
the
.
Black colorful the ball chased cat beautiful the.
6
What Information is Used?

• In building (recovering) syntactic structure,
  the information used includes
• LEXICAL INFORMATION
• MORPHO-SYNTACTIC INFORMATION
• PHONOLOGICAL INFORMATION (including prosody)
• SEMANTIC INFORMATION

GRAMMAR
7
Three Operations of the Syntax

• Building simple structure
• Mary often speaks about Joe.
• Mary often speak about Joe.
• Combining simple sentences into complex ones
• Mary told Joe something. Joe had his
  shoelace untied.
• ? Mary told Joe he had his shoelace untied.
• Moving elements of sentences around
• Mary told Joe what
• ? What did Mary tell Joe __ ?

8
It follows that

• If the grammar is consulted during sentence
  processing,we should expect that the system will
  dislike
• UNGRAMMATICAL sentences
• COMPLEX sentences
• sentences involving lots of MOVEMENT
• dislike take longer to process? be
  unable to process?
• just how could one measure the parsers tastes?
• could one just ask the listener/reader?

9
A Potential Problem

• You might reject a grammatical sentence because
  its prescriptively bad
• Wheres the library at?
• Who did Joe meet at the party?
• Marys love life doesnt concern you and I.
• Me and you shouldnt be talking about Marys
  affairs.
• This is historic times. (GWB)
• etc.!

10
Another (More Important) Problem

• You might reject a grammatical sentence because
  its hard (or impossible!) to process

Mary put the candy on the table in her
mouth. When Madonna sings the song is always a
hit. The son of Paraohs daughter looked at
himself in the mirror. The cat the dog the boy
walked bit meowed. The machine covered with paper
plates handles with chrome. etc.!
11
Processing Difficulty Why?

• Why does the parser dislike some sentences more
  than others?
• Because of the listeners inadequate knowledge
  of language?
• Because of sentence processing routines that
  have gone wrong, that have applied incorrectly?

12
Processing Routines Defined

• Mechanisms that operate during production and
  perception
• in constant contact with the grammar
• based on working-memory limitations
• Minimal Attachment build the simplest structure
• Late Closure attach locally
• Minimal Chains posit the fewest filler-gap
  dependencies

13
Psychological Reality, Again...

• How can you tell that the processing routines are
  being followed?
• ? Observe their work in action
• Physics which will fall faster, 1kg of feathers
  or 1kg of lead?
• Psycholinguistics which linguistic stimulus
  will be understood more easily and faster, one
  with a Minimal Attachment violation or one
  without?

14
Forster Chambers, 1973 (described in Forster,
1979)

• stimuli letter sequences were either words or
  orthographically legal non-words, e.g.,
  thamon
• tasks (i) naming (pronounce ASAP)
  (ii) lexical decision
• results lexical decision times (608 ms)
  gt naming times (508 ms)

15
Forster, 1974 (also described in Forster, 1979)

• stimuli grammatical, ungrammatical, non-word
  sentences
• The scouts annoyed the lady
• The bicycle the calculated cognac
• The plane gleashed the passengers
• tasks (i) are all items in input familiar
  words? (ii) is sentence meaningful?
• results no significant difference between
  decision times for the two tasks, and indeed
  for some types of sentences, the sentence task
  was slightly faster than the lexical task (p.
  30)

16
Organization of Language Processor and GPS (after
Forster, 1979)
MESSAGEPROCESSOR
SYNTACTICPROCESSOR
LEXICON
Gral PROBLEM SOLVER
Gral CONCEPTUAL KNOWLEDGE
LEXICALPROCESSOR
INPUTFEATURES
DECISIONOUTPUT
17
Ambiguity

• At almost any point during structure-building,
  the parser has multiple options (LOCAL
  AMBIGUITY!)

S
S
NP
VP
NP
VP
V
V
NP
S
We knew Anns date
We knew Anns date would embarrass her
at the party.
We knew Anns date
We knew Anns date, Joe.
18
Multiple Local Ambiguities

• Have the soldiers marched into the barracks
• ?
• , please!
• had their supper?
• take their boots off, would you?

19
Ambiguity

• Not to be confused with vagueness
• I guess the movie was interesting.
• Occurs when a lexical string has two possible
  structures,GLOBALLY
• The bird is ready to eat.
• Visiting relatives can be a real nuisance.
• Joe said Mary called him yesterday.
• Joe saw Mary with the telescope.
• Joe saw a dog next to a kitten with an orange
  sweater.
• or LOCALLY
• We knew Anns date
• Have the soldiers marched

, Joe. would embarrass her at the party.
into the barracks? into the barracks,
please! into the barracks eaten?
20
When theres an ambiguity

• How does the parser go about choosing among the
  alternatives?
• Does the parser notice all of the alternatives?
• If so, are all of the alternatives kept active?
• A debate in the literature, indeed, but the
  consensus
• THE PARSER NOTICES THE STRUCTURAL ALTERNATIVE
  THATS EASIEST TO BUILD, following strategies
  like
• Minimal Attachment build the simplest
  structure
• Late Closure attach locally
• Minimal Chains posit the fewest filler-gap
  chains

21
AN EXPERIMENT

• The legionnaires marched into the desertand
  searched for the nearest oasis.
• The legionnaires marched into the
  desertsurprised the Persian forces.

MY HYPOTHESIS
2nd sentence is harder violates Minimal
Attachment lets see how!
22
The legionnaires marched into the desert and
searched for the nearest oasis.
S
VP
conj
VP
VP
and
marched into the desert
searched for the nearest oasis
23
The legionnaires marched into the desert
surprised the Persian forces.
S
NP
marched into the desert
surprised the Persian forces Uh - oh!
24
THE EXPERIMENTAL METHOD

• Hypothesis testing
• you have an idea about how structures are built
  by the sentence processor
• you test it by designing an experiment
• you make predictions about how the experiment
  will come out
• you think about what it will mean if the
  experiment comes out the wrong way
• this will be easier if you use BINARY COMPARISONS

25
PARTICIPANTS

• never too many?
• clones of each other
• selection criteria
• language history, education history, sex,
  handedness, age, reading ability, etc.
• controlling
• use background questionnaire
• use behavioral criteria

26
MATERIALS

• never too many?
• instantiating the contrast youre interested in
• normed in whatever relevant way
• interspersed among distractor items
• never too many?
• usual filler-target ratio, 2-1 or 3-1
• pseudo-randomized lists
• featuring binary comparisons (unless absolutely
  necessary)

27
PROCEDURE

• speeded classification tasks
• the subject of the experiment is presented with
  some item of linguistic input, which must be
  classified according to some experimenter-defined
  criterion
• try several types! (ingenuity is essential)
• dont be fooled by technology
• just because it costs more doesnt mean its more
  effective
• just because everyone says its online doesnt
  mean it taps the phase of processing you want
  (e.g., how do you know that information hasnt
  flowed through a later processor before the GPS
  makes a decision?)

28
QUESTIONNAIRES

• a grammaticality / acceptability judgment task
• On a scale of 1-7, indicate how acceptable you
  think each of these sentences are.
• 1 perfectly acceptable
• 7 perfectly awful
• measure most frequent response type

29

• The legionnaires marched into the desert and
  searched for the nearest oasis.
• 1 2 3 4 5 6 7
• The legionnaires marched into the desert
  surprised the Persian forces.
• 1 2 3 4 5 6 7

30
ASSETS LIABILITIESof questionnaire procedures?

• flexible
• low-tech lab
• test 100 people at a time!
• off-line metalinguistic awareness, prescriptive
  rules
• subjects might engage in undesirable behavior
  strategies, looking back, etc.

31
RSVP

• rapid serial visual presentation
• words presented center-screen
• 500 msec for each word
• presentation rate could be varied
• task is to remember entire sentence andrepeat or
  write down
• measure accuracy of recall

32
?
The
legionnaires
marched
into
the
desert
and
searched
for
the
nearest
oasis
.
33
?
The
legionnaires
marched
into
the
desert
surprised
the
Persian
forces
.
34
SELF-PACED READING

• subject controls pace of presentation
• constraints on speed determined by experimenter
• time-outs, instructions
• materials presented in a series of chunks
• word-by-word, phrase-by-phrase, clause-by-clause
• different types of displays
• centered, incremental, moving window
• measure reading time

35
CTRD, WD-BY-WD
?
The
legionnaires
marched
into
the
desert
and
searched
for
the
nearest
oasis
.
36
CTRD, WD-BY-WD
?
The
legionnaires
marched
into
the
desert
surprised
the
Persian
forces
.
37
SPR, wd-by-wd measure
The lgnnaires marched into the desert
and searched for the nearest oasis

The lgnnaires marched into the desert
surprised the Persian forces
38
CTRD, PHR-BY-PHR
?
The legionnaires marched
into the desert
and searched
for the nearest oasis.
39
CTRD, PHR-BY-PHR
?
The legionnaires marched
into the desert
surprised
the Persian forces.
40
SPR, wd-by-wd measure
The legionnaires marched The legionnaires marched The legionnaires marched into the desert into the desert into the desert
and searched and searched for the nearest oasis for the nearest oasis for the nearest oasis for the nearest oasis

The legionnaires marched The legionnaires marched The legionnaires marched into the desert into the desert into the desert
surprised surprised the Persian forces the Persian forces the Persian forces the Persian forces
41
INCREMENTAL
?
The
student
told
the
professor
that
everyone
hated
a
lie.
42
MOVING WINDOW
The
student
told
the
professor
that
everyone
hated
a
lie.
?
43
MOVING WINDOW VARIATION
The
student
told
the
professor
that
everyone
hated
a
lie.
---
-------
----
---
---------
----
--------
-----
-
---.
44
KEEPING THE PARTICIPANTS HONEST

• asking questions
• after every trial
• after every N trials
• never?
• excluding error-prone subjects
• providing instant feedback
• the speed-accuracy trade-off
• the faster you respond, the more likely youll
  make errors

45
Who told a lie? the student
the professor
Did the student lie?
Did the professor lie?
The
student
told
the
professor
that
everyone
hated
a
lie.
---
-------
----
---
---------
----
--------
-----
-
---.
correct 2359
46
SAME-DIFFERENT SENTENCE-MATCHING
47
?
The legionnaires marched into the desert and
searched for the nearest oasis.
The legionnaires marched into the desert and
searched for the nearest oasis.
48
?
The alligator with the sharp teeth inspected the
rifle.
The alligator with the sharp teeth inspected his
rifle.
49
?
The legionnaires marched into the desert
surprised the Persian forces.
The legionnaires marched into the desert
surprised the Persian forces.
50
?
Everyone at the party knew Anns date had made a
fool of himself.
Everyone at the party knew Anns mate had made a
fool of himself.
51
EYE-TRACKING fine-grained

• visual span
• 9 letters, periphery
• focus move
• eyetracker records focus
• measurements
• first fixations
• regressions
• total reading

See how the eye moves across the page in normal
reading http//gandalf.psych.umn.edu/gellab/mrc
hips/chips2d.html animation by Steve Mansfield
Tim Klitz
52
Eyetracking challenges

• a more on-line method than SPR?
• the problem of time during saccades
• the problem of as much time as necessary for a
  given item
• the problem of peripheral vision
• more limitations
• length of items limited by size of display
• costly equipment
• bite-bar
• laborious analyses

53
EYETRACKING coarse-grainedReal World paradigm

• Head-mounted eyetrackers permit examining how
  the immediate visual context can affect sentence
  processing.
• Easy to use, with adults... and children
• Evidence of interactive system for language
  processing? (See Garrett, 2000, for discussion.)

54
BRAIN IMAGING

• distinct brains systems associated with several
  different structural domains phrase structure,
  inflection, movement and binding, and lexical
  semantics (Garrett, 2000, p. 43)
• semantic versus syntactic task priorities
• different electrode locations
• different timing
• early responses to syntax
• inflexible, localized in classic language
  processing areas
• later responses to syntax
• more sensitivity to interpretative factors
• more broadly distributed