PSY 369: Psycholinguistics

1
PSY 369 Psycholinguistics

• Language Comprehension
• Perception of language

2
Different signals

• Visual word recognition

• Speech Perception

• Some parallel input
• Orthography
• Letters
• Clear delineation
• Difficult to learn

• Serial input
• Phonetics/Phonology
• Acoustic features
• Usually no delineation
• Easy to learn

3
Different signals

• Visual word recognition

• Speech Perception

• Some parallel input
• Orthography
• Letters
• Clear delineation
• Difficult to learn

• Serial input
• Phonetics/Phonology
• Acoustic features
• Usually no delineation
• Easy to learn

4
Speech perception

• Brief review of last time
• Acoustic features of speech
• formants, transitions, bursts, VOT
• Hard problems in speech perception
• Linearity, invariance, co-articulation, trading
  relations
• Link between acoustics and articulation
• Categorical perception, motor theory of speech
  perception
• The focus was on bottom-up processing, today
  lets look at some top-down effects

5
Top-down effects on Speech Perception

• Speech sounds are not typically used in isolation
• Sentence context effects
• Phoneme restoration effect
• Segmentation effects

6
Phoneme restoration effect

• Warren (1970)

• Participants listened to a sentence which
  contained a word from which a phoneme was deleted
  and replaced with another noise (e.g., a cough)

The state governors met with their respective
legilatures convening in the capital city. /s/
deleted and replaced with a cough
Click here for a demo and additional information
7
Phoneme restoration effect

• Warren (1970)

• Typical results
• Participants heard the word normally, despite
  the missing phoneme
• Usually failed to identify which phoneme was
  missing
• Interpretation
• We can use top-down knowledge to fill in the
  missing information

8
Phoneme restoration effect

• Warren Warren (1970)

• What if the missing phoneme was ambiguous?

The eel was on the axle.
The eel was on the shoe.
The eel was on the orange.
The eel was on the table.

• Results
• Participants heard the contextually appropriate
  word normally, despite the missing phoneme

9
Phoneme restoration effect

• Possible loci of phoneme restoration effects
• Perceptual loci of effect
• Lexical or sentential context influences the way
  in which the word is initially perceived.
• Post-perceptual loci of effect
• Lexical or sentential context  influences
  decisions about the nature of the missing phoneme
  information.

10
Beyond the segment

• Shillcock (1990)
• Participants hear a sentence, make a lexical
  decision to a word that pops up on computer
  screen (cross-modal priming)

Hear
The scientist made a new discovery last year.
NUDIST
11
Cross-modal priming

• Shillcock (1990)
• Participants hear a sentence, make a lexical
  decision to a word that pops up on computer
  screen (cross-modal priming)

Hear
The scientist made a novel discovery last year.
NUDIST
12
Cross-modal priming

• Shillcock (1990)
• Participants hear a sentence, make a lexical
  decision to a word that pops up on computer
  screen (cross-modal priming)

Hear
The scientist made a novel discovery last year.
The scientist made a new discovery last year.
faster
NUDIST
13
Cross-modal priming

• Shillcock (1990)
• Participants hear a sentence, make a lexical
  decision to a word that pops up on computer
  screen (cross-modal priming)

Hear
The scientist made a novel discovery last year.
faster
The scientist made a new discovery last year.
NUDIST

• NUDIST gets primed by segmentation error

• Although no conscious report of hearing nudist

14
Speech recognition

• Models of spoken word recognition

• Cohort Model
• Items may be eliminated from the cohort if
  inconsistent with the context but only after the
  initial cohort has been activated by bottom up
  information.
• Activation levels among cohorts varies and this
  is the mechanism which allows for frequency
  effects and lexical similarity.

• TRACE Model
• Unlike the cohort model this model allows for
  top down effects at all levels of processing
• Thus this model can account for effects like
  the phonemic restoration effect at the earliest
  levels of processing (in the cohort model this
  could only occur after processing was complete).

15
Speech recognition

• Speech perception is an active, constructive
  process
• Listeners do not simply attend to spoken
  information
• Categorical perception
• Also attend to visual information
• McGurk effect
• Also use lexical (word) and contextual knowledge
  to generate hypotheses about the likely form of
  the spoken information.
• Phoneme restoration, segmentation studies

16
Different signals

• Visual word recognition

• Speech Perception

• Some parallel input
• Orthography
• Letters
• Clear delineation
• Difficult to learn

• Serial input
• Phonetics/Phonology
• Acoustic features
• Usually no delineation
• Easy to learn

17
Visual perception of language

• Why so much research using visual language?
• We do use it
• Easy to use in research
• The parts
• Letters
• Words
• Eye movements

18
Invariance a problem in vision too?
G
G
G
G
G
G
G
Same object category (G) may have different
shapes, sizes, and orientations
G
G
G
G
G
G
G
G
G
G
G
G
G
G
G
Perhaps the brain is able to represent these
objects in a way that is translationally invaria
nt and size invariant.
19
Letter Recognition

• How do we recognize a group of lines and curves
  as letters?
• Two common explanations
• Template matching
• Feature detection

• Okay, Im going to show you some stimuli really
  fast and you need to tell me what they are

20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Template matching

• Store in brain a copy of what every possible
  input will look like.
• Match observed object to the proper image in
  memory

28
Template matching
Memory Representations
29
Problems with Template matching

• Costly Massive numbers of templates are required
  (remember all those Gs?)..
• Predicts no transfer to novel views of the same
  object
• Normalization before matching - mentally
  cleaning it up before matching to templates

30
Problems with Template matching

• Costly Massive numbers of templates are required
  (remember all those Gs?)..
• Predicts no transfer to novel views of the same
  object
• Normalization before matching - mentally
  cleaning it up before matching to templates
• Objects are often obstructed/occluded

31
Prolblems with Template matching

• Costly Massive numbers of templates are required
  (remember all those Es?)..
• Normalization before matching - mentally
  cleaning it up before matching to templates
• Predicts no transfer to novel views of the same
  object
• Objects are often obstructed/occluded

E
FROG

• Objects are often disamiguated by context

32
Read aloud the following word
33
Read aloud the following word
34
So what is the middle letter?

• Clearly, top-down influences. However it is
  unclear how this works with template matching

35
Feature detection

• Analysis-by-synthesis
• 1. Letter broken down to its constituent parts
• 2. List of parts compared to patterns in memory
• 3. Best matching pattern chosen

36
Feature detection
A fixed set of elementary properties are
analyzed Independently and in parallel across
visual field.
Possible examples
Free line endings
45deg. -10deg.
Line Orientations
Different Sizes
Curvature
Colors
37
A simple theory of Feature detection
Perceptual Representation
Memory Representation
E
3 Horizontal lines 1 Vertical line 4 Right angles
3 Horizontal lines 1 Vertical line 4 Right
angles
F
2 Horizontal lines 1 Vertical line 3
Right angles
38
Evidence for Features
The visual search task is straightforward, you
are given some target to look for, and asked to
simply decide, as quickly as possible, whether
the target is present or absent in a set of
objects. For example, lets try a few searches
to give you a feel for this. Search 1 - Is there
an O present in the following displays?
39
Is an O present?
T T T T T O T T T
40
Is an O present?
T T T T TTT T T T T T T T T
T T O TTT T TT TT T TT T T TT T T T TT T
TTT T TT
41
Is an O present?
Q Q Q Q Q Q O Q Q Q Q
Q Q
42
Is an O present?
Q QQ Q Q QQQ QQQ Q QQQ Q O Q Q QQ Q Q QQ
Q Q Q Q QQQ Q QQ Q Q QQQ Q QQQQ Q
43
Is an O present?
T T T T T O T T T
T T T T TTT T T T T T T T T
T T O TTT T TT TT T TT T T TT T T T TT T
TTT T TT
Q QQ Q Q QQQ QQQ Q QQQ Q O Q Q QQ Q Q QQ
Q Q Q Q QQQ Q QQ Q Q QQQ Q QQQQ Q
Q Q Q Q Q Q O Q Q Q Q
Q Q
44
A theory of Feature detection
Selfridges Pandemonium system, 1959
45
Another theory of Feature detection
46
Interactive Activation Model (AIM)
Previous models posed a bottom-up flow of
information (from features to letters to words).
IAM also poses a top-down flows of information

• Nodes
• (visual) feature
• (positional) letter
• word detectors
• Inhibitory and excitatory connections between
  them.

• McClelland and Rumelhart, (1981)

47
Interactive Activation Model (AIM)

• Inhibitory connections within levels
• If the first letter of a word is a, it isnt
  b or c or
• Inhibitory and excitatory connections between
  levels (bottom-up and top-down)
• If the first letter is a the word could be
  apple or ant or ., but not book or
  church or
• If there is growing evidence that the word is
  apple that evidence confirms that the first
  letter is a, and not b..

48
The Word-Superiority Effect (Reicher, 1969)

Until the participant hits some start key
49
The Word-Superiority Effect (Reicher, 1969)
COURSE
Presented briefly say 25 ms
50
The Word-Superiority Effect (Reicher, 1969)
U A
Mask presented with alternatives above and
below the target letter participants must pick
one as the letter they believe was presented in
that position.
51
The Word-Superiority Effect (Reicher, 1969)



E
PLANE
KLANE
E T
E T
E T
Letter only Say 60
Letter in Nonword Say 65
Letter in Word Say 80
Why is identification better when a letter is
presented in a word?
52
IAM the word superiority effect

• We are processing at the word and letter levels
  simultaneously
• Letters in words benefit from bottom-up and
  top-down activation
• But letters alone receive only bottom-up
  activation.

53
Other Relevant Findings?

• Bias towards well-formed stimuli
• Misidentify words with uncommon spelling patterns
  
• BOUT as BOAT
• misidentify non-words (e.g., SALID) as words that
  are like it (SALAD).
• Difficulty identifying non-words with irregular
  spelling patterns (e.g., ITPR) more than those
  with regular spelling patterns (e.g., PIRT).

54
Sublexical units

• Sublexical units bigger than phonemes and
  graphemes?
• onsets and rimes
• onset initial consonant or consonant cluster in
  a word or syllable
• rime following vowel and consonants
• if words broken at onset-rime boundary, resulting
  letter clusters more easily recognized as
  belonging together than if broken at other points
• example FL OST ANK TR
• vs. FLA ST NK TRO

55
Adding a bigram level
By adding a frequency-sensitive bigram level, we
can account for the findings of well-formedness
along with the others.
56
Summing up

• Evidence supports the view that our word
  recognition processes are based on a
  feature-detector system
• Biased to perceive common or recently occurring
  features

57
Studying Word Identification

• Generally people ask what makes word
  identification easy or difficult?
• The assumption
• Time spent identifying a word can be a measure of
  difficulty
• Measures of identification time are usually
  indirect

58
Some Identification Time Measures

• Measure how long people take to say a string of
  letters is (or is not) a word (lexical decision)
• Measure how long people take to categorise a word
  (apple is a fruit)
• Measure how long people take to start saying a
  word (naming or pronunciation time)
• Measure how long people actually spend looking at
  a word when READING
• Line by line reading
• Word by word reading
• using eye movement monitoring techniques

59
Line-by-line

• A banker is a fellow

60
Line-by-line
who lends you his umbrella
61
Line-by-line
when the sun is shining
62
Line-by-line
but wants it back
63
Line-by-line
the minute it begins to rain.
64
Line-by-line

• Problem
• Overall reading time for entire sentence or
  phrase
• need for more on-line measurements
• Timing on a smaller scope
• See effects at level of word

65
Word-by-word

• RSVP (rapid serial visual presentation)

66
Word-by-word

• A

67
Word-by-word
lie
68
Word-by-word
can
69
Word-by-word
travel
70
Word-by-word
halfway
71
Word-by-word
around
72
Word-by-word
the
73
Word-by-word
world
74
Word-by-word
while
75
Word-by-word
the
76
Word-by-word
truth
77
Word-by-word
is
78
Word-by-word
putting
79
Word-by-word
on
80
Word-by-word
its
81
Word-by-word
shoes.
82
Word-by-word

• Moving window

83
Word-by-word

• I xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

84
Word-by-word

• x have xxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

85
Word-by-word

• x xxxx never xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

86
Word-by-word

• x xxxx xxxxx let xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

87
Word-by-word

• x xxxx xxxxx xxx my xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

88
Word-by-word

• x xxxx xxxxx xxx xx schooling xxxxxxxxx xxxx xx
  xxxxxxxxx.

89
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx interfere xxxx xx
  xxxxxxxxx.

90
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx with xx
  xxxxxxxxx.

91
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx my
  xxxxxxxxx.

92
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  education.

93
Word-by-word

• A couple of methods
• RSVP (rapid serial visual presentation)
• Moving window
• Better, more on-line
• But, these measures are also a little bit
  unnatural (especially RSVP)
• e.g., Dont allow regressions (looking back)

94
Eye-movements

• Eyemovement studies

The kite fell on the dog
95
Eye-movements

• Eyemovement studies

The kite fell on the dog
96
Eye-movements

• Eyemovement studies

The kite fell on the dog
97
The Human Eye

• At its center is the fovea, a pit that is most
  sensitive to light and is responsible for our
  sharp central vision.
• The central retina is cone-dominated and the
  peripheral retina is rod-dominated.

98
Retinal Sampling
99
Retinal Sampling
100
Eye Movements

• Within the visual field, eye movements serve two
  major functions
• Saccades to Fixations Position target objects
  of interest on the fovea
• Tracking Keep fixated objects on the fovea
  despite movements of the object or head

101
Fixations

• The eye is (almost) still perceptions are
  gathered during fixations
• The most important of eye movements
• 90 of the time the eye is fixated
• duration 150ms - 600ms

102
Saccades

• Saccades are used to move the fovea to the next
  object/region of interest.
• Connect fixations
• Duration 10ms - 120ms
• Very fast (up to 700 degrees/second)
• No visual perception during saccades
• Vision is suppressed
• Evidence that some cognitive processing may also
  be suppressed during eye-movements (Irwin, 1998)

103
Saccades
Move to here
104
Saccade w/o suppression
105
Saccades
Move to here
106
Saccades
107
Saccades

• Saccades are used to move the fovea to the next
  object/region of interest.
• Connect fixations
• Duration 10ms - 120ms
• Very fast (up to 700 degrees/second)
• No visual perception during saccades
• Vision is suppressed

• Ballistic movements (pre-programmed)
• About 150,000 saccades per day

108
Smooth Pursuit

• Smooth movement of the eyes for visually tracking
  a moving object
• Cannot be performed in static scenes
  (fixation/saccade behavior instead)

109
Smooth Pursuit versus Saccades

• Saccades
• Jerky
• No correction
• Up to 700 degrees/sec
• Background is not blurred (saccadic suppression)

• Smooth pursuit
• Smooth and continuous
• Constantly corrected by visual feedback
• Up to 100 degrees/sec
• Background is blurred

110
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

111
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

112
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

113
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

114
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

115
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

116
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

117
Eye-movements in reading

• Eye-movements in reading are saccadic rather than
  smooth

• Clothes make the man. Naked people have little
  or no influence on society.

118
Eye-movements in reading

• Limitations of the visual field
• 130 degrees vertically, 180 degrees horizontally
  (including peripheral vision
• Perceptual span for reading 7-12 spaces

Clothes make the man. Naked people have little
or no influence on society.
119
Measuring Eye Movements

• Purkinje Eye Tracker
• Laser is aimed at the eye.
• Laser light is reflected by cornea and lens
• Pattern of reflected light is received by an
  array of light-sensitive elements.
• Very precise
• Also measures pupil accomodation
• No head movements

120
Measuring Eye Movements

• Video-Based Systems
• Infrared camera directed at eye
• Image processing hardware determines pupil
  position and size (and possibly corneal
  reflection)
• Good spatial precision (0.5 degrees) for
  head-mounted systems
• Good temporal resolution (up to 500 Hz) possible