Speech perception

1
Speech perception

• Langston
• Psycholinguistics
• Lecture 3

2
Plan

• Top-down
• Comprehension
• Bottom-up

3
Plan

• Our goal is to start with the input and see how
  far we can take it.
• Constraint satisfaction problem.
• We will introduce top-down influences when the
  situation demands it.

4
What is speech?

• Levels of analysis
• Acoustic The physical speech signal.
• Articulatory How it's made.
• Phones Individual sounds (approximately 4000
  available, about 869 in some language, about 100
  account for most, Kluender, 1994).
• Phonemes Mental representation of sounds or
  sounds that affect meaning. Often made up of
  several phones treated as alike (keep cool).

5
What is speech?

• Levels of analysis
• Phonemes Not all differences in sounds are
  phonemic (pin spin). Allophones Set of phones
  treated as identical by a language.
• Changing a phoneme will change the meaning (bit
  pit).
• You can map a language's phonemes by looking for
  minimal pairs.

6
What is speech?

• Levels of analysis
• Phonemes Languages seem to choose phonemes to
  maximize distinctiveness
• (Kluender, 1994)

7
What is speech?

• Levels of analysis
• Morphemes Units that actually have meaning
  (we'll come to these later).

8
Articulatory Phonetics

• Based on how sounds are produced.
• A consonant is
• Air
• Voicing (on or off)
• Manner (some form of disruption)
• Place (where the disruption happens)

9
Articulatory Phonetics

• The places

10
Articulatory Phonetics

• Here's a link to a map with a clickable glossary
  http//www.sil.org/mexico/ling/glosario/E005ci-Pla
  cesArt.htm

11
Articulatory Phonetics

• The manners
• Plosive (stop) Completely stop the air flow.
• Fricative Interrupt the air flow and create
  friction.
• Affricate Stop released to a fricative.
• Nasal Stop with sound coming out the nasal
  passages.
• Flap Brief stoppage.
• Trill Hold it in place and let it vibrate.

12
Articulatory Phonetics

• The manners
• Approximant Like a fricative, little
  obstruction.
• Liquids Central (flow over the middle of the
  tongue) or lateral (flow around the sides of the
  tongue).
• Glides Similar to a vowel but with the tongue
  creating a small amount of turbulence (also
  called semivowels).

13
Articulatory Phonetics

• IPA table

14
Articulatory Phonetics

• English

15
Articulatory Phonetics

• English
• Bilabial
• Stop voiced bin, unvoiced pin
• Nasal man
• Approximants wind
• Labiodental
• Fricative voiced vat, unvoiced fat
• Dental
• Fricative voiced then, unvoiced thin

16
Articulatory Phonetics

• English
• Alveolar
• Stop voiced dip, unvoiced tip
• Nasal nap
• Flap city
• Fricative voiced zap, unvoiced sap
• Approximants central rip, lateral lip
• Post-alveolar (palatal?)
• Fricative voiced azure, unvoiced sure
• Affricate voiced jug, unvoiced chug

17
Articulatory Phonetics

• English
• Palatal
• Approximant your
• Velar
• Stop voiced got, unvoiced cot
• Nasal sing
• Glottal
• Stop satin
• Fricative hen

18
Articulatory Phonetics

• A vowel is
• Part front, center, back
• Height High, mid, low

19
Articulatory Phonetics

• English

20
Articulatory Phonetics

• English
• Front beet, bit, baby, bet, bat
• Central hut, sofa, bird, heater
• Back boot, book, bode, bought, hot

21
Articulatory Phonetics

• English
• Also dipthongs cute, bite, bough, boy
• Also suprasegmentals (added on to the vowels)
• Stress blackbird, blackbird
• Length
• Tone contour

22
Acoustic Phonetics

• You can use a spectrograph to produce a
  spectrogram. This is a graphic representation of
  speech.

23
Acoustic Phonetics

• If you download Praat you can produce your own
  spectrograms relatively easily. Get Praat here
  http//www.fon.hum.uva.nl/praat/

24
Acoustic Phonetics

• The acoustic approach is to analyze the physical
  speech signal without making reference to how it
  was produced.

25

• As an aside, we can think about vision for a
  minute
• D/H tan(?)

26
Acoustic Phonetics

• Formant a concentration of acoustic energy
  around a particular frequency in the speech wave
  (Praat Tutorial, see next page for link).

27
Acoustic Phonetics

• You can learn more about formants in the Praat
  tutorial here http//person2.sol.lu.se/SidneyWood
  /praate/whatform.html

28
Acoustic Phonetics

• Formant transition A sharp rise or fall in a
  formant. Usually a consonant.

29
Acoustic Phonetics

• Steady state Part of a formant with little or no
  change. Generally vowels.

30
Acoustic Phonetics

• The darker the band the more energy there is
  there.
• You can see sounds change over time by going from
  left to right.

31
Acoustic Phonetics

• Problems for perception
• Parallel transmission You do not produce
  phonemes like beads on a necklace. Instead, you
  are transmitting overlapping parts of phonemes in
  parallel (Easter eggs).

32
Acoustic Phonetics

• Problems for perception
• Parallel transmission

33
Acoustic Phonetics

• Problems for perception
• Context conditioned variation Each phoneme is
  affected by surrounding phonemes (lack of
  invariance).

34
Acoustic Phonetics

• Problems for perception
• Context conditioned variation

35
How Does Perception Work?

• From Kerzel Bekkering (2000 doi10.1037/0096-15
  23.26.2.634)
• Direct realism listeners to speech recover
  information about the articulatory activities of
  the vocal tract from various sources of
  information (p. 635).
• But Not motor based. The articulators structure
  the informational medium.

36
How Does Perception Work?

• From Kerzel Bekkering (2000)
• Direct realism when the ear of the listener is
  stimulated by the acoustic medium, the structure
  is imparted and the listener perceives the
  speaker's gestures (p. 635).
• Can also come from structuring of optic medium.
• Direct perception.

37
Example 1
http//sunburn.stanford.edu/nick/compdocs/,
click on Practical HI Examples.pdf
38
Examples 4 5
http//www.baddesigns.com/file.html
http//www.baddesigns.com/sidewalk.html
39
How Does Perception Work?

• Direct realism
• Carello, Anderson, Kunkler-Peck (1998
  doi10.1111/1467-9280.00040) Information in the
  auditory signal can be used to recover
  information about lengths of dowels (I'll be
  dropping some dowels).

40
How Does Perception Work?
Carello, Anderson, Kunkler-Peck (1998, p. 212)
41
How Does Perception Work?
Carello, Anderson, Kunkler-Peck (1998, p. 212)
42
How Does Perception Work?

• Direct realism
• Kunkler-Peck Turvey (2000 doi10.1037/0096-1523
  .26.1.279) Auditory information can also be used
  to recover information about an object's shape.

43
How Does Perception Work?

• Direct realism
• To sum up The signal contains sufficient
  structure to recover a distal property (shape).
  Speech could work the same way (the distal
  property is phonetic gesture).

44
How Does Perception Work?

• From Kerzel Bekkering (2000)
• Fuzzy logical model of perception (FLMP)
• features are evaluated in terms of prototypes of
  syllables (p. 635).
• degree of correspondence to the prototype is
  determined (p. 635).
• the relative goodness of match of each prototype
  is evaluated, and the prototype with the best
  match is selected (p. 635).

45
How Does Perception Work?

• From Kerzel Bekkering (2000)
• Fuzzy logical model of perception (FLMP)
• speech perception is explained by a best-match
  procedure (p. 635).

46
(No Transcript)
47
How Does Perception Work?

• From Galantucci, Fowler, Turvey (2006) Motor
  theory of speech perception. 3 parts
• speech processing is special (p. 361)
• perceiving speech is perceiving vocal tract
  gestures (p. 361)
• speech perception involves access to the speech
  motor system (p. 361)

48
How Does Perception Work?

• speech processing is special
• Perception of distal properties unique to speech.
  No. (See the shape stuff above.)
• Recruitment of the motor system unique to speech.
  No.
• Special neural hardware. Not enough evidence to
  tell, but probably no.

49
How Does Perception Work?

• perceiving speech is perceiving vocal tract
  gestures
• the objects of speech perception are the
  speakers' vocal tract gestures and not the
  acoustic patterns that the gestures generate in
  the air (p. 365)

50
How Does Perception Work?

• perceiving speech is perceiving vocal tract
  gestures
• When articulation and sound go their separate
  ways, which way does perception go? With
  articulation (di-du).

51
How Does Perception Work?

• perceiving speech is perceiving vocal tract
  gestures
• gestures may be specified by information other
  than that in air pressure waves (p. 365)
• McGurk effect (http//psiexp.ss.uci.edu/research/t
  eachingP140C/demos/McGurk_large.mov).
• Perception of speech in a noisy environment
  facilitated by seeing the speaker (Sumby
  Pollack, 1954).

52
How Does Perception Work?

• perceiving speech is perceiving vocal tract
  gestures
• Speech imitation is fast.
• In choice tasks, there is not the increase in
  reaction time expected.

53
How Does Perception Work?

• perceiving speech is perceiving vocal tract
  gestures
• the signal should be processed so that acoustic
  information for a given gesture is used as
  information for that gesture even when its
  acoustic consequences overlap with the acoustic
  consequences of another gesture (p. 366).
• Evidence is yes.

54
How Does Perception Work?

• speech perception involves access to the speech
  motor system
• Bell-Berti, Raphael, Pisoni, Sawusch (1979)
  The way people produce vowels is related to the
  way they perceive them.
• http//www.jango.com/music/MerleHaggard?l0 (go
  to 34 seconds).
• Or http//www.youtube.com/watch?vvfcv74S1Zmo
• Artist Merle Haggard Song I Think I'll Just
  Stay Here And Drink   Take all the money in the
  bank.Think I'll just stay here and drink.

55
How Does Perception Work?

• speech perception involves access to the speech
  motor system
• Kerzel Bekkering (2000 doi10.1037/0096-1523.26
  .2.634) Seeing a face make a syllable affects
  responses to written stimulus even though face is
  irrelevant.

56
How Does Perception Work?

• speech perception involves access to the speech
  motor system
• In sum, there is reason to believe that
  perception is particularly attuned to the general
  anatomical and dynamical constraints on
  biological movements.... In other words, the same
  conclusion that Liberman and colleaguesdrew
  specifically for speech, that speech motor
  competence must inform speech perception, can be
  drawnfor motor competence and perception quite
  generally. (p. 371).

57
How Does Perception Work?

• Also
• Mirror neurons Neurons that fire during action
  also fire when seeing someone else do the action.
  Perhaps action recognition comes from neural
  simulation of action. (But, mirror neurons in
  humans?)

58
How Does Perception Work?

• Also
• Canonical neurons Respond during grasping an
  object and when seeing the object. responsive to
  the actions that an object potentially affords,
  even when acting on the object is not required
  (Galantucci, et al., 2006, p. 371).

59
How Does Perception Work?

• Also
• the perceptual relationship between an animal
  and its surrounding physical world is reflected
  in the nervous system in a way that is intimately
  intertwined with the neural means for preparing
  to produce compatible actions (Galantucci, et
  al., 2006, p. 372).

60
Questions

• What about phonemic restoration?
• A shorter one for class https//www.youtube.com/w
  atch?v12G_YzHDmUs
• What about categorical perception?
  http//www.ling.gu.se/anders/KatPer/Applet/index.
  eng.html

61
Questions

• What about phonetic symbolism? (size-sound)

62
Questions

• What can be learned from statistical regularities
  in the input? Aslin, Saffran, Newport (1998
  doi10.1111/1467-9280.00063)
• Can learners use statistical information to
  discover word boundaries?
• Boundaries not marked by any consistent acoustic
  cues (p. 321).

63
Questions

• Aslin, Saffran, Newport (1998)
• Presented infants with words from an artificial
  language (3 syllables, 4 words in each of two
  languages).
• Put into a 3-minute stream
• Pabikugolatudaropitibudodaropigolatu
• Infants listen longer to part-words (crossing
  boundaries) than words.

64
Questions

• Aslin, Saffran, Newport (1998)
• Stimuli were constructed specifically not to have
  information from prosody, pauses, or
  phonotactics. These would provide more sources of
  constraint in real inputs.

65
The End