Conclusions

1
Influences of Visible Place Versus Manner
Distinctions on Perception of Audio-Visual
English CV Syllables Catherine T. Best Daniel
Lazarek c.best_at_uws.edu.au
AVSP05
Précis In audio-visual (AV) speech perception the
two modalities convey largely complementary
information (V Place, A Manner). But place can
be low visibility, and manner visible.
Articulatory Phonology and ecological/direct
realist views imply that examining visible vs.
audible gestural structure may offer novel
insights.Perceptual effects of active articulator
vs. constriction degree were examined in a
McGurk task using anterior consonants that differ
visibly on both dimensions. Visual impact was
greatest for incongruent A-V signals that used
different articulators but same constriction
type, stronger for fricatives than stops/glides,
yet failed to yield an articulator effect. Thus,
constriction affects AV perception, more so than
active articulator, in identification of visually
distinct anterior consonants.

• Results contd Experiment 1
• Constriction Type main effect, p .0001
• Visual influence on perception was greater for
  fricatives than stops
• Gestural Incongruity x Constriction, p lt .0001
• /v/-/D/ pairs showed the strongest visual effect,
  followed by video stop paired with
  opposite-articulator fricative
• Articulator x Constriction, p lt .017
• Both fricatives had strong visual effects, but
  labial stop gt lingual stop

• Results contd Experiment 2
• Gestural Incongruity x Articulator, p lt .053
• marginal largest visual effects for /v/-/D/
  /b/-/d/ pairs and video fricative audio
  stop/glide yielded next largest visual effect
• Gestural Incongruity x Constriction, p lt .0001
• Replication/extension of Exp. 1 interaction
  effect. /v/-/D/ showed strongest effect by far.
  Video glides with opposite-articulator
  stop/fricative was next-strongest.

Research Question How do visible distinctions in
active articulator and constriction degree
contribute to AV speech perception?

• Method
• Stimuli anterior Cs (USA English) that differ
  visibly re
• Active Articulator
• lower lip
• tongue tip/blade
• Constriction
• closed (stop)
• critical (fricative)
• narrow (glide) (included only in Exp. 2)
• Subjects English (USA) Exp 1 (n 14), Exp 2 (n
  12)
• Task report C heard AV-congruent
  AV-incongruent
• Data Visual Speech Index (VSI), calculated on
  proportion correct audio identifications
• VSI AVcongruent - AVincongruent

• Background
• Audio-visual (AV) speech perception shows
  modality-specific contributions (MacDonald
  McGurk, 1978 VPAM Summerfield, 1987)
• Audio provides primarily manner information
• Visual provides place of articulation information
• Yet, some qualifications re those assumptions
• place and manner imperfectly related to
  visibility
• place (POA) visibility varies
• labials vs. non-labials
• also some visibility for some coronals
• face dynamics re other POA info (below)
• manner also varies stops - fricatives - glides
• unclear how narrowly to define POA, e.g. /b v/
• SAME labial ( broad transcripttion)
• DIFFERENT labiodental vs. bilabial (narrow
  transcription)
• dynamic visual speech information is distributed
  across the talking face/head (Yehia et al., 1998)
• correlates with tongue as well as lip and jaw
  movements
• this info can guide intelligible audio synthesis
• Articulatory Phonology (Browman Goldstein,
  1992, 2000) suggests an alternative A-V
  perception re articulatory gestures (cf Fowler
  Dekle, 1991)
• Active articulator lower lip vs. tongue
  tip/blade

• Results Experiment 1
• Gestural Incongruity Type main effect, p lt .0001
• Visual influence was strongest when A and V
  tokens differed in Articulator but shared the
  same Constriction degree
• Gestural Incongruity x Articulator, p .0084
• The preceding effect was more pronounced when the
  video token used lips than tongue tip

• Conclusions
• Constriction Type does influence AV speech
  perception when it is visibly distinct
• Constriction is more effective than Articulator
  in this stimulus context
• critical constriction degree (fricatives) shows
  the strongest visual influence
• Active Articulator had little visual effect
• labials did not have greater effect than linguals
• However, passive articulator differences may
  account for the strong /v/-/D/ effects
• Articulatory Phonology implications for AV speech
  perception/production research
• gestural parameters may offer better (or
  additional) guidance than phonetic features