Studying Intonation

1
Studying Intonation

• Julia Hirschberg
• CS 4706

2
Today

• Approaches to studying contour meaning
• Questions people ask
• Does contour X convey a different meaning from
  contour Y?
• Is contour X used more often in context Z than
  contour Y
• Despite what people say/think, not all phenomena
  X are uttered with contour Y
• What kind of evidence could we get?
• Found data
• Laboratory experiments production, perception
• Corpus collection

3

• What features can we look at and how do we obtain
  them?
• Intonation labeling by hand
• Acoustic/prosodic analysis by automatic methods
• Pitch tracking, pause detection, intensity,
  duration, speaking rate extraction
• Computational linguistic techniques to extract
  transcript-based (text) features
• Part-of-speech
• Sentence length,
• What techniques do we use for analysis?
• Statistical methods (Splus, Matlab)
• Machine learning techniques

4
Some Sample Approaches

• Natural Corpus Hedberg Sosa 2002
• Introspective, observational Wilson 1993,
  Pierrehumbert Hirschberg 1990/2
• Laboratory -- Production/Perception Syrdal
  Jilka 2004
• Laboratory Brain Imaging (e.g. fMRI) Doherty
  et al

5
A Prescriptive Approach Wilson 1993

• Declarative statements fall and yes-no-questions
  rise?
• Wh-questions fall?
• Small final rise signals more to come?

6
Corpus Studies of Questions Hedberg Sosa 2002

• How are yes-no and wh-questions uttered and how
  might we explain differences?
• Where is the nuclear stress?
• Where is the semantic focus? What is the
  topic?
• Are the wh words accented or not?
• Corpus 73 questions
• Who saw John?/Who didnt see John?
• Did John leave?/Didnt John leave?
• 35 whqs and 38 ynqs from the McLaughlin Group
  and Washington Week

7

• Analysis
• Intonational labeling (ToBI) from pitch tracks
• Topic/focus coding
• Frequency distributions of features with question
  categories
• Prosody of locus of interrogation
• Wh word in wh-questions
• Fronted auxiliary in yes-no questions
• Results
• Ynqs generally uttered w/ falling or level
  intonation, not rising (69)
• Wh-qs most often uttered with falling (80)

8

• Wh-words (60) in all wh-questions and neg aux in
  negative ynqs (89) most often uttered with LH
  accent (contrastive accent) -- why?
• Aux in positive ynqs often deaccented (41) or
  realized with L (17) accent why?
• Conclusions/open questions
• Why do ynqs and wh-qs sometimes rise and
  sometimes fall?
• Locus of interrogation is accented in wh-qs and
  in negative ynqs to signal interrogative status
  of sentence but not in positive ynqs due to
  need to highlight a following element

9
Critique

• Is this a good corpus for this investigation?
• Size
• Genre
• What about the speakers?

10
Syrdal Jilka 2004

• How are whqs and ynqs produced most naturally
  (for TTS)?
• Same initial hypothesis whqs fall and ynqs
  rise in American English
• Different approach production and perception
  studies
• Production
• 8 (professional) speakers (5F, 3M)
• Read transcripts of actual dialogues

11

• Analysis
• Intonational (ToBI) labeling from pitch tracks of
  extracted questions
• Results
• Ynqs rose in 83 of cases for females and 53
  for males
• Wh-qs always fell for females and fell 79 of
  time for male speakers wh-qs and statements
  generally fell
• Nuclear accents in ynqs majority L

12

• Perception studies acceptability judgments
• Forced choice, 12 listeners
• Stimuli Pairs of ynq and whqs with same
  voice/different intonation
• 17 natural (9 ynqs, 8 whqs)
• 12 synthesized
• 12 subjects (6 and 6)
• Judgments
• Ynq
• Natural speech people preferred standard rise
  (L H- H)
• Synthetic speech no results
• Whq
• Natural speech people preferred falling
  contours (L- L) to rising (H-H) and slightly to
  continuation rise (L- H)
• Synthetic no preference

13
Critique

• How many questions were produced?
• Are professional speakers a good choice?
• Read vs. spontaneous speech? For TTS?
• Why no results for synthetic speech?
• Comparison to Hedberg and Sosa

14
Doherty et al 2004

• How do people process intonation, e.g., in rising
  questions vs. falling statements vs. falling
  questions?
• She was talking to her father?
• She was talking to her father.
• Was she talking to her father.
• Research questions
• Where is the prosody portion of the brain?
• What other sectors is it close to and what is
  their function?
• Do particular contours have particular locations?

15

• Method functional Magnetic Resonance Imaging
  (fMRI) of subjects presented with digitized
  recordings
• 11 subjects (4M, 7F)
• Note experimental condition!
• 150 triples, of which each subjects heard only 1
  version
• She was talking to her father?
• Was she talking to her father.
• She was talking to her father.
• Monitoring task Is this a question or a
  statement?
• Press one key for question, another for statement

16

• Results Increase in activation when subjects
  made judgments about tokens w/ rising intonation
  -- but not falling, whether syntactic question or
  syntactic statement
• Why?
• Semantic processing? No illocutionary force is
  same in rising and falling questions
• Acoustic processing? Maybe
• Interpreting the rising contour as a question?
• Check lesion studies to see if people with damage
  in these areas can interpret rising contours

17
Critique

• No rising inverted questions? Was she talking
  to her father?

18
Next Class

• How do we represent intonational variation?