1
Speech-Smile, Speech-Laugh, Laugherand
their sequencing in dialogic interaction

• Klaus J. Kohler
• IPDS, Kiel, Germany

Satellite Workshop The Phonetics of Laughter,
ICPhS Saarbrücken, 5 August, 2007
2
1 Types of laughing

• free laughter
• complex breath control esssential element
• air stream direction
• energy time course
• modified in fairly simple ways (supra)glottally
• different from fairly simple subglottal and
  complex (supra)glottal settings in speech
• speech-laugh attributes of free laughter
  superimposed on speech coordination of two
  types of breath control

3

• speech-smile
• categorically different from laughter and speech
  laugh
• breath control of speech
• super-imposed long-term attributes that are
  related to smiling, physically and functionally
• lip spreading
• tongue fronting
• pitch raising
• may be preceded by facial expression of smile
  without speech or paralinguistic vocalization

4
2 Functions of laughing

• detailed interpretation of laughing depends on
  contextual settings of speaker interactions
• general feature of
• free laughter and speech-laugh seems to be the
  expression of amusement, high-key hilarity
• spech-smile a signal of happiness, low-key joy

5
Database and method

• two spontaneous data sources
• stereo recording of dialogue session consisting
  of 6 sub-dialogues between two secretaries in the
  Appointment-making Scenario with overlap f061
• stereo recording of two male speakers in the
  Video Task Scenario LINDENSTRASSE l061
• discussing differences in video clips from the
  German TV series
• presented to them separately beforehand
• speakers knew each other well, high degree of
  spontaneity and naturalness

6

• in f061, jm and mg arrange two 2-day business
  meetings in a 2-month period
• cannot find mutually suitable dates
• not enough successive daily slots in their
  calendars
• mg suggests 2 meetings immediately following each
  other, turning them into a 4-day meeting
• jm considers it possible but not appropriate
• finds clash between task and solution funny
• speech-laugh, followed by subdued laughter

7

• mg comments with speech-laugh
• points out that this concern is not important in
  such an appoinment-making game
• then turns into laughter
• picked up by jm
• followed by several exchanges of laughter between
  the 2 speakers

Dialogue ex. 1
8

• in l061, speaker mpi sets theme of the utter
  stupidity of the German TV soap series
  LINDENSTRASSE
• caps this with hilarious laughter
• whole dialogue revolves round this theme between
  speakers mpi and tra
• leads to several exchanges of laughter

Dialogue ex. 2
Dialogue ex. 3
9

• phonetic and interactional data described in
  detail
• through auditory assessment
• visual inspection of acoustic records
• spectrograms
• f0 and energy traces
• functional interpretation
• for full analysis of speech-smiles
• video recordings necessary
• not in the database
• deduction from sound patterns

10
4 Phonetic exponents of speech-smile

• The following sequence of speech-laugh and
  laughter ends in a speech smile on the phrase ja,
  gut speaker mg.
• lip spreading/fronting and rising pitch of gut,
  compared with na gut from same speaker in a
  non-smiling context
• F1 468 Hz, F2 1071 Hz vs. F1 409 Hz, F2 936 Hz

Ex. 1.1 laugh smile
Ex. 1.2 smile vs. non-smile
11

• The following is a sequence of speech-smile on
  the phrase und so (followed by hesitations)
  turning into laughter speaker tra.
• lip spreading/fronting of und so, compared with
  Show from the same speaker in a non-smiling
  context
• F1 590Hz, F2 1160 Hz vs. F1 450 Hz, F2 840 Hz

Ex. 3.2 smile vs. non-smile
Dialogue Ex. 3.1 smile laugh
12
5 Phonetic exponents of loud laughter

• laugh preceding smile in Ex. 1, female speaker mg
• egressive and ingressive air streams, oral
• voice bursts, same duration, equi-distant, in
  voiceless breathing, half-open to close vowel
• energy control, equi-prominent, f0 grouping

Ex. 1.1 laugh ( smile)
13

• laugh in Ex. 2, male speaker mpi
• egressive air stream, oral
• voice bursts, in voiceless breathing, 2 short
  up-beat, 4 longer bursts, open vowel resonance
• energy control, 4 double-peaked energy bursts,
  strong weak trochee patterns

Ex. 2. 1 laugh
14

• laugh following smile in Ex. 3, male speaker tra
• egressive air stream, oral
• voice bursts, in voiceless breathing, less
  regular, half-close vowel resonance
• energy control, less rhythmic, f0 grouping

Ex. 3. 3 (smile) laugh
15
6 Phonetic exponents of restrained laughter

• laugh in Ex. 4, same speaker as in Ex. 2, mpi
• egressive air stream, nasal, restrained laughter
• 3 blocks of voice bursts, 4 short, 3 long, 3 very
  short, each equi-distant, in voiceless breathing,
  
• energy control, equi-prominent, f0 grouping

Ex. 4 laugh
16

• laugh in Ex. 1, after speech ne, female speaker
  jm
• egressive air stream, oral, subdued laughter
• 3 blocks of weak - strong rhythmic patterns,
  1 breath, 2 voice
• former by duration, latter by energy control
• sounds regular in spite of different distances

Ex. 1.3 speech laugh
Ex. 1.4 laugh
17

• laugh in Ex. 1, after speech-laugh on glücklich
  female speaker jm
• long ingressive short egressive air stream,
  oral, subdued laughter
• 2 strong weak rhythmic patterns,
• decreasing by energy control across the 2
• sounds very regular

Ex. 1.5 speech laugh
Ex. 1.6 laugh
18
7 Speech-laugh

• laugh in Ex. 1.5, female speaker jm
• ingressive oral breath, continues into acc. glück
• change to egressive strong energy increase in
  glück, phonation break
• ingressive egressive in lich, phonation break
• double-peaked energy control in the fricative,
  egressive, mouth opening

Ex. 1.5 laugh on nicht so glücklich
19

• laugh in Ex. 1.1, female speaker mg
• egressive oral air stream
• double-peaked energy control in ja, 2nd peak
  higher, vowel split in two
• followed by strong egressive breath, gradually
  trailing off over long period

Ex. 1.1 laugh on ja
20
8 Sequencing of laugh types

• any sequence is possible
• speech-laugh laughter speech-smile, mg
• speech-laugh subdued laugh, jm
• loud laughter speech smile, mpi
• speech-smile laughter, tra
• laughter sets in after mips beginning of the
  speech-smile
• speech-smile laughter, jm

21
9 Laughing interaction in dialogue

• laughing phenomena sequenced and timed for
  communicative functions
• within one speaker
• between speakers
• f061 coordinated action between the two speakers
• part of their joint task-solving in the
  Appointment-making game
• in l061 the two speakers talk about differences
  they have seen in video clips
• mutually trigger laughter by facetious
  descriptions of the soap opera excerpts

22
10 Conclusion

• kind of Conversation Analysis investigation of
  sampled instances of spontaneous occurrences of
  laughing phenomena
• small database suggests that fine phonetic detail
  is highly structured acoustically and
  functionally, including rhythmical factors

23

• extension of the auditory and acoustico-graphic
  investigation to broad databases from spontaneous
  speech of various scenarios, considering
• speaker types
• and different languages
• detailed feature description is prerequisite to
  large-scale acoustic analysis and introduction
  into speech technology