When Voice and Content Dont Match'''

1

• When Voice and Content Dont Match...
• Gabriel Adauto
• Ulla Foehr
• Megan Huddleston
• Michael Somoza

2
Key Questions

• Is emotion attributed to synthetic voices?
• How does voice emotion affect
• perceptions of the content?
• perceived suitability of the content?
• How does the match/mismatch between voice
  characteristics and content affect
• liking of the information?
• credibility of the information?

3
Importance of the Questions

• Psychological Issues
• Good/bad is a fundamental psychological category
• Media Equation If human voice transmits emotion,
  so does synthetic voice
• Can one simulate emotion?
• Consistency expectations is a fundamental
  psychological principle

4
Importance of the Questions

• Design Issues
• Usually one voice, regardless of content
• Detection of emotion in text content is unsolved
  problem
• How important is this?

5
Context

• Telephone system
• Varied Topics
• News
• Movie Descriptions
• Health Information
• Happy and sad content in each category
• Web Questionnaire

6
Procedure

• 1. Participants recruited by email and directed
  to a website
• 2. Participant goes to a website for instructions
• 3. Participant calls the experiment and enters a
  group code to hear content
• 4. After hearing each story, the participant
  answers questions on the Web

7
Manipulations

• TTS vs. Recorded (between)
• Match/mismatch of voice emotion and content
  (between)
• Two stories per topic (within)

8
Demo of the Stimulus
TTS Voice Sad News Another dead gray whale
washed ashore in San Francisco Bay yesterday,
deepening a mystery that is baffling scientists
about the recent spate of deaths.
Recorded Voice Happy Health For the first time,
scientists say gene therapy has unequivocally
succeeded. Doctors used the treatment to save the
lives of several infants who have a severe immune
disorder.
9
Interest in Topics
Extroverts more interested
Introverts more interested
10
Perceived Content Valence
11
Liking of the Content
12
Credibility of the Content
13
Psychological Implications

• Even synthetic speech can manifest emotion
• Extroverts will like it more
• Emotion of voice influences perception of content
• Even in TTS!

14
Psychological Implications

• Consistency has important psychological
  consequences
• Great liking
• Less credibility
• Liking/credibility tradeoff

15
Design Implications

• Decisions of whether to match or mismatch voice
  and content is crucial
• Emotion is important even for non-emotional
  machines
• Voices should be cast based on
• Content
• Target audience
• Products on site (party items vs. funeral home)
• All modalities should be consistent
• Text
• Voice
• Facial expression

16
Open Questions

• What other voice characteristics must be
  consistent with content?
• Gender
• Age
• Arousal
• Do synthetic faces manifest emotion? What about
  synthetic bodies?