Issues in Spoken Dialogue Systems

1
Issues in Spoken Dialogue Systems

• Julia Hirschberg
• LSA07 353

2
Why Spoken Dialogue Systems?

• Science
• A chance to study and try to emulate human
  behavior in spoken communication
• Technology
• Demonstrate utility of several speech technolgies
• A means to save human labor and permit flexible
  management of simple tasks
• Online banking services
• Reservations (air, train, bus, rental car)
• Information access

3
The Instructor

• Julia Hirschberg
• Columbia Speech Lab

4
Course Overview

• Discussion of past and current SDS, with examples
  of interactions
• Major problems in building SDS
• Problems in understanding and modeling human
  conversational behaviors
• Limits of the available technologies
• Problems in evaluating SDS
• SDS of the future

5
Some Early SDS Projects at KTH

• The Waxholm Project
• Kiosk provides information on travel in the
  Stockholm archipelago
• August
• Kiosk provides information on August Strindberg
• Other demos at http//www.speech.kth.se/multimodal
  

6
The Waxholm system
Where can I find hotels?
Information about hotels is shown in this table
Which hotels are in Waxholm?
Information about the hotels in Waxholm is shown
in this table
Where is Waxholm?
Waxholm is shown on this map
I think I want to go to Waxholm
I am looking for boats to Waxholm From where do
you want to go
The city
Which day of the week do you want to go?
I want to go tomorrow
There are lots of boats from Stockholm to Waxholm
on a Friday, At what time do you want to go?
When do the evening boats depart?
This is a table of the boats...
Is it possible to eat in Waxholm?
Information about the restaurants in Waxholm is
shown in this table
Thank you
Thank you too
7
Issues

• Limited domain the more so the better
• Use of graphical display (maps, tables, charts)
  to present complex information and enhance visual
  appeal
• Types of initiative in SDS
• System
• User
• Mixed
• Importance of lexical priming

8
The August system
What is your name?
I call myself Strindberg, but I dont really
have a surname
When were you born?
Strindberg was born in 1849
What do you do for a living?
I can answer questions about Strindberg, KTH and
Stockholm
How many people live in Stockholm?
Over a million people live in the Stockholm area
Do you like it here?
People who live in glass houses should not throw
stones
Yes, that was a smart thing to say!
I come from the department of Speech, Music and
Hearing
The Royal Institute of Technology!
The information is shown on the map
Thank you!
You are welcome!
Good bye!
Perhaps we will meet soon again!
Yes, it might be that we will!
Strindberg was married three times!
Strindberg was married three times!
9
Issues

• More open domain entertaining vs. accomplishing
  a real task
• Use of multimodal (speech and face) synthesis
• Gesture (face)
• System personality
• A good tool for data collection
• How to evaluate?

10
More Recent ATT Experimental Work

• Toot system created to study differences in types
  of initiative and confirmation strategies and
  identify user preferences 98-99
• Some successful dialogues
• Some not so successful

11
Issues

• Automatic Speech Recognition (ASR) errors
• Importance of age, gender, accent
• Text-to-Speech (TTS) issues
• Error detection and correction
• Hyperarticulation
• Confirmation strategies
• Explicit
• Implicit
• None
• Managing turn-taking

12

• Evaluation again.how do we know when a change
  has been beneficial?

13
A Real SDS System from the Lab

• Lets Go from CMU
• What type of initiative supported?
• What type of confirmation stragegy?
• Difficult or easy to evaluate?
• Compare to the ATT system? The KTH?
• How natural is the interaction?
• What are they doing to enhance, over the other
  systems?

14
The Course

• Classes, readings, and assignments
• Modeling Turn-taking Behaviors
• Components of SDS
• Automatic Speech Recognition (ASR)
• Text-to-Speech (TTS)
• Natural Language Understanding (NLU)
• Building and Evaluating SDS
• Interpreting and Generating Dialogue Acts
• Error Detection and Repair Strategies
• Making SDS More Human Entrainment/Adaptation
• Can systems become more like their users?
• Should they?

15
The Course Paper

• Eliza is a classic text-based AI Dialogue System
  which often fools casual users into thinking they
  are conversing with a human being. For this
  assignment
• Become acquainted with the text Eliza
• Identifying 5 or more strategies you can use to
  demonstrate that Eliza is not a human
  conversationalist (e.g. linguistic constructions
  she does not handle well, pragmatic behaviors she
  does not generate appropriately or recognize).
  As evidence describe your inputs and her outputs
• Next, design a speech-enabled version of Eliza
• Explain the difficulties you will have to
  overcome in recognition, generation, and dialogue
  management, compared to the text version, based
  upon the topics we have studied in class and
  additional observations you may have
• Suggest specific ways of dealing with these
  problems based upon what you know of the state of
  the art in SDS or ideas you may have to improve
  upon it
• Indicate the ways in which a spoken version of
  Eliza might be even better than the text version,
  in terms of how it might interact with the user
  and the features of spoken language you might be
  able to make use of in generation and recognition
  of user behavior

16
SDS and Popular Culture SNL

• Amtraks Julie
• Next class JM 22.1, Clark 03, Beattie 82