Spoken%20Dialogue%20Technology%20Achievements%20and%20Challenges

1
Spoken Dialogue TechnologyAchievements and
Challenges

• Michael McTear
• University of Ulster

2
Overview

• Introduction - What is a spoken dialogue system?
• Examples of spoken dialogue systems
• Technical issues and challenges
• Future Prospects

3
What is a spoken dialogue system?

• A spoken dialogue system is an automated system
  that engages in a dialogue with a human user
  using spoken language as the medium of
  interaction.

4
Types of dialogue system
Two main types of spoken dialogue system

• Task-oriented involves the use of dialogues to
  accomplish a task, e.g. making a hotel booking,
  or planning a family holiday

• Non-task-oriented engaging in conversational
  interaction, but without necessarily being
  involved in a task that needs to be accomplished
  e.g conversational companion for the elderly

5
Application Domains for SDS

• Telephone-based services and transactions
• Call-routing, Directory assistance, Travel
  enquiries, Bank balance, Bank transactions,
  Flight / hotel / car rental reservations
• In-car interactive and entertainment systems
• Automated trouble-shooting
• Smart homes applications
• Health-care systems e.g. patient monitoring
• Educational e,g. Intelligent Tutoring Systems,
  Foreign Language Learning
• Computer games

6
Three generations of task-oriented spoken
dialogue system

• Informational to retrieve information e.g.
  flight times, football scores,
• Transactional to assist the user to perform a
  transaction e.g. book a flight, pay a bill

7
Why is dialogue interesting?

• Fundamental aspect of human behaviour
• Model human conversational competence
• Simulate human conversational behaviour
• Provide tool for interacting with data, services,
  resources on computers
• Research challenges
• Applications in assistive and educational
  environments
• Commercial opportunities

8
Commercial Systems

• Focus on
• Business opportunities, return on investment
  (ROI)
• Benefits for end users
• Benefits for providers
• Human factors performance, usability
• Tools and languages for design and
  maintainability
• Application areas call centre, enquiries,
  transactions, healthcare,

9
Academic Systems

• Focus on
• Technologies speech recognition, spoken language
  understanding, dialogue management
• AI inspired planning, reasoning, machine
  learning
• Statistical v symbolic approaches
• Advanced dialogue control, error handling,
  adaptivity, context representation

10
Overview

• Introduction - What is a spoken dialogue system?
• Examples of spoken dialogue systems
• Technical issues and challenges
• Future Prospects

11
Example 1 Voice Menu

• System Hello and welcome .
• Main menu. For customer service, say service.
• To enquire about an existing order, say order
  
• User Service
• System Customer service. Would you like to
  report a fault or enquire about an extended
  warranty?
• User Fault
• System Do you have a PC or a laptop?
• User Laptop
• System And the name of the manufacturer?
• User Sony
• System Thank you. Please hold while I transfer
  you to the Sony

http//www.speechstorm.com/
12
Example 2 Research System (Mercury MIT)

• Open ended prompt
• How may I help you?
• Disfluencies in input
• August twenty-first no August twelfth
• I'd like to fly from Boston to Minneapolis on
  Tuesday no Wednesday November 21st
• Inexact response
• Prompt Can you provide the approximate departure
  time or airline preference
• User Yeah I'd like to fly United and I'd like to
  leave in the afternoon

http//groups.csail.mit.edu/sls/research/mercury.s
html
13
Example 2 continued

• Response generation
• There are more than 3 flights.
• The earliest departure leaves at 1.45 pm.
• Mixed initiative user asks question
• Do you have something leaving around 4.45?
• Relative date reference
• Id like to return the following Tuesday

14
Example 3 Voice Search GOOG411
GOOG-411 (or Google Voice Local Search) is
Google's new 411 service. With GOOG-411, you
can find local business information completely
free, directly from your phone. You can access
1-800-GOOG-411 from any phone, anywhere, at
anytime.
http//www.google.com/goog411/
15
GOOG411 Prompts

• What city and state?
• What business name or category?
• (Lists services) Number one, ..
• Connects to requested service

16
GOOG411 What can you say?

• At any point in the call 
• To go back say "go back"
• To start over say "start over" or press All
  phones
• When asked for a city and state 
• Say the full names for example, "Palo Alto
  California
• To enter a zip code say it or enter with keypad
• When asked for business name or category 
• Say the full names for example, "Joe's Pizzaria"
  or "Pizza
• When given results 
• To navigate between results say or press the
  listing number
• To receive an SMS say "text message"
• To receive a map say "map it"
• To get more details say "details"

17
Overview

• Introduction - What is a spoken dialogue system?
• Examples of spoken dialogue systems
• Technical issues and challenges
• Future Prospects

18
Architecture of a spoken dialogue system
19
Component Technologies

• Automatic Speech Recognition (ASR)
• Spoken Language Understanding (SLU)
• Response Generation (RG)
• Text to speech synthesis (TTS)
• Dialogue Management (DM)

20
Issues in ASR for Dialogue

• recognising spontaneous speech in noisy
  environments
• word accuracy does not have to be 100
• use of confidence scores in combination with
  other information to determine DM actions
• use of additional information (ASR and parse
  probabilities, semantic and contextual features)
  to re-score recognition hypotheses

21
Issues in SLU for Dialogue

• grammars and parsers for spontaneous speech
  (disfluencies, errors)
• robust understanding
• problems with hand-crafted approaches
• use of statistical/ data-driven methods
• combined approaches e.g TINA (MIT)
• hand-crafted rules with trained probabilities
• robust strategy if full sentence cannot be
  parsed, parse and combine fragments, else use
  word spotting

22
Issues in Response Generation for Dialogue

• Content selection
• Determining what to say, selecting and ranking
  options
• Discourse planning
• discourse relations e.g. comparison, contrast
• user-adapted information
• Presentation ordering
• Referring expression generation
• Aggregation grouping propositions into clauses
  and sentences
• Use of discourse cues (e.g. firstly, finally,
  however, moreover, )

23
Issues in Dialogue Management

• Dialogue Control
• Scripts, frames, intelligent agents
• Representations
• Information State Theory
• Error handling
• Dialogue design
• Traditional approaches
• Statistical approaches
• Reinforcement learning
• Corpus / example based approaches

24
Overview

• Introduction - What is a spoken dialogue system?
• Examples of spoken dialogue systems
• Technical issues and challenges
• Future Prospects

25
A vision for the future

• Develop systems that can interact intelligently
  and co-operatively across a range of environments
  using a range of appropriate modalities to
  support people in the activities of their daily
  lives.

26
Fundamental research topics

• Modelling human conversational competence
• Dialogue-related issues for ASR, SLU, NLG, TTS
• Comparison of methods for dialogue management
  rule-based v stochastic
• Representation and use of contextual information
• Integration and usage of modalities to complement
  and supplement speech
• Incremental processing in dialogue

27
Areas of application

• Voice search
• Dialogue in vehicles
• Mobile speech applications
• Multimodal embodied and situated systems
• Troubleshooting applications
• Dialogue systems for ambient intelligence and as
  assistive technologies

28
Concluding remarks

• Spoken Dialogue Technology
• embraces a range of speech and language
  technologies
• poses lots of theoretical as well as practical
  challenges
• is interesting for commercial developers as well
  as academic researchers
• has a wide range of potential applications

29
Recommended reading

• McTear, M. (2004) Spoken Dialogue Technology.
  Springer.
• Lopez Cozar, R. Araki, M. (2005) Spoken,
  multilingual and multimodal dialogue systems.
  John Wiley Sons.
• Aghajan, H., Augusto, J.C., Lopez Cozar, R.
  (2009) Human-Centric Interfaces for Ambient
  Intelligence. Elsevier.
• Jokinen, K. McTear, M. (2010) Spoken Dialogue
  Systems. Morgan Claypool Publishers.
• Wilks, Y. (ed.) (2010) Close Engagements with
  Artificial Companions Key social, psychological,
  ethical and design issues. John Benjamins
  Publishing Company.

30
Thank you

• Questions?