Generic%20dialogue%20modeling%20for%20multi-application%20dialogue%20systems

1
Generic dialogue modeling formulti-application
dialogue systems

• Trung H. Bui
• Job Zwiers
• Anton Nijholt
• Mannes Poel

2
Outline

• Introduction
• 3-step approach
• Conclusions
• Future work

3
Introduction Motivation

• Example (extracted from the ICIS scenario)
• S1 You have found the route to the tunnel.
  What else do you want?
• U2 Umm, I want to have some information about
  a patient.
• S3(App.-specific) Sorry, the system only
  supports route navigation.
• S3 (Multi-application) Seeking for the
  patient information. What is the patients name?
• Multi-application dialogue system
• A dialogue system which allows transparent
  switching between a set of applications

A1 Route navigation
A2 Patient information search
4
Introduction Goal
Generic Dialogue Modeling for the efficient
production of interfaces for multi-application
dialogue systems
5
3-step approach

1. Producing finalized dialogue models for each
   application using the Rapid Dialogue Prototyping
   Methodology (RDPM)
2. Designing an application interaction hierarchy to
   connect the applications
3. Navigating between the applications

6
Step 1 Producing finalized dialogue models using
the RDPM

• Main idea of the RDPM Rapid design and
  production of a deployable frame-based dialogue
  model and its improvement through an iterative
  process
• Producing finalized dialogue model
• 1. Produce a task model for the targeted
  application
• 2. Derive an initial dialogue model and the
  associated multimodal dialogue-driven interface
  from the produced task model
• 3. Carry out Wizard-of-Oz experiments to improve
  the dialogue model
• 4. Carry out an Internal Field-test to further
  refine the dialogue model
• 5. Carry out an External Field-test to evaluate
  the final dialogue model

7
RDPM Task modeling

• The task is described in the form of a set of
  relational tables (frames), where the rows
  correspond to the possible functions (also called
  solutions or targets) and the columns are the
  attributes needed to uniquely identify each of
  the functions and to invoke it
• Example

Patient information search
select_patient(Name, Date of birth, Address, )
Patient ID Name Date of birth Address
001 X1 1/1/2005 Saturnusstraat 33
002 X2 15/2/1980 Drienerlolan 5
003 X3 11/3/1930 Emmastraat 10
... ... ... ... ...
8
RDPM Finalized dialogue model

• Local Interaction
• Help
• Repeat
• NoInput
• NoMatch

• Global Dialogue Strategies
• Branching Logic
• Name and Date of birth ? Address
• Dead-end
• No patient found
• Confirmation
• Tue, 10/2, 10.2 ? 10/2/2004
• Termination
• Result lt 5 patients
• Incoherencies
• Postcode conflicts with Address

multimodal Generic Dialogue Node (mGDN)
9
RDPM System architecture
(solution table, mGDN config)
10
Step 2 Designing an application Interaction
Hierarchy

• Goal
• Connecting the different applications
• Related work
• Call-routing dialogue system using vector space
  model techniques (Carroll Carpenter 1999)
• Application-independent Knowledge processing
  using ontology descriptions (Vrugt Portele
  2004)
• Our approach
• Vector space model
• Hierarchical clustering

11
Vector space model

• Finalized dialogue model M1,M2,,Mn ? Textual
  description di,d2,,dn
• Textual sources from solution table, mGDN
  config, NLU mapping,
• Produce index terms t1,t2,,tk from the textual
  descriptions
• NLP pre-processing
• Construct occurrence matrix F
• Lexical profile representation
• F n x k matrix
• Compute the score
• E.g. Cosine similarity

Finalized dialogue model
Textual description
Index terms
Occurrence Matrix
Description similarity
12
Hierarchical clustering algorithm

1. Consider each di as single cluster
2. Find the most similar pair of clusters
3. Compute distances between the new cluster and
   each of the old clusters
4. Repeat steps 2 and 3 until all items are
   clustered into a single cluster, size n.
5. Transform to the application interaction hierarchy

13
Step 3 Navigating between applications based on
users application of interest

• Start
• Active node determination
• Score computation
• Upward propagation
• Downward traversal to determine the active node
• Response generation
• Case 1 Active node is the root or an internal
  node
• Case 2 Active node is a leaf

14
Active node determination (1/2)
Score computation and upward propagation
0.9
M0-9
Users query
0.9
0.4
M0-4
M5-9
0.15
0.4
0.9
0.7
M5-7
M8-9
M0-2
M3-4
0.3
0.25
0.1
0.15
0.7
0.5
0.9
0.85
0.8
0.4
M1
M2
M3
M4
M5
M7
M9
M0
M8
M6
Threshold 0.15
15
Active node determination (2/2)
Downward traversal to determine the active node

1. Mactive root
2. Compute the difference between two highest score
   child nodes of Mactive ? Mi gt Mj
3. If (Mi Mj) lt ts then STOP, else Mactive Mi,
   and return to ii.

16
Response Generation

• Case 1 Active node is a non-leaf
• Application selection process
• List processing commands next, previous, stop,
  up, down, select
• Case 2 Active node is a leaf
• The application takes control and interacts with
  the user as an application-specific dialogue
  system
• If the users request is out of the applications
  domain ? go back to Start

17
Complete example
Start
M0-9
S1 What can I do for you? U2 Give me the
direction to the tunnel. S3 Please select your
application from the list (1) car route
navigation, (2) air route navigation, (3) traffic
lane. U4 One. S5 First, you need to go from
Twente airport, Sk-1 What else do you
want? Uk Umm, I want to have some information
about a patient. Sk1 Seeking for the patient
information. What is the patients name?
1
k
k
2
2
3
M0-2
4
4
5
M2
M1
M0
M8
M9

k-1
Car route navigation
Patient information search
Air route navigation

k
k1
18
Conclusion

• A 3-step framework for the development of the
  interfaces for multi-application dialogue
  systems.
• RDPM toolkit is available and has been used in
  several research projects InfoVox, INSPIRE,
  IM2.MDM and is being extended for ICIS.
• Application independent dialogue strategies in
  RDPM can be re-used for the development of
  multi-application dialogue systems.

19
Future work

• Evaluation of the approach in the ICIS project
  through the integration of 10 applications (car
  route navigation, air route navigation, traffic
  lane, )
• Crossing-application
• concurrent applications and tasks
• Task selection
• application interaction hierarchy ? task
  interaction hierarchy
• task-sharing

20
Thank you!

• Questions?