Good Morning!

1
Good Morning!

• How many times have you tried to get ahold of
  someone and failed?
• Need to keep trying
• Need to find alternate ways
• How many times have you forgotten an important
  appointment?

2
With VERA you can

• Schedule wake-ups for yourself and others
• Schedule message delivery to yourself and others
• Schedule recurring calls
• Send urgent messages with return receipt
• Automatically try multiple contacts for a person
• Automatically try again later

3
Voice Enabled Reminding Agent

• Vera-OUT Ulas Bardak, Scott Judy, Vasco Pedro
• Vera-IN Tal Blum, Jeongwoo Ko, Ryosuke Miyata

4
Overview

• Overall Architecture
• VeraOUT Architecture
• Skyper
• Dialog Manager Internals
• Web interface
• Demo
• Pros and Cons
• Future Work

5
Overall Vera Architecture
PSTN
Vera-IN
Vera-IN by Tal Blum, Jeongwoo Ko, and Ryosuke
Miyata
Skype
Skype Switching Center
DB
Web Interface
Internet
VoIP Contacts
Vera-OUT
Skype
Vera-OUT by Ulas Bardak, Scott Judy, and Vasco
Pedro
6
Vera Server Audio
Cepstral Speech Synthesizer
Monitor Speakers for Synthesized Speech
Internet
Sound Card 1 (Windows Default Audio)
Sound Card 2 (Skype Audio)
VoIP
Skype Software
Out
Mic
Out
Mic
Sphinx Speech Recongizer
Monitor Speakers for User Speech
7
VeraOUT Building Blocks
Vera-OUT
Web Interface
DM System
VeraTM
VeraDM
Skyper
Other Components
VoIP
DB
Vera-IN
Backend
8
VeraOUT Building Blocks
9
Dialog Management Internals
Lang. Understand. PHOENIX/HELIOS
Dialog Manag. RavenClaw
Vera Backend
Lang. Generation ROSETTA
From Dan Bohus talk on Ravenclaw
10
Dialog Management Internals
Lang. Understand. PHOENIX/HELIOS
Recognition SPHINX
Dialog Manag. RavenClaw
Vera Backend
HUB
Lang. Generation ROSETTA
Synthesis THETA
From Dan Bohus talk on Ravenclaw
11
Dialog Management Internals
Multiple, parallel decoders
SPHINX
Recognition Server
Dialog Manag. RavenClaw
Vera Backend
HUB
Lang. Generation ROSETTA
Synthesis THETA
PROCESSMONITOR
From Dan Bohus talk on Ravenclaw
12
Vera-OUT Architecture
13
Database
14
Vera-OUT Building Blocks
15
Web Interface
16
VERA-OUT Demo
We showed a demo at this point in the
presentation. Here are some recorded sample
conversations
17
RavenClaw Agent Tree
AVeraOut
AWrongUser
EGetContext
AGetUser
RWaitForSpeech
RLogout
ARightUser
ESetCompleted
ADoTask
AIsUser
RSpeakWithUser
RWaitForUser
ICallLater
ECallNext
ECheckTaskID
RgetUserString
IDeliverMessage
IWakeUpSystem
IMessageDelivered
IMessageNotDelivered
EsetSuspend
IWakeUp
EsetRightUser
IInformMessage
AmessageHandling
EsetWrongUser
IPlayMessage
ESetMessagePlayed
EsetNotHere
RgetTaskDoneString
IRepeatMessage
18
VERA-Out User Tests

• A user pool of around 10 fully cooperating users
• Used a relatively simpler dialogue
• Problems
• Non-native speakers have a lot of trouble
• Background noise and booming on the phone can
  affect the system
• Speakers get very confused when the system
  repeats itself with no clarification.

19
Vera-Out User Test Outcome

• Increased the dialog cooperation
• Clarification on whats being asked for
• Stating what is expected
• Added more coverage for speech recognition
  grammar.
• Added more coverage for dialog manager.

20
VERA-OUT Advantages

• Skype is inxexpensive
• Skype has worldwide reach.
• No phone line or special equipment required
  (except for an extra sound card).
• Privacy is preserved
• No numbers are shared with other users.
• No calls made to a number once the user removes
  it.
• Ability to call someone at a time when you may
  not be free yourself.
• Persistence
• Ability to confirm delivery of messages.
• Support for recurring tasks

21
VERA-OUT Issues

• Skype does cost money when phones are called.
  Whats the best way to do accounting / charge the
  right user?
• Skype calls show up as unknown on caller ID.
• Being socially appropriate
• Wheres the line between trying hard enough to
  make contact and harassment?
• How late/early should VERA call? (time zones?)
• Can others override this in an emergency?
• Who decides what an emergency is?

22
VERA-OUT Issues (ctd.)

• Should users be able to cancel all the calls that
  others have set up for them?
• Handling failure to make contact.
• Calling other contacts.
• Calling back later (after 1 hour)
• More work needed.
• Answering machines
• How to handle someone whos not expecting a call
  from a computer

23
VERA-OUT Future Work

• Better dialog to confirm user identity
• Blocked users / Allowed users
• Calls to multiple users
• Getting user contacts from the Skype directory
• Ability to forward and reply to messages.

24
VERA-OUT Future Work (ctd.)

• Setting up meetings with multiple participants.
• Call VERA-IN or use the web interface to specify
  available times and target meeting participants.
• VERA-OUT calls everyone and finds common times.
• VERA-OUT gets final approval from the initiator.
• VERA-OUT calls Roomline to book a room????
• VERA-OUT informs everyone of the result.

25
VERA-OUT Conclusion

• With Vera, reach the people you need to reach,
  when you want to reach them!

26
Special Thanks!

• Our professors
• Alan Black
• Alex Rudnicky
• RavenClaw experts
• Dan Bohus
• Jahanzeb Sherwani
• Antoine Raux
• Our longsuffering office-mates
• Yee-Man (Betty) Cheng
• Yanjun Qi

27
VERA-OUT

• The End