Engineering Dialog for Gadgets

1
Engineering Dialog for Gadgets

• Thomas K Harris
• September 12, 2003

2
???Questions???

• Motivation Is speech a useful modality for
  electronic devices?
• Hardware How would one get speech in other
  devices?
• Architecture What should the system look like?
• Dialog What should/will these conversations be
  like?

3
Current Speech Application Concept
Phone Client
PDA Client
Computer Client
Speech Application
Backend
4
Current Electronic Devices
???
???
???
Speech Application
Frontend
5
Protocol-based Architecture
Havi adapter
Speech Graffiti Personal Universal Controller
X10 adapter
6
(No Transcript)
7
Speech Graffiti Dialog

• Artificial subset language
• Tree-structured functions
• Universal conversational primitives
• User-directed
• Great for recognition
• Entirely declarative (and automatic)

8
Minimal Keywords

• hello-james
• options
• where-am-i, where-was-i
• go-ahead, ok
• status
• goodbye
• what-is, what-is-the
• how-do-i
• more

9
Session Management

• hello-james/goodbye
• User blah blah blah...System ignoring
  userUser hello-jamesSystem stereo, digital
  cameraUser stereoSystem stereo hereUser
  goodbyeSystem goodbyeUser blah blah
  blah...System ignoring user

10
Query

• what-is path/status
• User what-is-the am frequencySystem the am
  frequency is five hundred thirtyUser what-is
  randomSystem random is offUser what-is-the
  stereoSystem the stereo is tuner

11
help/exploration/implicit navigation

• how-do-i.../options/path options
• User control alarm clock radio optionsSystem
  alarm, clock, radio, sleep...User moreSystem
  x10, stereoUser stereo optionsSystem while
  turning stereo on off, am, fm, auxiliary,
  cd...

12
invocation/specification/implicit
exploration/navigation

• Path
• User stereo auxiliarySystem while turning the
  stereo on and switching to auxiliary
  auxiliaryUser cdSystem while switching to cd
  mode cdUser playSystem while playing a cd
  play

13
list navigation

• More
• User radio band am optionsSystem bracketed
  list frequency, kabc, k001, k002, k003, k004,
  k005fmoff, volumealarm, clock, sleepx10,
  stereo rendered frequency, kabc, k001,
  k002...User moreSystem k003, k004,
  k005...User moreSystem fm, off, volume...

14
orientation

• where-am-i
• User what is the discSystem the disc is
  threeUser where am iSystem stereo cd disc

15
Research Questions

• Is the subset language learnable?
• Once learned, is it efficient?
• Are user mistakes infrequent enough?
• Are system mistake infrequent enough?
• Can one generalize from one device to another?
• Is the subset language well retained?