James

1
James

• A Personal Mobile Universal Speech Interface for
  Electronic Devices

2
Current Speech Application Concept
Phone Client
PDA Client
Computer Client
Speech Application
Backend
3
Current Electronic Devices
???
???
???
Speech Application
Backend
4
???Questions???

• History Why is there a conceptual gap?
• Motivation Is speech a useful modality for
  other electronic devices?
• Hardware How would one get speech in other
  devices?
• Architecture What should the system look like?
• Dialog What should/will these conversations be
  like?

5
History

• Why is there a conceptual gap?
• Speech is still hard.
• That will change.

6
Motivation

• Is speech a useful modality for other
  electronic devices?
• It seems probable.
• There has been some positive research (see
  Microsoft)
• Ideas?

7
Hardware

• How would one get speech in other devices?
• No need to as long as devices are remote
  controlled via a known interface.
• Refer to system architecture.

8
Architecture
Havi adapter
Mobile Speech Client
X10 adapter
9
Dialog

• USI Model
• Artificial subset language
• Tree-structured functions
• Universal primitives
• User-directed
• Great for recognition
• Entirely declarative (automatic)

10
(No Transcript)
11
Keywords

• hello-james
• options
• where-am-i, where-was-i
• go-ahead, ok
• status
• goodbye
• what-is, what-is-the
• how-do-i
• more

12
Session Management

• hello-james/goodbye
• User blah blah blah...System ignoring
  userUser hello-jamesSystem stereo, digital
  cameraUser stereoSystem stereo hereUser
  goodbyeSystem goodbyeUser blah blah
  blah...System ignoring user

13
Query

• what-is path/status
• User what-is-the am frequencySystem the am
  frequency is five hundred thirtyUser what-is
  randomSystem random is offUser what-is-the
  stereoSystem the stereo is tuner

14
help/exploration/implicit navigation

• how-do-i.../options/path options
• User control alarm clock radio optionsSystem
  alarm, clock, radio, sleep...User moreSystem
  x10, stereoUser stereo optionsSystem while
  turning stereo on off, am, fm, auxiliary,
  cd...

15
invocation/specification/implicit
exploration/navigation

• Path
• User stereo auxiliarySystem while turning the
  stereo on and switching to auxiliary
  auxiliaryUser cdSystem while switching to cd
  mode cdUser playSystem while playing a cd
  play

16
list navigation

• More
• User radio band am optionsSystem bracketed
  list frequency, kabc, k001, k002, k003, k004,
  k005fmoff, volumealarm, clock, sleepx10,
  stereo rendered frequency, kabc, k001,
  k002...User moreSystem k003, k004,
  k005...User moreSystem fm, off, volume...

17
orientation

• where-am-i
• User what is the discSystem the disc is
  threeUser where am iSystem stereo cd disc

18
Research Questions

• Is the subset language learnable?
• Once learned, is it efficient?
• Are user mistakes infrequent enough?
• Are system mistake infrequent enough?
• Can one generalize from one device to another?
• Is the subset language well retained?