ITCS 6010

1
ITCS 6010
SALT
2
SALT

• Speech Application Language Tags (SALT)
• Speech interface markup language
• Extension of HTML and other markup languages
• Adds speech and telephony features to Web
  applications and services for both voice only and
  multimodal browsers

3
SALT Overview

• SALT
• Small set of XML elements
• Elements have
• Attributes
• DOM (Document Object Model) object properties
• Events
• Methods
• Applies speech to source page when used in
  conjunction with source markup document

4
SALT Design Principles

• Clean integration of speech with Web pages
• Leverages event-based DOM execution model of Web
  pages
• Integrates cleanly into visual markup pages
• Reuses knowledge and skill of Web developers
• Does not reinvent page execution or programming
  models

5
SALT Design Principles (contd)

• Separation of speech interface from business
  logic and data
• Individual markup language not directly extended
• Provides separate layer extensible across
  different markup languages
• Allows for loose or tight coupling of speech
  interface to underlying data structure
• Enables reuse of speech and dialog components
  across pages and applications

6
SALT Design Principles (contd)

• Power and flexibility of programming model
• SALT elements are simple and intuitive
• Offer fine-level control of dialog execution
  through DOM event and scripting model
• Leverages benefits of rich and well-understood
  execution environment

7
SALT Design Principles (contd)

• Reuses existing standards for grammar, speech
  output and semantic results
• Range of devices
• Designed for range of architectural scenarios
• Not for particular device type

8
SALT Design Principles (contd)

• Minimal cost of authoring across modes and
  devices
• Enables 2 important classes of application
  scenario
• Multimodal
• Visual page enhanced with speech interface on
  same device
• Cross-modal
• Single application page reused for different
  modes on different devices

9
Top-level Elements

• There are 4 main top-level elements
• ltprompt gt
• For speech synthesis and prompt playing
• ltlisten gt
• For speech recognition
• ltdtmf gt
• For configuration and control of DTMF collection
• ltsmex gt
• For general purpose communication with platform
  components

10
Top-level Elements

• listen and DTMF elements
• May contain ltgrammargt and ltbindgt elements
• listen element
• May contain ltrecordgt element

11
ltlistengt Element

• Used for speech input
• Specifies grammars
• Specifies means of dealing with speech
  recognition results
• Used for recording spoken input
• Handles speech events and configures recognizer
  properties
• Activates/deactivates grammars
• Starts/stops recognition

12
ltlistengt Element (contd)

• ltlistengt example
• ltsaltlisten idtravelgt
• ltsaltgrammar src./city.xml /gt
• ltsaltbind targetElementtxtBoxOriginCity
• value/result/origin_ci
  ty /gt
• lt/saltlistengt

13
ltlistengt Element (contd)

• ltlistengt element
• Can be executed with Start() method in script
• Can be executed declaratively in scriptless
  environment
• Handlers include events for
• Successful recognitions
• Misrecognitions
• Timeouts
• Each recognition event can be configured via
  attributes for
• Timeout periods
• Confidence thresholds

14
ltgrammargt Element

• Used to specify grammars
• Inline or referenced
• Multiple grammar elements may be used in single
  ltlistengt
• Individual grammars may be activated/deactivated
  before recognition begins
• Independent of grammar format
• Will support at minimum XML form of W3C Speech
  Recognition Grammar Specification

15
ltbindgt Element

• Used to inspect result of recognition
• Conditionally copies relevant portions to values
  in page
• Multiple bind elements may be used in single
  ltlistengt
• Recognition result returned in XML document form
• Uses XPath syntax in value attribute
• Uses and XML pattern query in test attribute

16
ltbindgt Element (contd)

• Value attribute
• To reference particular node of result
• Test attribute
• To specify binding conditions
• If condition evaluates to true, node content
  bound to page element specified by targetElement
  attribute

17
ltbindgt Element Example

• Recognition example
• ltresult textId like to go to London, please
  confidence0.45gt
• ltdest_city textto London confidence0.55gt
  Londonlt/dest_citygt
• lt/resultgt
• ltbindgt code
• ltinput nametxtBoxDestCity typetext /gt
• ltsaltlisten .gt
• ltsaltbind targetElementtxtBoxDestCity
  value/result/dest_city test/result/dest_city
  _at_confidence gt 0.4 /gt
• lt/saltlistengt

18
ltrecordgt Element

• Used to specify audio recording parameters
• Results may be processed with bind or scripted
  code

19
ltpromptgt Element

• Used to specify system output
• Content may include
• Text
• Speech output markup
• Variable values
• Links to audio files
• Mix of any of the above

20
ltpromptgt Element (contd)

• Executed in 2 ways ways
• Declaratively on scriptless browser
• By object methods in script
• Contains methods to start, stop, pause and resume
  prompt playback, and alter speed and volume
• Handlers include events for user barge-in,
  prompt-completion and internal bookmarks

21
ltpromptgt Element Example

• ltsaltprompt idConfirmTravelgt
• So you want to travel from
• ltsaltvalue targetElementtxtBoxOriginCity
  targetAttributevalue /gt
• to
• ltsaltvalue targetElementtxtBoxDestCity
  targetAttributevalue /gt
• ?
• lt/saltpromptgt

22
ltdtmfgt Element

• Used to specify DTMF grammars in telephony
  applications
• Deals with keypress input and other events
• Executed declaratively or programmatically with
  start and stop commands

23
ltdtmfgt Element (contd)

• Main elements include ltgrammargt and ltbindgt
• Holds resources for configuring DTMF collection
  process
• Configured via attributes for configuring
  timeouts
• Handlers include keypress events, valid dtmf
  sequences and out-of-grammar input

24
ltdtmfgt Element Example

• ltsaltdtmf iddtmfPhoneNumbergt
• ltsaltgrammar src7digits.gram /gt
• ltsaltbind value/result/phoneNumber
  targetElementiptPhoneNumber /gt
• lt/saltdtmfgt

25
Event writing

• SALT elements contain methods, properties and
  event handlers accessible to script
• Enable interaction with other events and
  processes in Web page
• Because SALT elements are XML objects in DOM of
  page

26
Event writing (contd)

• Top-level elements contain asynchronous methods
  for initiation and completion of execution
• Contain properties
• For configuration and result storing
• Event handlers
• For events associated with speech

27
Event writing

• onReco
• Event fired when recognition results successfully
  returned
• onBargein
• Event fired on prompt element if user input
  received during prompt playback

28
Code Examples

• ltinput nametxtBoxDestCity typetext
  onclickrecoDestCity.Start() /gt
• ltsaltlisten id"recoDestCity"gt
•     ltsaltgrammar src"city.xml" /gt
•     ltsaltbind targetElement"txtBoxDestCity"
•                value"/result/city" /gt
•   lt/saltlistengt

29
Code Examples (contd)

• ltinput type"button" onclick"recoFromTo.Start()"
  value"Say From and To Cities" /gt
•  
• ltinput name"txtBoxOriginCity" type"text" /gt
• ltinput name"txtBoxDestCity"   type"text" /gt
•  
• ltsaltlisten id"recoFromTo"gt
•     ltsaltgrammar src"FromToCity.xml" /gt
•     ltsaltbind targetElement"txtBoxOriginCity"
  
•                value"/result/originCity" /gt
•     ltsaltbind targetElement"txtBoxDestCity"
•                value"/result/destCity" /gt
•   lt/saltlistengt

30

• lt!- HTML --gt
• lthtml xmlnssalt"urnsaltforum.org/schemas/020124
  "gt
•   ltbody onload"RunAsk()"gt
•   ltform id"travelForm"gt
•    ltinput name"txtBoxOriginCity" type"text"
  /gt
•    ltinput name"txtBoxDestCity" type"text" /gt
  
•   lt/formgt
• lt!- Speech Application Language Tags --gt
•   ltsaltprompt id"askOriginCity"gt Where would
  you like to leave from? lt/saltpromptgt
•   ltsaltprompt id"askDestCity"gt   Where would
  you like to go to?      lt/saltpromptgt
•   ltsaltprompt id"sayDidntUnderstand"
  onComplete"runAsk()"gt 
• Sorry, I didn't understand.  
•   lt/saltpromptgt
•  
•   ltsaltlisten id"recoOriginCity"
  onReco"procOriginCity()" onNoReco"sayDidntUnders
  tand.Start()"gt
•             ltsaltgrammar src"city.xml" /gt
•   lt/saltlistengt
•  
•   ltsaltlisten id"recoDestCity"
   onReco"procDestCity()" onNoReco"sayDidntUnderst
  and.Start()"gt