Using VoiceXML, XHTML, and SCXML to Build Multimodal Applications

1
Using VoiceXML, XHTML, and SCXML to Build
Multimodal Applications
James A. Larson
2
Agenda

• SCXML controls the flow of an application
• SCXML controls modalities
• VoiceXML
• XHTML
• Others, e.g., InkML, SVG
• SCXML controls Web services

3
State Chart XML

• Represents a state transition system
• States
• Transitions

A
B
4
Possible Actions within a ltstategt

• Multimodal life cycle events
• ltstart /gt
• ltcancel /gt
• ltpause /gt
• ltresume /gt
• ltdata /gt
• Executable content
• ltassign/gt
• ltvargt
• ltscript/gt
• ltif/gt, ltelseif/gt, ltelse/gt
• ltlog/gt

A
ltif gt ltstart /gt ltelse/gt ltlog /gt lt/ifgt
5
SCXML

• Sequentially perform the actions in two states

• ltstate id"A"gt
• ltprepare /gt
• lttransition event "done" target"B"/gt
• lt/stategt
• ltstate id"B"gt
• ltstart /gt
• lt/stategt

A
ltprepare /gt
done
ltstart /gt
B
6
SCXML

• Conditional branch

• ltstate id"A"gt
• ltprepare /gt
• lttransition event "done"
• cond"result'OK?'"
  target"B"/gt
• lttransition event "done"
• cond"result'Opps'"
  target"C"/gt
• lt/stategt

A
ltprepare /gt
done OK
done Opps
B
C
7
SCXML

• Parallel States

• ltstate id"A"gt
• lttransition . target"B"/gt
• lt/stategt
• ltstate id"B"/gt
• ltparallelgt
• ltstate id"C"/gt
• ltstate id"D"/gt
• lt/parallelgt
• lt/stategt

A
B
C
D
8
Agenda

• SCXML controls the flow of an application
• SCXML controls modalities
• VoiceXML
• XHTML
• Others, e.g., InkML, SVG
• SCXML controls Web services

9
SCXML VoiceXML
ltstate id"A"gt ltstart name
"getAirport.vxml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
A
lt?xml version"1.0"?gt ltvxml version"3.0"
xmlns "http//www.w3.org/2001/vxml"gt ltform id
"destination" gt ltfield name "Airport"gt
ltgrammar src "airport.grxml"/gt
ltpromptgtSay your destination airport lt/promptgt
ltgrammar src Airport.grxml
lt/fieldgt ltfilledgt ltsend event
"done" namelist "Airport" /gt lt/filledgt
lt/formgt lt/vxmlgt
getAirport.vxml
done
B
state id"B"gt ltstart /gt lt/stategt
10
SCXML XHTML
ltstate id"A"gt ltstart name
"getAirport.xhtml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
A
lthtml/gt ltheadgt ltevlistener
evevent"onchange"
evobserver"bodyId"
evhandler "sendDone()" /gt ltscriptgt function
sendDone lt/scriptgt lt/headgt lt!--xhtml code
--gt ltbody id"bodyId"/gtltpgtenter monthlt/pgt
ltinput type"text" id"month"/gt lt/bodygt lt/htmlgt
getAirport.xhtml
done
B
state id"B"gt ltstart /gt lt/stategt
11
SCXML XHTML VoiceXML
A
B
ltstate id"A1"gt ltstart name
"getAirport.vxml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
ltstate id"A2"gt ltstart name
"getAirport.xhtml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
A1
A2
done
done
B
state id"B"gt ltstart /gt lt/stategt
12
SCXML InkML VoiceXML
Another example of multiple modalities
A
B
ltstate id"A1"gt ltstart name
"getStuff.vxml" /gt lttransition event "done"
target"B"/gt lt/stategt
ltstate id"A2"gt ltstart name
"getStuff.Inkml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
A1
A2
done
B
state id"B"gt ltstart /gt lt/stategt
13
SCXML InkML VoiceXML
A
B
ltstate id"A1"gt ltstart name
"getStuff.vxml" /gt lttransition event "done"
target"B"/gt lt/stategt
ltstate id"A2"gt ltstart name
"getStuff.Inkml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
A1
A2
ltinterpretation mode "speech"gt lttravelgt
ltto hook"ink"/gt ltfrom
hook"ink"/gt ltdaygt Tuesday lt/daygt
lt/travelgt lt/interpretationgt
done
B
state id"B"gt ltstart /gt lt/stategt
14
SCXML InkML VoiceXML
A
B
ltstate id"A1"gt ltstart name
"getStuff.vxml" /gt lttransition event "done"
target"B"/gt lt/stategt
ltstate id"A2"gt ltstart name
"getStuff.inkml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
A1
A2
ltinterpretation mode "ink"gt lttravelgt
lttogtLas Vegas lt/togt ltfromgtPortland
lt/fromgt lt/travelgt lt/interpretationgt
ltinterpretation mode "speech"gt lttravelgt
ltto hook"ink"/gt ltfrom
hook"ink"/gt ltdaygt Tuesday lt/daygt
lt/travelgt lt/interpretationgt
done
B
state id"B"gt ltstart /gt lt/stategt
15
SCXML InkML VXML
ltinterpretation mode "interp1"gt lttravelgt
lttogt Las Vegas lt/togt ltfromgt
Portland lt/fromgt ltdaygt Tuesday lt/daygt
lt/travelgt lt/interpretationgt
A
B
ltstate id"A1"gt ltstart name
"getStuff.vxml" /gt lttransition event "done"
target"B"/gt lt/stategt
ltstate id"A2"gt ltstart name
getStuff.inkml" /gt lttransition event
"done" target"B"/gt lt/stat
egt
A1
A2
ltinterpretation mode "ink"gt lttravelgt
lttogtLas Vegas lt/togt ltfromgtPortland
lt/fromgt lt/travelgt lt/interpretationgt
ltinterpretation mode "speech"gt lttravelgt
ltto hook"ink"/gt ltfrom
hook"ink"/gt ltdaygt Tuesday lt/daygt
lt/travelgt lt/interpretationgt
done
B
state id"B"gt ltstart /gt lt/stategt
16
Agenda

• SCXML controls the flow of an application
• SCXML controls modalities
• VoiceXML
• XHTML
• Others, e.g., InkML, SVG
• SCXML controls Web services

17
SCXML Control More Than Just Modalities
vxml
ltstart /gt

• SQL
• Complex calculations
• Web services

ltstart /gt
sql
ltstart /gt
complexCalculation
xhtml
ltstart /gt
18
References

• State Chart XML (SCXML) State Machine Notation
  for Control Abstraction
• http//www.w3.org/TR/scxml/
• Multimodal Architecture and Interfaces
• http//www.w3.org/TR/mmi-arch/
• Commons SCXML (Open Source)
• http//jakarta.apache.org/commons/scxml/

19
Summary

• SCXML controls the flow of an application
• SCXML controls modalities
• XHTML
• VoiceXML
• Others, e.g., inkML, SVG
• SCXML controls Web services

20
(No Transcript)