MAGA: a Mobile Archaeological Guide at Agrigento

1
MAGAa Mobile ArchaeologicalGuide at Agrigento
SpeechTek 07 - New York City - 21 august 2007
Agnese Augello1, Antonella Santangelo1, Salvatore
Sorce1, Giovanni Pilato2, Antonio Gentile1,
Alessandro Genco1 and Salvatore Gaglio1,2
1DINFO - Dipartimento di ingegneria Informatica University of Palermo Viale delle Scienze - 90128 Palermo - Italy 2ICAR - Istituto di CAlcolo e Reti ad alte prestazioni branch of Italian National Research Council Viale delle Scienze - 90128 Palermo - Italy
2
Summary

• Aim of the work
• Enabling Technologies
• MAGA System architecture
• Implementation details
• Examples
• Conclusions

3
Aim of the work

• To build smart multimodal systems for information
  retrieval
• user-friendly
• suitable for different mobile devices
• We propose the use of a verbal interface (for
  interactions) along with RFID (for in-door
  positioning purposes)
• Requirements
• Natural language interface
• Availability of the system in environments where
  mobility of the user is essential (e.g. the
  Archaeological Museum of Agrigento)

4
Enabling technologies

• Personal Digital Assistants (PDAs)
• Multimodal Browsers (Verbal Interaction)
• Conversational Agents (Chatbots)
• Semantic Networks (Reasoning Capabilities)
• Radio Frequency Identification (RFID)

5
Personal Digital Assistant (PDA)

• Mobile devices and wireless communication allow
  people to access information everywhere

Information retrieval on small devices is limited
I/O interface
Boring interaction and large waste of time during
information research!!!
6
Verbal interaction

• Experimental results demonstrate that verbal
  interaction allows people to reduce the time
  spent on manual input (Lyons et al, 2005)
• Vocal interaction is a more direct approach, but
  it comes with a set of constraints
• processing power
• speakers adaptation
• narrow dictionaries
• bounding grammars

7
Conversational Agents

• A simple approach to build a dialogue system
• Pattern-matching based rules
• Chatbots can be used as interfaces for several
  applications (e.g. entertaining, educational,
  e-learning )

8
Semantic networks

• Allow to
• describe a specific domain
• Concepts, properties of concepts, relations
  between concepts are clearly defined in that
  domain
• represent commonsense knowledge covering aspects
  of everyday life
• make inference to generate new knowledge from the
  existing one
• share and reuse common understanding of the
  structure of information
• Es OpenCyc, OpenMind, WordNet

9
RFID
Passive tags limited reading range -gt limited
position estimation error Useful in environments
with spot-areas
10
MAGA

• Integration of chatbot, commonsense reasoning,
  speech recognition/synthesis and RFID
  technologies to build a virtual guide accessible
• in a cultural heritage environment (such as the
  Archaelogical Museum of Agrigento) by means of
  different mobile devices
• by simulation on a multimodal browser

11
MAGA System Architecture
12
RFID tag detection
Database
Tag 2
Tag 1
Tag 3
Tag 4
13
Verbal interaction engine
14
Verbal InteractionXV Technology

• XHTMLVoice (XV) as implementation language
• Multimodal Tools 4.1.2.2 for WebSphere Studio
  V5.1.2
• IBM Multimodal Toolkit
• IBM Multimodal Browser, (XV interpreter)

XV page
15
XV Client-server architecture
16
XV and Grammar Creation
php request (question)
ChatBot
question in textual form
PHP for XV creation
XHTML code
ChatBot (question)
replay in textual form
VXML code with the reply
php request (domain)
VXML code with grammar
PHP for Grammar creation
XHTML code
Domain Database
info domain
Rule Pre
info domain
Rule Key-Word
info domain
Rule Post
MXML
GRXML
17
Grammar example
ltrule id"main_rule"gt ltitem repeat"0-1"gt ltruleref uri"pre"/gt lt/itemgt ltruleref uri"parola_chiave"/gt ltitem repeat"0-1"gt ltruleref uri"post"/gt lt/itemgt lt/rulegt ltrule idparola_chiave"gt ltone-ofgt ltitemgt Archeological Museum lt/itemgt ltitemgt Talamone lt/itemgt ltitemgt Acropolis lt/itemgt lt/one-ofgt lt/rulegt ltrule id"pre"gt ltone-ofgt ltitem repeat"0-1"gt tell me lt/itemgt ltitem repeat"0-1"gt i need information lt/itemgt ltitem repeat"0-1"gt to when lt/itemgt lt/one-ofgt ltitem repeat"0-1"gt about lt/itemgt ltitem repeat"0-1"gt the lt/itemgt lt/rulegt ltrule id"post"gt ltitem repeat"0-1"gt is dated lt/itemgt ltitem repeat"0-1"gt in Agrigento lt/itemgt lt/rulegt
18
Chatbot Knowledge Base
19
Chatbot technology ALICE

• Chatbots systems dialogue based on natural
  language
• ALICE free software technology

ALICE is an artificial intelligence, natural
language chat robot
The knowledge base is composed of question-answer
modules, called categories and structured with
Artificial Intelligence Mark-up Language (AIML)
20
Chatbot technology ALICE

• Each category is a rule for matching an input and
  converting to an output, and consists of a
  pattern, which represents the user input, and a
  template, which represents the chatbots answer.
• ltcategorygt
• ltpatterngtWHAT IS A CIRCLEgtlt/patterngt
• lttemplategtA circle is the set of points
  equidistant from a common point called the
  centerlt/templategt
• lt/categorygt
• The chatbots knowledge is stored in a tree
  called GraphMaster

Alices brain
21
OpenCyc Ontology

• Knowledge base logic assertions, common sense
  rules and heuristics for the reasoning about
  quotidian life objects and events
• Inference system is based on a series of
  heuristic related to a reasoning techniques about
  microtheories
• CycL the language in which Cyc (and OpenCyc) has
  been written, is a formal language from Lisp
  language

22
The AgrigentoMuseumMt created in OpenCyc
23
Example a Microtheory
24
ALICE-KB and CycInference capabilities

• We have written a java application that bridges
  the AIML interpreter with the OpenCyc inference
  engine (based on the CyN project).
• The knowledge base of the chatbot is then
  enriched with new AIML tags that enable the
  ontology querying, executing and asserting
  CycL/SubL statements.
• This allows (common sense) reasoning capabilities
  inside Alice chatbot.
• A template can be seen as a meta-answer that must
  be processed to compose the appropriate response.

25
ALICE-KB and Cyc Inference Capabilities

• The created ontology enables the composition of
  answers that are not present in the traditional
  AIML knowledge base.
• It is possible to
• extend the number of AIML categories to obtain a
  more fluent dialogue
• extend the ontology with new concepts, facts and
  relations of the specific domain to provide the
  chatbot a smarter behavior.

26
An example of AIML Category

• ltpatterngt
• I NEED INFORMATION ABOUT THE lt/patterngt
• lttemplategt
• ltcycsystemgt(cyc-query '(Comment
  ltcyctermgtltstar/gtlt/cyctermgt
• lt/cycsystemgt
• It is located at
• ltcycsystemgt(cyc-query '(located_in ltstar/gt
  ?X)) lt/cycsystemgt
• and has been created by
• ltcycsystemgt(cyc-query '(createdBy ltstar/gt ?X))
  lt/cycsystemgt
• lt/templategt
• and the resulting interaction
• User I need information about the Archeological
  Museum in Agrigento.
• Chatbot It is one of the most modern museums in
  Sicily and it contains archeological finds from
  the provinces of Agrigento and Caltanissetta,
  extending from prehistoric to Roman times. It is
  located at Agrigento, and has been created by the
  architect Franco Minissi.

27
An example of AIML Categories using the RFID
information

• ltcategorygtltpatterngt LOCALIZATION
  DETECTEDlt/patterngtlttemplategt
• You are in ltset nameshowcasegtltsystemgtjava RFID
  showcaselt/systemgtlt/setgt
• ltsraigt INFO SHOWCASE ltget nameshowcase/gt
  lt/sraigt
• lt/templategtlt/categorygt
• ltcategorygtltpatterngtINFO SHOWCASE
  lt/patterngtlttemplategt
• The showcase contains ltset nameartifactgtltcycsys
  temgt(cyc-query '(Contains ltstar/gt ?X))
  lt/cycsystemgtlt/setgt.
• ltsraigt INFO ARTIFACT ltget nameartifact/gt
  lt/sraigt
• lt/templategtlt/categorygt
• ltcategorygt
• ltpatterngt INFO ARTIFACT lt/patterngt
• lttemplategt belongs to
• ltcycsystemgt(cyc-query '(CreatedDuring
  ltstar/gt ?X)) lt/cycsystemgt,
• it is attributed to
• ltcycsystemgt(cyc-query '(CreatedBy
  ltstar/gt?X)) lt/cycsystemgt and
• and represents

28
A dialogue example

• User I need information about the Archeological
  Museum in Agrigento.
• Chatbot It is one of the most modern museums in
  Sicily and it contains archeological finds from
  neighbors of Agrigento and Caltanissetta, ranging
  from prehistoric to Roman times.
• Chatbot You are in room 3, in front of the
  showcase 17. Do you want more information?
• User Yes.
• Chatbot In this showcase you can see an Attic
  amphora. It belongs to Arcaic Greek period, it is
  attributed to Dikaios and represents Athena.
• User What is in the next room?
• Chatbot In the room 4 there are Capitals and
  gargoyles with lion's heads from various temples.

29
Demo
30
Conclusions and future work

• Chatbot with reasoning capabilities and speech
  recognition technologies have been embedded in a
  Personal Digital Assistant to assist user in
  visiting historical sites
• Natural interaction overcomes the limitation of
  traditional pre-recorded audio/visual guides
• Future work will regard the enhancement of the
  system
• Enhance the chatbot reasoning capabilities and
  natural language interaction
• Use of different wireless technologies for
  context-aware service provision
• Use of user profiles for ad-hoc service
  composition

31
Thank you!

• Vocal User Interface, System Architecture Design
• Antonella Santangelo santangelo_at_csai.unipa.it
• Antonio Gentile gentile_at_unipa.it
• Information retrieval, reasoning
• Agnese Augello augello_at_csai.unipa.it
• Giovanni Pilato pilato_at_pa.icar.cnr.it
• Salvatore Gaglio gaglio_at_unipa.it
• RFID, pervasive access, HCI
• Salvatore Sorce sorce_at_unipa.it
• Alessandro Genco genco_at_unipa.it