Project

1
Project aireand Semantic Networks

• Seminar Multimodal Rooms
• Jochen von Spiczak

2
overview

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• project aire
• overview / interesting examples
• semantic networks
• introduction
• knowledge representation in IEs
• description of SNs,especially in intelligent
  rooms
• realization of SNs
• conclusion / future work

3
project aire

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• aire
• Agent-based Intelligent Reactive Environments
• MIT Artificial Intelligence Lab
• examining perceptual computing systems and
  applications

4
overview

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• tools for collaboration support
• supporting interactions among people, spaces and
  mobile devices

coll. support
project aire
5
overview

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• software infrastructure
• communication
• resource management
• device control

coll. support
infrastructure
project aire
6
overview

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• human-computer interfaces
• new interfaces for people to interact with their
  environments

coll. support
infrastructure
HCI
project aire
7
overview

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• aire spaces
• large range of form factors and sizes

conference rooms,personal workspaces,transitiona
l spaces
coll. support
infrastructure
HCI
project aire
8
examples

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

conference room,personal workspace,transitional
spaces
coll. support
infrastructure
HCI
project aire
9
examples

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• Metaglue
• Java-based agent infrastructure
• providing communication and levels of abstraction
• base of the rest of the project

coll. support
infrastructure
HCI
project aire
10
examples

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

coll. support
infrastructure
HCI
project aire
11
examples

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• visit this very day
• http//www.ai.mit.edu/projects/aire

coll. support
infrastructure
HCI
project aire
12
semantic networks

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• one outstanding aspect of the aire project
• knowledge representation as a fundamental design
  question
• semantic networks as such a representation

13
introduction

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• ever-increasing amounts of data about users
• who they communicate with,
• their location,
• visited web pages, ...
• aim
• quick, correct assumptions about what the user
  would like to do next

14
knowledge representation in IE

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• five different roles that KRs play
• surrogates for the real world
• ontological commitments
• theories of reasoning
• media for computation
• media for human expression

• easily/less easily identifiable entities
• important for reasoning about items and making
  deductions based on their layout

• KR imposes a filter on what the environment can
  see and act on
• example of camera-/microphone-signal

• KR defines how inferences are made by the
  reasoning system
• constraints kinds of inferences which are allowed
  or recommended by the system

• KR must be able to operate in an efficient manner
  via computational processes
• without taking up inordinate quantities of time,
  memory, or disk space

• KR represents the human view of the world
• must be simple to use
• turn into human-readable representations

15
knowledge representation in IE

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• much literature about developing contextual
  systems for IE
• problems of other approaches
• interpretation and inferencing is done by the
  application
• application must collate and assemble disparate
  pieces of data
• no cohesive representation of the gained knowledge

16
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• need for a representation that
• can store information on objects
• can easily track and follow the relationships
  between them
• introducing semantic networks as one such
  representation

17
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• concept of semantic networks
• information represented in a directed graph
• nodes ? objects
• links ? relationships

18
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• variety of node and link types
• nodes just like
• people,
• meetings,
• documents,
• issues,
• discourse items,
• commitments,

19
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• variety of node and link types
• links just like
• relationship between people,
• responsibilities for objects (groups/people),
• hierarchies between contributors,
• attendance to meetings,
• interest for objects,

20
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• further link types
• endpoints for links not limited to storage nodes
• can also be created to other links
• allows more complex interactions in the data model

21
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• easy to discover
• information about a particular object
• relationships between different objects
• little information stored in nodes
• network relies more on the links to define the
  relationships

22
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• some inherent advantages
• adding data is highly straightforward
• changing data is highly localized
• making inferences is fast and easy

23
SN in general

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• further advantages
• great flexibility in terms of the data which is
  captured and interpreted
• extremely efficient
• easy to bring into human-readable form

24
SN in intelligent rooms

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• several areas for using a SN-based representation
• user knowledge
• meeting management
• location management
• SN as an appropriate and valuable piece of
  infrastructure

25
SN in intelligent rooms

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• user knowledge
• knowledge of the users andthe individual spaces
  they work with
• simplistic level
• simple set of objects, spaces andthe users
  current location
• queries like Where is Steve located?
• system as an intelligent assistant
• far more information and relations must be
  included
• more complex queries likeWho is Joes superior?

26
SN in intelligent rooms

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• meeting management
• capturing meetings as the occur
• typical information
• agenda topics,
• action items,
• supporting and dissenting arguments,
• documents such as presentations orweb
  references,
• can be linked to a video or audio capture of the
  meeting

27
SN in intelligent rooms

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• location management
• location information important for IEs
• especially true when integratinghand-held
  devices
• require knowledge of the current location
• own coordinate framework of each system
• SN as a location infrastructure
• integration and translation between different
  coordinate systems
• specify own location descriptions,such as at
  home

28
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• technology of SNs deployed within the project
  aire
• current implementation
• works on top of a SQL database
• easily extendable to use
• object-oriented databases
• resource description framework (RDF)back-ends

29
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• access to the semantic network
• using the Metaglue agent infrastructure of
  project aire
• several agents used to access the SN
• MeetingModel agent
• GUI agent
• other viewing/editing agents
• network query language

30
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• semantic network itself
• provides only basic interface methods

semantic network
31
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• MeetingModel agent
• serves merely as an interface to SN
• abstracts out some of the lookup tasks
• broadcast of any changes

MeetingModel
semantic network
32
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• network query language
• allows user or agents to specify queries in a
  formal language
• definition of rules in a syntax similar to Prolog

MeetingModel
query language
semantic network
33
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• GUI agent
• tree-structure view of stored information
• creation/editing of information nodes
• currently the primary conduit for information to
  get into the SN

GUI agent
MeetingModel
query language
semantic network
34
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

35
realization of SNs

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• other viewing/editing agents
• providing different views of the data
• web interfaces,
• graph networks,
• specialized meeting views which simply present
  the agenda or commitments,

GUI agent

MeetingModel
query language
semantic network
36
example of a SN

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

37
example of a SN

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

face tracking
Mercedes SLK
expert-in
expert-in
assisted
rest
Jochen
Stiefelhagen
bores-to-death
currently-doing
responsible-for
responsible-for
seminar talk
belongs-to

• John McDonough.

• In room 225.

38
example of a SN

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

face tracking
Mercedes SLK
speech recognition
room 225
expert-in
expert-in
expert-in
works-in
assisted
rest
Jochen
McDonough
Rogina
Stiefelhagen
bores-to-death
currently-doing
responsible-for
responsible-for
responsible-for
multimodal rooms
seminar talk
belongs-to
39
conclusion / future work

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• seems to be the most appropriate knowledge
  representation for IEs
• flexibility of data
• adding/changing data highly localized
• inference generation extremely efficient
• future work
• integration within other projects
• automatic sensor-based information recognizing
• incorporate user preference information

40
summary

• overview
• project aire
• overview
• examples
• semantic networks
• - introduction
• knowledge representation in IE
• SN in general
• SN in intelligent rooms
• realization of SN
• example of a SN
• conclusion /future work
• summary

• project aire
• semantic networks
• one approach to KR
• inherent advantages in general
• and in the context of IEs
• realization
• implementation
• access hierarchy
• example
• most appropriate approach to KR

41
summary

• project aire
• semantic networks
• one approach to KR
• inherent advantages in general
• and in the context of IEs
• realization
• implementation
• access hierarchy
• example
• most appropriate approach to KR

discussion
?