An Intelligent Broker for Pervasive Context-Aware Systems

1
An Intelligent Broker for Pervasive
Context-Aware Systems

• Harry Chen
• University of Maryland, Baltimore County
• December 3, 2004

2
Outline

• Introduction (10 mins)
• The problem, my solution contributions
• Related Works (5 mins)
• Different approaches exist why mine is better
• Context Broker Architecture (20 mins)
• Ontologies, context reasoning privacy
  protection
• Implementations (10 mins)
• EasyMeeting, CoBrA Demo Toolkit CTMC
• Conclusions (7 mins)
• Lessons learned future works

3
Evolution
4
Could this be the solution?
5
The Vision

• Pervasive Computing a natural extension of the
  present human computing life style
• Using computing technologies will be as natural
  as using other non-computing technologies (e.g.,
  pen, paper, and cups)
• Computing services will be something that is
  available anytime and anywhere.

6
Pervasive Computing is the Future
Pervasive Computing Saves the World
7
Context-Aware Systems

• Context-awareness is a key aspect of the
  intelligent pervasive computing systems
• Systems that can anticipate users needs and act
  in advance by understanding their context
• A system that knows I am the speaker
• A system that knows you are the audience
• A system that knows we are in a conference

8
Whats Context?

• The situational conditions that are associated
  with a user
• Location, time, room temperature, lighting
  conditions, noise level, social activities, user
  intentions, user beliefs, user roles, personal
  information, etc.

9
Research Issues

• Context Modeling Reasoning
• How to represent context, so that it can be
  processed and reasoned by the computers
• Knowledge Maintenance Sharing
• How to maintain consistent context knowledge and
  share that information with other systems
• User Privacy Protection
• How to let users to control the sharing and the
  use of their contextual information that is
  acquired by the hidden sensors

10
Thesis Statement

• By developing a broker-centric agent architecture
  with expressive ontologies, context reasoning
  procedures, and a policy-based privacy protection
  mechanism,
• we can help computing entities to represent and
  share context, to detect and resolve inconsistent
  knowledge, and to protect user privacy.

11
My Research Contributions

• CoBrA a broker-centric agent architecture for
  supporting pervasive context-aware systems
• Use a Semantic Web language to define ontologies
  for context modeling and reasoning
• Use logical inference to interpret context and to
  detect and resolve inconsistent knowledge
• Allow users to defined policies to control the
  sharing of their contextual information

12
Other Contributions

• EasyMeeting
• A smart meeting room prototype that is built on
  CoBrA provides context-aware services to
  speakers and audiences.
• CoBrA Demo Toolkit
• An open source software package for demonstrating
  various aspects of CoBrA
• CoBrA Text Messaging Commands
• A text messaging interface for mobile users to
  interact with a context broker via SMS

13
Related Work
14
Weve Come a Long Way

• Since the early 90s, people have been interested
  in building context-aware systems
• Olivetti Call forwarding teleporting systems
• Xerox PARC Active map, PARC Tab
• Georgia Tech. Context toolkit, cyberguide
• MIT Office assistant, location-aware information
  delivery, intelligent room
• UC Berkley Context Fabric
• UIUC Gaia
• HP Labs Cooltown, CoolAgents
• Microsoft EasyLiving

15
A Call-Forwarding System
By Roy Want el at. 1992
16
Types of Context Used
Location Context
Activity Context
17
A Shopping Assistant
By Abhaya Asthana el at. 1994
18
Types of Context Used
Location Context
Identity Context
19
Why context-awareness is still a research topic?

• Wherere they now?
• The call-forwarding system (1992)
• The shopping assistant (1994)
• Microsofts EasyLiving (1998)
• Why havent we seen any successful commercialized
  context-aware systems?

20
Many pieces are still missing
Social Acceptance
???

???
Demand

Memory
???
Context Modeling
Applications
???
???
???

Privacy Protection
Business
UI
Context Reasoning
Battery Life


Trust
???
Wireless
???

Sensors
???
???
Security

???



CPU Power
Cross Platform

Profit
???
???
???
21
My work adds few more pieces
Social Acceptance
???

???
Demand

Memory
???
Context Modeling
Applications
???
???
???

Privacy Protection
Business
UI
Context Reasoning
Battery Life


Trust
???
Wireless
???

Sensors
???
???
Security

???



CPU Power
Cross Platform

Profit
???
???
???
22
Comparing Different Systems
Context Modeling Context Reasoning Privacy Protection
Gaia/Mist Data Structure DAMLOIL Hardwired procedures Mist Communication Protocol (hides location)
Video Streaming App. (Odyssey) Data Structure Hardwired procedures N/A
MIT Intelligent Room Data Structure Model-based Reasoning N/A
XeroxPARC Active Badge Apps Data Structure Hardwired procedures N/A
Cooltown Museum XML Hardwired procedures SQL N/A
MerCeDeS RDF/RDFS RDFS Reasoning N/A
Context Broker Architecture OWL OWL Assumption-based Reasoning User Defined Policy
23
Shortcomings of the Existing Systems

• Lacking an adequate representation for modeling
  context
• Individual agents are responsible for managing
  and reasoning their own contextual knowledge
• Users dont have control over how their
  contextual information is shared and used

24
Context Broker Architecture
25
The Context Broker Architecture
26
Key Features of CoBrA

• Using OWL to define ontologies for context
  modeling and reasoning
• Using logical inferences to detect and resolve
  inconsistent knowledge
• Using declarative policies to control the sharing
  of user context

27
An EasyMeeting Scenario
28
An EasyMeeting Scenario
29
CoBrA Components
Context Reasoning
Ontologies
PrivacyProtection
30
Context Reasoning
Ontologies
Privacy Protection
31
Ontologies in CoBrA

• Ontologies are expressed using the Web Ontology
  Language OWL
• Ontologies are used to support
• Context modeling
• Knowledge sharing
• Context Reasoning
• SOUPA (Standard Ontology for Ubiquitous and
  Pervasive Applications)
• COBRA-ONT extends SOUPA used in the prototypes

32
Semantic Web Languages
http//www.w3.org/2001/sw/

• KR languages for defining ontologies
• W3C Recommendations
• RDF/RDFS -- represents information as N-Triples
  (subject, predicate, object) supports basic
  class-subclass properties.
• OWL (Web Ontology Language) -- adds more vocab.
  for describing classes and properties,
  cardinality, equality, XML datatypes,
  enumerations etc.

33
The Semantic Web Layer Cake
The Semantic Web will globalize KR, just as the
WWW globalize hypertext -- Tim Berners-Lee
we arehere
34
Why Ontologies? Why OWL?
ontology language
service description lang.
context model
PerCom
Interop language
meta lang (policy)
XSLT/XML friendly
OWL provided a uniformed language which met many
needs in developing a complex pervasive computing
system.
35
The SOUPA Ontology Effort
Maintained by the Semantic Web in UbiComp
SIG http//pervasive.semanticweb.org

• Develop a standard ontology to support pervasive
  computing applications
• Helps to facilitate
• Knowledge sharing reasoning
• Service discovery
• Security and privacy protection
• Context modeling
• Reuse/adopt the existing ontologies define use
  cases to guide the development process

36
Some Existing Ontologies

• Social Network
• People
• FOAF
• SOUPA Person

• Space location
• CYC space ontology
• Region Connection Calculus
• SOUPA space location

• Agent BDI
• MoGATU BDI Ontology
• SOUPA Agent Action

• Policy
• Rei
• KAoS policy
• Policy in e-Wallet
• SOUPA Policy

37
The SOUPA Ontology
38
Time Ontology

• Objective aims to develop a representative
  ontology of time that expresses temporal concepts
  and properties common to any formalization of
  time
• Common Usage
• Describing the start and the end time of an event
• Reasoning about the temporal order of different
  events
• Incorporating time/date XML datatypes into
  representations that are suitable for temporal
  reasoning
• Core constructs (common in DAML Time SOUPA)
• Time instant -- point like thing
• Time interval -- consists of a beginning and an
  end point
• XML Datatypes -- date time, duration, time zone
  etc.

39
Temporal Ordering

• Temporal ordering relations
• Between two time intervals

equals
overlaps
starts finishes
during
nonoverlap
A.K.A. Allens Interval Calculus
40
A Time Ontology Example
An event E1 starts from 2004/09/25 130000 to
2004/09/26 143000
41
Temporal Ordering (1)
The property before is owlinverseOf the property
after
42
Temporal Ordering (2)
The time ontology also includes constructs for
expressing temporal order between time instants
and intervals
43
Space Location Ontology

• Objective provide vocabularies for expressing
  spatial properties and for supporting spatial
  reasoning
• Where did this event happen?
• What rooms are spatially subsumed by this
  building?
• Is Bob at work or at home?
• Is Bob located in the United States?
• Common Usage
• Describe and reason about peoples location
  context
• Detect inconsistent information about peoples
  location
• Facilitate knowledge sharing between
  location-aware services
• Core Constructs
• Geo-metric longitude, latitude, altitude (see
  also OpenGIS)
• Symbolic named location (e.g., room, building)

44
Describing Space

• Geo-metric Representation
• George W. Bush is located at
• 38 53' 55" , -77 2' 14 (Lat-Long)
• Symbolic Representation
• George W. Bush is located at
• White House
• Both representations are useful for building
  location-aware systems
• Calculate the distance between two points
• Reason about the proximity of a persons location

45
SOUPA Space Ontology

• The SOUPA space ontology defines vocabularies for
  describing spatial regions and location
  properties
• Covers both symbolic geo-metric representation
• Also defines an ontology of the Region Connection
  Calculus (RCC) for qualitative spatial reasoning

a
b
ab
a
b
a is identical with b
a partially overlaps b
a is externally connected with b
a
b
a
b
a is disconnected from b
a is a tangential proper part of b
46
A Simple UMBC Space Ontology
47
Wheres Harry?
48
Detect Inconsistent Information
49
Policy Ontology

• Objective define vocabularies for expressing
  declarative policies that guide the behavior of
  computing entities in an open, distributed
  environment
• Policies are useful at virtually all levels (OS,
  networking, data management, security privacy
  applications)
• Policies are less technical than programming
  code
• Change policies doesnt require re-compile the
  source code
• Common Usage
• Authorization for services
• Privacy protection in pervasive computing
• Policy for guiding team formation and
  collaboration
• Core Constructs
• Positive negative authorizations obligations
• Domain actions

50
SOUPA Policy Ontology

• For defining policies to control the performance
  of actions
• Each policy defines a set of actions that are
  either permitted or forbidden (prohibited)
• Uses Description Logic for policy reasoning
• It imports other ontologies
• SOUPA Action ontology
• SOUPA Time ontology

51
The SOUPA Policy Ontology
52
A Policy Example
lthttp//umbc.edu/hchen4/hchen.polgt a
polPolicy polpolicyOf a perPerson
pername "Harry Chen"xsdstring
poldefaultPolicyMode polConservative ...
Rule 2 all individuals of CLS2 are permitted
actions polpermits haCLS2 haCLS2 a
Class intersectionOf (
ebactShareLocationInfo onProperty
actrecipient allValuesFrom ebmEbiquityMember
onProperty acttarget
someValuesFrom haMyRestrictedLocationContext
) . haMyRestrictedLocationContext a Class
intersectionOf ( locLocationContext
onProperty locboundedWithin someValuesFrom
hafoo-a1 ) . hafoo-a1 a Class oneOf
(ebgeoITE210A ebgeoITE325B ebgeoUMBCMainCampus
) .
53
Summary

• CoBrA uses the OWL language to define ontologies
  for context modeling and knowledge sharing
• An explicit semantic representation of context is
  more expressive than the data structure
  representations used in the past systems
• Contributions
• A standard ontology for pervasive computing
  applications (SOUPA)
• Showed the use of Semantic Web technologies in a
  pervasive computing domain

54
Context Reasoning
Ontologies
Privacy Protection
55
Kinds of Context Reasoning

• Reasoning for interpreting context
• e.g., infer location context, user intention
  user desire
• Reasoning for detecting and resolving
  inconsistent knowledge
• e.g., conflicting location information about the
  same person or device

56
The Use of Rule-Based Systems

• CoBrA exploits the use of rule-based systems for
  context reasoning
• JESS, Prolog Jena ontology reasoners
• Different engines are used for different types of
  context reasoning
• Some for context interpretation
• Some for detecting and resolveing inconsistency

57
Context Reasoning Framework
58
Integrating Rule-Based Reasoners
59
Time Reasoner

• Two different implementations
• One in Prolog and the other in Jena
• Both implement Allens Interval Calculus
• Both support XML date time standards
• ISO 8601 Date and Time format
• UTC normalization
• Jenas implementation built on Jena Generic Rule
  Reasoner
• Supports SOUPA Time ontology (RDF/OWL)

60
Temporal Reasoning Example
61
Spatial Reasoner

• Built on the Jenas Generic Rule Reasoner
• Supports the SOUPA Space ontology (RDF/OWL)
• Supports mapping between the symoblic and
  geo-metric representations
• Whats the longitude latitue of UMBC?
• Can be extended to support RCC

62
Supported Spatial Inferences

• Given geographical space X Y
• Infer whether or not X spatially subsumes Y
• Infer whether or not X is spatially subsumed by Y
• Infer whether or not X is disconnected from Y
• Given X Y with defined latitudes and longitudes
  
• Infer whether or not X spatially subsumes Y
• Infer whether or not X is spatially subsumed by Y
• Infer whether or not X is disconnected from Y

63
Meeting Status Reasoner

• Built on the JESS engine
• For supporting the EasyMeeting system
• Infers the context of a scheduled meeting
• Sensing information is translated from RDF/OWL to
  Jess rules
• Reasoning results are mapped back into RDF and
  stored in the KB

64
Meeting Context Reasoning
65
Inconsistency Found! What Should I Do?
66
Resolve Inconsistent Knowledge

• Information acquired from different sources may
  be in conflict
• Sensor A says, Harry is in ITE-201
• Sensor B says, Harry is is ITE-338
• Q Who should we believe?
• Given everything we know about the context, which
  information is more reliable or less reliable
  than the others?

67
Reason over Assumptions

• Assumptions (defaults) are made when interpreting
  contexts, and some of which are unreliable
• Unreliable assumptions cause unreliable
  conclusions
• Solution by disproving unreliable assumptions,
  we disprove unreliable conclusions that cause
  conflicts
• Theorist an assumption-based reasoning
• FIPA argumentation ACL (Broker ? Broker)

68
An Example

• A Broker receives info about peoples location
• Agent A using a voice recognition software
• Agent B using a Bluetooth cellphone sensor
• Implicit assumptions about the reliability of the
  location information
• The noise level in the room is normal, and which
  doesnt affect the accuracy of the voice
  recognition software (lt60 dBA)
• The co-location relation often holds between a
  mobile user and the users cellphone

69
Resolve Conflicting Knowledge
Agent A
Context Broker
Agent B
I believe Harry is in RM201A
I believe Harry is in RM338
Inconsistency detected
What was the rooms noise level?
How many missed outgoing calls on his
cellphone?
I believe it was 40 dBA
I believe it had 10 missed calls 0 outgoing
calls
Agent As info seems to be more reliable
70
Shortcomings in the Current Approach

• An explicit assumption model of different agents
  must be pre-defined for the context broker
• I.e., what does Agent X believe and why?
• Changes in other agents reasoning
  implementations will require modifications to be
  made in the the brokers assumption model.

71
Solution Argumentation

• Argumentation is the process of weighing various
  arguments, pro and con, for the truth of an
  assertion, and arriving at a truth value for the
  assertion
• Idea through argument-based dialogs, the context
  broker dynamically acquires assumptions from
  other agents
• Minimizes the built-in model of other agents
  assumptions

72
Fatio an FIPA Argumentation Protocol
By Peter McBurney and Simon Parsons (AAMAS 2004)
73
An Example

• Two context brokers have distinct beliefs about
  the presence of a meeting organizer
• B1 believes the organizer is present
• B2 believes the organizer is absent
• To resolve inconsistency, one must convince the
  other why its belief is more reliable
• B2 sends messages to question B1 about its belief
• B1 replies with justifications
• B2 accepts B1s assertion if the received
  justifications are consistent with its own belief

74
An Argument-based Dialog
75
Summary

• CoBrA uses rule-based logical inference to
  support context reasoning
• It has a hybrid system architecture that uses
  different kinds of reasoners (Jena, Jess, Prolog)
• Contributions
• Using ontologies to interprete context
• An assumption-based approach to resolve
  inconsistent information
• Implemented an FIPA argumentation protocol using
  an assumption-based reasoner (Theorist)

76
Context Reasoning
Ontologies
Privacy Protection
77
Privacy Protection

• Privacy is about control of information
• Who has access to what information
• Related to security but not equal
• Key issues in a context-aware system
• Users may be unaware of the hidden sensors and
  services that share their private information
• Users cant hide everything from the system
• Even if they can, its undesirable to hide every
  bits of information from the system

78
Policy-Based Privacy Protection

• CoBrA allows users to define policies to protect
  their privacy
• Policies are expressed and reasoned using the
  SOUPA policy ontology
• Policy reasoning is built on the classfication
  feature of a description logic reasoner (Racer)

79
The SOUPA Policy Ontology
80
A Policy Example

• Harry Chens policy
• http//cobra.umbc.edu/ont/2004/05/harrychen-policy
  
• Permits
• Share location info with close friends (Action
  class)
• The recipient is restricted to a list of my
  friends URI
• The target is restricted to the
  LocationContext class
• Forbids
• Share location info with untrusted agents (Action
  class)
• The recipient is restricted to a list of
  untrusted agent URI
• The target is restricted to the
  LocationContext class

81
Policy Reasoning Behavior
82
Policy Reasoning Algorithm(highlight)

• Define an action instant to represent the current
  knowledge sharing action
• Check if the user has a defined policy. If not,
  use a global policy
• Load the ontology descriptions of the action
  instant and the policy into Racer
• Call Racers classification function.
• The action is permitted if the instant is a type
  of PermittedAction class.
• The action is forbidden if the instant is a type
  of ForbiddenAction class.

83
Adjusting Information Granularity

• CoBrA supports information granularity adjustment
• Instead of hiding all information from the
  system, only hide the details of a users context
• Extends the policy reasoning algorithm (see Ch. 6)

84
A Diagram of the Behavior
85
Adjusting Location Granularity

• I dont want others to know the specific room
  that Im currently in, but its okay to tell
  others about the my general location
• Permit to share my location with a granularity gt
  State
• The broker
• isLocated(US) gt Yes!
• isLocated(Maryland) gt Yes!
• isLocated(UMBC) gt Uncertain..
• isLocated(ITE-RM210) gt Uncertain..

86
Adjusting Status Context

• I dont want others to know Im meeting my
  advisor, but its okay to tell others that Im in
  a meeting
• Permit to share my status gt inMeeting
• The broker
• statusOf(Harry,inMeeting) gt Yes!
• statusOf(Harry,meetingColleagues) gt Uncertain
• statusOf(Harry,meetingAdvisor) gt Uncertain

87
Summary

• CoBrA uses policies to protect user privacy
• Policies are expressed using the SOUPA policy
  ontology editing policies are similar to editing
  ontologies
• Contributions
• A DL-driven policy reasoning algorithm based on
  the SOUPA policy ontology
• Showed the use of information granularity
  adjustment in policy reasoning

88
Implementations
89
Feasibility Demonstration

• Show the use of CoBrA can help to reduce the
  efforts in rapidly prototyping pervasive
  context-aware systems
• Show CoBrA can be built on the existing FIPA
  agent and semantic web technologies
• Show CoBrA testing and monitoring tools can be
  integrated into the mainstream IDE environment
• Show new applications can be built on
  resource-limited devices that exploit CoBrA

90
Implemented Prototypes

• EasyMeeting
• CoBrA Demo Toolkit
• CEV ScriptPlay Context Broker
• CoBrA Text Messaging Commands

91
The EasyMeeting Architecture
92
CoBrA Demo Toolkit
http//projects.semwebcentral.org/projects/cobra/
(since April 2004, 400 downloads)
93
CoBrA Demo Toolkit

• Built-in script files
• Examining the temporal reasoning of a context
  broker (intervals, instants)
• Examining the spatial reasoning of a context
  broker (geo-metric, symbolic)
• Examining the policy reasoning of a context
  broker (permitted or forbidden actions, location
  granularity adjustment)
• Examining a brokers ability to detect and
  resolve location inconsistency (assumption-based
  reasoning)

94
CoBrA Eclipse Viewer (CEV)
Inspired by the Java Spider application
http//www.javaspider.org
For exploring the knowledge and user policies
that are stored in the Context Broker for
monitoring the brokers reasoning process.
95
ScriptPlay Agent

• Testing and debugging the internal behavior of a
  context via exchanging ACL messages
• Simulate the roles of sensors, services, or
  context brokers
• Script files are defined in XML

96
CoBrA Text Messaging Commands (CTMC)

• A text messaging interface for mobile users to
  interact with the context broker
• Use SMS to receive and send commands to the
  broker
• Similar to Upoc Dodgeball

SMS
97
CTMC Prototype

• Works eBiquity weekly meetings that are published
  on the web understands the RGB (Research Group
  In a Box) event ontology
• Runs 24/7 on ebiquity.org
• Supported commands
• qMEETING
• qSPEAKER meeting-id
• qINFO meeting-id
• qFOLLOWUP meeting-id email-address

98
A Brokers CTMC Behavior
99
Conclusions
100
Conclusions

• The intelligence of context-aware systems will be
  limited if they are unable to represent and
  reason about context
• Users will abandon the most useful context-aware
  systems if they are unable to control the sharing
  of their private information
• CoBrA is a new architecture that addresses these
  issues

101
Lessons Learns

• CoBrA improves rapid prototyping
• Context-aware systems needs ontology
• Logical inference helps to enable
  context-awareness

102
Rapid Prototyping using CoBrA

• In EasyMeeting, the use of contextual knowledge
  is easy because there is a standard ontology for
  describing various kinds of information
• Efforts required to build a context-aware service
  is reduced because the context broker takes care
  of context acquistion and reasoning

103
Ontologies are Important

• The use of ontologies improves knowledge sharing
  and representation
• OWL can express more rich information than
  programming languages such as Java or C
• OWL provides a uniformed language for knowledge
  representation, context reasoning, knowledge
  sharing, and meta-language definitions.

104
Rule-Based Logical Inference is Better

• Hardwired context interpretation procedures are
  ridge and difficult to maintain
• A rule-based approach separates high-level
  context reasoning from low-level system behaviors
• Sophisticated reasoning behaviors (e.g.,
  arugmentation) can only be built if the logical
  inference of context reasoning is explicitly
  represented

105
Future Works

• Scalability issues in real deployment
• The single-point-of-failure scenario (see Ch.3)
• Explore probabilistic reasoning to complement
  rule-based reasoning in CoBrA
• Understand improve the time complexity
  associated with ontology and logical inference
  (need better tools?)
• Policy related issues
• Editing and modifying policies
• Resolve policy conflicts
• How to enforce policies after some information
  has already been shared?

106
My work adds few more pieces
Social Acceptance
???
Demand

???

Memory
???
Context Modeling
Applications
???
???
???

Battery Life
Business
UI
Context Reasoning
Privacy Protection


Trust
???
Wireless
???

Sensors
???
???
Security

???



CPU Power
Cross Platform

Profit
???
???
???
107
Thank You!