Distributed Learning for Protocol Selection in Multiagent Systems

1
Distributed Learning for Protocol Selection in
Multi-agent Systems

• Vladimir Gorodetsky,
• Intelligent System Laboratory of the
• St. Petersburg Institute for Informatics and
  Automation
• E-mail gor_at_mail.iias.spb.su http//space.iias.spb
  .su/ai/gorodetski/gorodetski.jsp

2
Outline

• Part 1. Experience in distributed learning
  Technology and applications of Data Fusion
  learning MAS
• Part 2. Distributed Learning Multi-agent Systems
• Task statement
• Technology, architecture of the respective
  software tool and basic protocols supporting MAS
  distributed learning technology
• Distributed off-line learning protocol
• Distributed on-line decision making protocol
• Conclusion Basic Problems of Distributed MAS
  Learning and Perspectives of Future Research

3
Outline

• Part 1. Experience in development and use of
  Multi-agent learning technology
• Multi-agent Data Fusion off-line learning system
• Part 2. Distributed learning for protocol
  selection in multi-agent systems
• Task statement
• Technology, architecture of the respective
  software tool and basic protocols supporting MAS
  distributed learning technology
• Distributed off-line learning protocol
• Distributed on-line decision making protocol
• Conclusion Basic Problems of Distributed MAS
  Learning and Perspectives of Future Research

4
Objective of Data Fusion

• DF is a task aiming at making decisions on the
  basis of distributed heterogeneous data sources.
• Formally, decision making in a DF system is
  understood as classification task.
• For example, an objective of a DF system can
  be classification of an object states, assessment
  a class of situation constituted by a number of
  autonomous interacting objects, prediction a goal
  of a situation developing in time, etc.

5
Applications of Data Fusion

• Critical areas of human society security
• security of critical state infrastructures,
  safeguard and restoration of critical enterprises
  like nuclear power plants, electrical power
  grids, etc.
• planning on large scale natural and man-made
  disasters rescue operations, assessment and
  prediction of situations development and resource
  management, mitigation of environmental
  consequences
• prediction for terrorist intents and
  anti-terrorist activity planning
• resource planning and impact assessment in
  large-scale logistics in business, manufacturing,
  
• etc.

6
Task under Research

• Multi-agent data-fusion systems
• Design technology
• Distributed learning DF MAS
• Distributed decision making in DF MAS

7
Research Objective of the Previous Project

• Development of an off-line learning technology
  for multi-agent DF systems solving on-line
  classification task on the basis of distributed
  heterogeneous data sources
• Development and prototyping of a software tool
  supporting the learning technology for
  multi-agent DF systems
• Case studies of applied multi-agent DF system
  provided for off-line learning capabilities.

8
Peculiarities of DF Learning Technology and
Software Tool

• Architecture of DF system Multi-agent
• Software tool Generic Learning System MAS
  Development Kit (MASDK)
• Technology Specialization of Generic Learning
  System carried out in interactive and iterative
  mode by user/users utilizing MASDK.

9
Peculiarities of Learning Procedure

• Data sources are heterogeneous and distributed
• Training and testing procedures are distributed
• Classification procedure is multi-level the
  necessity to combine decisions produced on the
  basis of particular data sources.

10
Interaction of Basic and Training Testing
Agents of DF MAS
Multi-agent Data Fusion System
Host of situation of meta-level components of DF
MAS
Meta-classifier
TT-meta agent 1
TT-meta agent 2
Host R
Base classifier 1
Host 2
Base classifier 2
Base classifier 1
Base classifier 1
Base classifier k
TT-agent 1
Base classifier 2
Base classifier k
Base classifier k
TT-agent 2
Learning DF MAS
11
Architecture of DF System

• Architecture of the source-based components of DF
  MAS

Local data source
User interface
Training and Testing agents (TT-agents)
Data source managing agent
Local (source-based) classification agents of DF
system
Testing
KDD agent
Training
Server (library) of learning methods
To the Meta-classification agent
To the KDD Master
12
Agents in the Host of Data Source

• Data source managing agent
• participates in the distributed design of the
  consistent shared and private components of the
  application ontology
• collaborates with meta-level agents in management
  of training and testing of particular
  source-based classifiers and in forming meta-data
  sample for meta-level training and testing
• supports gateway to databases performing
  transformation of queries from the language used
  in ontology into SQL language.
• KDD agent of data source
• training and testing of source-based
  classification agents of the DF system and
  assessment of the designed classifier quality.
• Classification agents of data source
• a subject of training performed by DF software
  tool. The purpose of this agent is producing
  decisions on the basis of source-based data.
• Server of learning method (not an agent)
• This software component comprises a multitude of
  classes implementing KDD methods, metrics and
  other functionalities.

13
Architecture of DF system

• Architecture of meta-level component of DF MAS

TT-agents of meta-level
Agent-classifier of meta-level
KDD Master agent
Mechanism for decision combining
Meta-classifier
User interface
Meta-level KDD agent
Referee
Server of learning methods
To the Data source managing agent
To the KDD agent
Local classification agents
14
Agents of Meta-level

• Meta-Learning agent (KDD Master)-TT-agent
• Manages the distributed design of DF system
  application ontology,
• Computes the training and testing meta-data
  sample,
• Manages the design of the meta-model of decision
  making.
• Meta-level KDD agent-TT-agent
• trains and tests of the agent-classifier of
  meta-level
• Agent-classifier of meta-level
• (subject of training and testing) combines
  decisions produced by classification agents of
  data sources.
• Data Fusion management agent
• coordinates performance of Agent-classifier of
  meta-level and Meta-level KDD agent both in
  learning and decision fusion.
• Server (library) of KDD methods (not an agent)

15
Protocol of DF Software Tool Agents' Interaction
in Design Technology (IDEFO Diagram)
Sub-protocols of the high level protocol of the
design technology A0. Distributed ontology
design. A1. DEF meta-model design. A2.
Distributed data mining. A4. Monitoring of
arrival of new data to data sources. A5. Data
Fusion (Distributed decision making.
16
Meta-models of data fusion

• Meta-model of combining decision, that is a
  structure according to which decisions of
  particular classifiers are combined.
• Classification tree is a meta-structure according
  to which multi-class decision making procedure is
  reduced to a number of binary classification task
  
• Meta-model of training and testing data, which
  presents splitting data of each particular data
  source into subsamples and allocation them to
  base-level classifiers of particular data source

17
Meta-model of Combining Decisions Hierarchy of
Tasks
To DF system meta-level classifier
To DF system meta-level classifier
To DF system meta-level classifier
Meta-level classifier of source
Base classifier 2
Base classifier 1
Base classifier k
...
Local database (database of source)
Variant 2
18
Distributed data management

• Distributed data management that is allocation
  training and testing data sets for learning
  particular classifiers management by computation
  of meta-data for upper level example-based
  learning, etc. These tasks are solved through
  using in DF system special agents operating on
  source-located components and meta-level
  component of DF system.
• These agents solve the task in question through
  special negotiation protocol under management of
  local source and meta-level analysts.

19
Classification tree
Example of classification tree
20
Distributed Design of Distributed Ontology
Meta-data manager
Data Source management agent
Data Source Manager
Data Source Manager
Ontology-based meta-model of Data sources
.
Data Source Manager
Data Source Manager
21
Problems to Be Solved in Distributed Application
Ontology Design

• Providing for monosemantic understanding of
  terminology used in data specification by
  distributed analysts
• Solution of the entity instance identification
  problem
• Providing consistency of data representation (in
  case if the same attributes are presented
  differently in different data sources)
• Providing a gateway between ontology and
  distributed databases accessibility making
  possible interaction between ontology and
  distributed databases, and several other tasks.

22
IDEFO Diagram of the Protocol for Distributed
Design of Distributed Ontology
Sub-protocols of on-line Protocol Selection A0.1.
Design and providing for consistency of naming of
the set of ontology notions. A0.2. Design of
coherent measurement scales and value domains of
entity attributes. A0.3. Tuning the gateway for
database access. A0.4. Application ontology
edition. A0.5. Design of the keys for entity
instance identification.
23
Monosemantic understanding of terminology

Monosemantic understanding of terminology among
DF system components is provided by shared
vocabulary used by DF system distributed entities
for communication. This excludes different
naming of the same entities and their properties
in different sources, and equal naming of
different entities within different data sources
thus providing integrity and consistency of
shared vocabulary. Protocols Supports
distributed collaborative design of coherent
ontology by distributed analysts.
24
Entity Instance Identification Problem
Explanation of the Entity Instance Identification
Problem
Data Source 3
Data Source 2
Data Source 1
25
Consistency of Data Representation

• Let X be an attribute in application ontology
  that is measured differently in different
  sources.
• In the shared component of application ontology,
  the type and the measurement unit of the
  attribute X are determined. Selection of
  attribute X specification within shared part of
  application ontology is made by experts during
  their negotiation according to a synchronization
  protocol.
• In all the sources where X is present,
  expressions are determined for this attribute,
  through which it can further be converted into
  the same scale in all the sources.
• This allows using the values of attributes
  on the meta-level regardless of the data source
  from which they originated.

26
Approaches for Combining Decisions

• 1. Meta-classification scheme for combining
  decisions
• (based on stacked generalization)

Meta-learning level
Result Meta-classifier
Algorithm for learning meta-classifier
Meta-classifiers training and testing data
("meta-data")
Testing data
Base Classifier 2 to be learned
Base Classifier 1 to be learned
Base Classifier k to be learned

Algorithms for Base classifier learning
Data
Legend
KDD algorithms
Resulting classifiers
27
Approaches for Combining Decisions-2
2. Competence-based Approach
Referee 1
Decision of the most competent classifier
Referee 2
..
Partition of learning data for classifier training
Partition of learning data for referee training
..
Selection of the most competent classifier and
its decision
Referee K
Correctly classified examples
Training and testing data
Erroneously classified examples
28

• Part 1. Experience in distributed learning
  Technology and applications of Data Fusion
  learning MAS
• Part 2. Distributed Learning Multi-agent Systems
• Task statement
• Technology, architecture of the respective
  software tool and basic protocols supporting MAS
  distributed learning technology
• Distributed off-line learning protocol
• Distributed on-line decision making protocol

29
Research Objectives

• Development of a technology for the design and
  implementation of Multi-agent learning systems
• Development of a Software tool prototype
  supporting Multi-agent learning system
  technology
• Validation of the technology and software tool on
  the basis of application prototyping.

30
Learning Tasks Peculiar to MAS

• 1. Learning of individual agents.
• 2. Adaptation and Learning in Multi-agent systems
  -individual learning to act in the environment in
  presence of other agents.
• 3. Learning of coordination - acquire
  coordination skill by learning.
• 4. Distributed learning-how to learn in
  collaborative way as a group.

31
Learning Task under Research

• It is a mixture of two tasks
• 3. Learning of coordination - acquire
  coordination skill by learning.
• 4. Distributed learning-how to learn in
  collaborative way as a group.

Objective is to acquire coordination skill by
learning in collaborative way as a group
32
Task under Consideration
To off-line learn of MAS to provide it an ability
to on-line select the best coordination protocol
from predefined set of available protocols.

• Examples of Applications
• RoboCup Off-line training of virtual players to
  on-line select a scenario of collective behavior
  for given set of standard situations.
• RoboCup Rescue Off-line training of rescue
  teams of different destinations and their
  cooperation.

33
Task Statement
Given Interpreted training and testing data
samples of agents Task Off-line distributed
sample-based training in a collaborative way as a
group Find Distributed algorithm (protocol)
for on-line selection of the best coordination
protocol from predefined set of
available protocols

34
Distributed Learning and Distributed Protocol
Selection Main Ideas

• Off-line Distributed learning in a collaborative
  way as a
• group
• MAS is added Training and Testing (TT-)
  agents that operate according to a number of
  protocols and manage distributed learning of
  on-line selection of coordination protocol.
• On-line Distributed Protocol Selection
• Each agent on-line perceives available
  information from environment and, probably, from
  other agents. It can initiate negotiations in
  order to propose to change the protocol being
  executed and select new one. These negotiations
  have to be performed according to a protocol.
  This protocol is the subject of learning.

35
Task Statement To Learn of MAS to On-line Select
the Best Coordination Protocol
Interactions corresponding to Distributed On-line
Protocol Selection Information about available
protocols
Available Protocols
36
Conceptual View of Multi-agent Learning System
Multi-agent Data Fusion System
Host of situation of meta-level components of DF
MAS
Meta-level auxiliary (decision combining) agent
TT-meta agent 1
TT-meta agent 2
Off-line Learning MAS
37
General View of DF Learning MAS Technology
Multi-agent software tool for distributed
learning
Protocols
MAS DK
Generic multi-agent learning system
User interface
Training and Testing agents ("TT-agents) of
applied MAS
Distributed training and testing protocol
Local Basic agents
Local Protocol selection mechanisms
Local Basic agents
Local Protocol selection mechanisms
Local Basic agents
Local Protocol selection mechanisms
Meta-level Protocol selection mechanism
38
Protocol Supporting Distributed Learning
Technology (IDEFO Diagram)
Sub-protocols of DL technology protocol A.1.
Protocol for distributed application ontology
design. A.2. Protocol for the design of
"decision making structure". A3. Distributed
Learning protocol A4. Protocol used for on-line
selection of the coordination protocol
39
Distributed Learning Protocol
Sub-protocols of DL Protocol A3.1. Allocation of
data for training and testing of the basic
(local) agents of Applied MAS. A3.2. Training and
testing of the basic (local) agents of Applied
MAS. A.3.3. Management of training and testing
of the Protocol selection meta-agent . A3.4.
Computation of data for training and testing of
Protocol selection meta-agent. A3.5. Training and
testing of Protocol selection meta-agent. A3.6.
Preparation of the Applied MAS for use.
40
Protocol Supporting Distributed On-line Protocol
Selection
Sub-protocols of on-line Protocol Selection
A4.1. Data source monitoring. A4.2. Basic
agents' Decision making management A4.3. Data
preparing A4.4. Making decisions by basic agents
A4.5. Analysis of available information used for
decision making A4.6. Protocol selection
management A4.7. Meta-data preparing A4.8.
Decision making by meta-level protocol selection
agent A.4.9. Dissemination of information within
MAS about selected protocol of behavior
41
Conclusion Basic Problems and Perspectives of
Future Research

• The main problems of MAS learning design using
  software tool concern development of design
  protocols providing interactions of Generic
  Learning System, components of tool kit and
  user(s) performing design and implementation of
  MAS provided for learning capabilities.
• The future research intends to focus on the
  off-line MAS learning to on-line select the best
  protocol specifying collective behavior of MAS.
• Basic ideas of how to solve the above task were
  verified by prototyping of DF software tool and
  validated on the basis of several case studies.

42
Contact data

• For more information and related publications
  please contact
• E-mail gor_at_mail.iias.spb.su
• http//space.iias.spb.su/ai/gorodetski/gorodetski.
  jsp