Systmes MultiAgents

1
Systèmes Multi-Agents

• Conception
• Applications
• J. Ferber, "Les systèmes multi agents",
  InterEditions, 1995
• http//www-poleia.lip6.fr/drogoul/cours/links.htm
  l

2
Part I - Conception

• 1. Motivations et origines
• 2. Problèmes et definitions
• 3. Le principe dinteraction
• 4. Larchitecture blackboard

3
1. - Motivations et origines

• Systèmes actuels unicité de l'expert trop
  souvent considérée
• se rapprocher de la réalité décisionnelle
• faire apparaître la multiplicité des experts et
  la multiplicité des relations entre experts
  (coopération, compétition, négotiation,)
• du décideur individuel aux réseaux de décideurs
• population d'agents autonomes en interaction
• métaphore des organisations
• on met l'accent sur l'interaction

4
1. - Motivations et origines

• Première définition
• SMA un système dans lesquels des agents
  artificiels opèrent collectivement et de façon
  décentralisée pour accomplir une tâche.
• Ces entités peuvent être implantées sur un
  support physique ou logique (entités matérielles
  ou immatérielles)

5
1. - Motivations and origins

• 1.1. Evolution of the theory of mind
• 1.2. Limitation of classical AI
• 1.3. Evolution of the computer programming
  paradigm

6
1.1. - Evolution of the theory of mind

• Development in the 70th of the theory of mind
  which postulate that
• Intelligence is relying on individual competences
  ability to interact with a physical and social
  environment (eg perceive and communicate)
• Reasoning does not resume to applying an a priori
  fixed sequence of expert rules but rather imply a
  collection of concurrent, heterogeneous and
  dynamically evolving processes
• Two simultaneous and complementary trends
  Minsky - The Society of Mind / Vygotsky - The
  Mind in Society

7
Minsky - The Society of Mind

• A useful metaphor to think of intelligence is to
  consider a large system of experts or agencies
  that can be assembled together in various
  configurations to get things done
• Minsky said,  ...each brain contains hundreds of
  different types of machines, interconnected in
  specific ways which predestine that brain to
  become a large, diverse society of partially
  specialized agencies 
• Cognition is a distributed phenomenon
• Minsky 85

8
Vygotsky - The Mind in Society

• The mind in society the origins of individual
  psychological functions are social
• Every high-level cognitive function appears
  twice first as an inter-psychological process
  and only later as an intra-psychological process
  
• The new functional system inside the child is
  brought into existence in the interaction of the
  child with others (typically adults) and with
  artifacts
• As a consequence of the experience of
  interactions with others, the child eventually
  may become able to create the functional system
  in the absence of the others
• Vygotsky 78

9
The distributed cognition paradigm

• Cognition is no more envisaged as a purely local
  and isolated information processing but rather
  considered as
• Context-dependent
• Temporally distributed past reasoning may
  influence current processings
• Involving cooperation and communication with the
  physical and social environment
• Dynamically evolving as the result of its
  processings and interactions
• Hutchin 95

10
1.2. - Limitation of classical AI

• Considering problems of increasing complexity
• Problems that are physically and functionally
  distributed
• Problems that involve heterogeneous data and
  expertise 
• Problems in which data, information and knowledge
  is uncertain, incomplete and dynamically evolving
  
• Problems that can not be tackled by global
  problem solving methods

11
Physical distribution
From Miksch 96
12
Physical distribution
13
Functional distribution

• Task example patient monitoring
• Sensing, interpretation and summarization of
  patient data 
• Detection, diagnosis, and correction of critical
  situations
• Construction, refinement and revision of
  short-term and long-term therapy plans
• Control and supervision of monitoring devices
• Explanation of observations, diagnoses,
  predictions, and therapies based on the
  underlying anatomy and physiology

14
(No Transcript)
15
Heteregoneus knowledge and expertise

• Various type of knowledge
• Clinical Knowledge of common problems, symptoms
  and treatments
• Biological knowledge of anatomy, physiology and
  pathophysiology
• Knowledge of fundamental physical models and
  fault conditions
• Various types of expertise
• Patient monitoring a team work involving
  members with complementary tasks and skills,
• which is most often staffed with new or
  inexperienced physicians and nurses

16
Complexity requires a local view

• Complex system behaviour often emerge as the
  dynamic interaction between
• The system components
• The system as a whole with the environment
• The environment with the individual components
• The resulting dynamics, at the system level, may
  influence the environment which in turn will
  influence the component dynamics
• Even when a clean formulation is possible,
  analytical approaches often involves concurrent
  expansion of recursive functions

17
Complexity as dynamicity of interactions
System
Environment
Comp.2
Comp.1
Comp.N
18
Decentralization as an alternative view

• An alternative to the classical approach based on
  a single monolithic system is the divide and
  conquer principle where a phenomenon is viewed as
  composed of a set of related and interacting
  sub-phenomena
• The whole phenomenon is then described by several
  (hererogeneous) models accounting for its
  component behaviours, together with several
  (heterogeneous) models accounting for their
  interactions
• Instead of designing a single  heavy 
  all-purpose system, this approach creates
   light , case-based, narrow-minded units that
  have clearly identified objectives and background
  information necessary to successfully achieve
  their objectives 

19
Decentralization as an alternative view

• While in the first case the model of the whole
  phenomenon to be regulated is contained in a
  single unit, in the second case a number of
  partial models of the phenomenon are contained in
  several units
• Each of these units can regulate just a single
  part of the entire phenomenon
• A global view for the whole phenomenon simply
  emerges from the structured interaction of the
  partial units

20
Complexity to deal with complexity

• Main advantages
• A complex global model usually depends on several
  parameters that are difficult to identify and to
  measure 
• Models with higher degree of approximation with
  respect to the real phenomenon may be derived,
  because the decomposition allows to develop
  sub-models for very specific contexts
• Alternative sub-models may be employed for
  describing the same phenomenon (competitive
  models)
• Since the sub-parts of the phenomenon may
  overlap, the actions that each unit undertake to
  regulate these sub-parts may conflict fusion
  and/or negotiation mechanisms are then required

21
1.3. - Evolution of the computer programming
paradigm

• Toward more effective design and re-use
• Looking for high specification levels
• Looking for fault tolerant design
• Looking for more expressive representation, more
  accurate operative perspective
• Toward increased man-machine communication
  capabilities

22
(No Transcript)
23
Towards autonomous systems

• Complexity increases in such a way that the
  expression or prevision of all possible cases
  becomes prohibitive.
• There is a shift, from a compositionality
  hypothesis to an autonomy hypothesis of the
  system components
• This suggests to design entities fitted with own
  laws, to augment their capacity of internal
  adaptation, and thus of autonomy and
  autoorganisation
• Courant 94

24
2. - Issues and definitions

• An agent is a computer system situated in some
  environment, that is capable of autonomous action
  in this environment in order to meet its design
  objectives
• Autonomy the agent should be able to act
  without the direct intervention of humans (or
  other agents), and should have control over its
  own actions and internal state
• Multi-agent system a set of agents interacting
  in the exploitation of a common environment,
  toward a common global goal

25
By definition

• Multi-Agent Systems are such that
• Each agent has incomplete information or
  capabilities to solve a problem
• There is no global system control, nor any global
  view of the system given to any single agent
  (except the human one)
• Computation is asynchronous
• In addition, mobile agents may be designed, that
  have the ability to traverse a computer network
  accumulating information from several sites (eg
  online monitors, nurses reporting stations,
  patient records, doctors at remote locations)

26
Designing styles

• Deux aspects à traiter
• Aspects microscopiques (orientés agent)
• comment construire un agent capable d'agir de
  manière autonome,
• quelles sont ses représentations et ses
  comportements
• Aspects macroscopiques (orientés système)
• comment construire une organisation capable
  d'agir de manière coopérative
• quels sont ses moyens de communication et de
  coordination

27
Designing styles

• A multi-agent system may be
• Open the set of agents is not predefined, new
  agents may be created on demand
• Closed the set of agents is fixed in advance
• Homogeneous all agents obey the same model
• Heterogeneous agents fitted with different
  models, operating at various levels of grain, may
  co-exit
• Hybrid human and non-human agents may
  collaborate  anonymously  to perform the task
  at hand

28
Agent models
Knowledge base
Control unit
cooperative planning layer
social models
local planning layer
mental models
Knowledge Abstraction
behaviour- based layer
world models
Perception
Action
Environnement
29
Agent Models Cognitive

• 1. Contrôle
• (buts, plans, tâches)
• 2. Expertise du domaine
• 3. Connaissances
• sur soi-même
• et sur les autres (croyances)
• 4. Communications

30
Agent Models Réactive

• 1. Contrôle
• 2. Comportements
• 3. Perception
• 4. Reproduction

31
Agent behaviour

• Do forever
• Receive observation (percept)
• Update internal model (beliefs)
• Deliberate to form intentions
• Use intentions to plan actions (means-end
  reasoning)
• Execute plan
• Two essential points
• The agents have bounded resources (including
  time)
• The world changes while deliberating, planning
  and executing and this can result in intentions
  and plans being invalidated

32
Agents as intentional systems

• Predominant approach treat agents as intentional
  systems that may be understood by attributing to
  them mental states such as
• The beliefs that agents have
• The goals that agents will try to achieve
• The actions that agents perform
• The ongoing interaction

33
3. - The interaction principle

• Interaction
• Communication
• Task allocation
• Cooperation
• Coordination of actions
• Resolution of conflict

34
Modes de Communication

• Communications directes (ou explicites)
• l'échange direct est réalisé volontairement en
  direction d'un individu ou groupe d'individus
• communication par partage d'informations
• les agents lisent et déposent une information sur
  une zone de données commune (eg tableau noir)
• communication par envoi de messages (notion de
  protocole)
• communication point à point (téléphone)
• communication par diffusion (broadcast)
• Communications indirectes (ou implicites)
• les agents laissent des traces (signaux) de leur
  présence ou de leur action qui sont perçues par
  d'autres agents
• lenvironnement propage (et éventuellement
  déforme) les signaux déclenchés par la
  réalisation dune action cela entraîne des types
  d'échanges limités et permet de ne pas avoir à
  déterminer précisément le rôle de chaque individu
  dans le traitement collectif (ex les objets dans
  l'environnement émettent des signaux ou des
  champs de potentiels guidant les agents)

35
Commmunication types
Message passing
Message
Information sharing
Infor- mation
36
Messages et Acteurs

• Le modèle acteur centré sur le principe du
  message
• les acteurs sont réactifs, ils mettent en oeuvre
  un traitement en réponse à un message reçu d'un
  autre acteur, et sont capables d'envoyer des
  messages à d'autres acteurs.
• Comportement
• exécuter une action
• envoyer un message à lui-même ou à d'autres
  acteurs
• créer d'autres acteurs
• spécifier un comportement de remplacement
• Fonctionnement
• à réception d'un message, vérifie si le message
  matche le comportement de l'acteur
• si OK, exécute l'action correspondante
• principe de continuation désigne l'acteur auquel
  envoyer le résultat du message
• peut éventuellement déléguer à un autre (proxy)

37
Agent situé ou communiquant

• Agent purement situé
• l'environnement possède une métrique,
• les agents sont situés à une position dans
  l'environnement qui détermine ce qu'ils
  perçoivent
• ils peuvent se déplacer
• il n'y a pas communications directes entre
  agents, elle se font via l'environnement
• Agent purement communiquant
• il n'y a pas d'environnement au sens physique du
  terme,
• les agents n'ont pas d'ancrage physique,
• ils communiquent via des informations qui
  circulent entre les agents

38
Situé ou Communiquant

• Société de Fourmis
• La résolution du problème s'inscrit dans
  l'environnement physique et dans l'organisation
  physique trouvée par les agents
• Réseau de décideurs
• la résolution du problème s'inscrit dans une
  structure conceptuelle et dans les modes de
  coopération enre agents

39
Agents Réactifs Situés (exemple)

• Problème un ensemble de robots doivent trouver
  du minerai et le rapporter à la base

40
Agents Réactifs Situés (exemple)

• Règle Explorer
• si je ne porte rien et je ne perçois aucun
  minerai et je ne perçois aucune marque
• alors j'explore de manière aléatoire
• Règle SuivreMarque
• si je ne porte rien et je ne perçois aucun
  minerai et je perçois une marque
• alors je me dirige vers cette marque
• Règle Trouver
• si je ne porte rien et je perçois du minerai
• alors je prends un échantillon de minerai
• Règle Rapporter
• si je porte du minerai et je ne suis pas à la
  base
• alors retourner à la base et déposer une marque
• Règle Déposer
• si je porte du minerai et je suis à la base
• alors déposer le minerai

41
Task allocation

• Objectives
• Decompose the problem into sub-problems
• Allocate the tasks to agents, according to their
  competences and specialities
• Re-organize during execution if necessary
• Approach
• Static the allocation is performed a priori by
  the system designer 
• Dynamic the allocation is performed by the
  agents themselves (eg contract net)
• Hybrid the initial allocation my be revised to
  account for changes in the environment (case of
  an open architecture in particular)

42
The Contract Net

• Objective given a task to perform, allocate it
  to the  best  agent, knowing the task
  characteristics, its eventual realization
  constraints, and considering the agent potential
  and effective capabilities to succeed 
• 3 main steps
• Sending of a call for a task / reception of the
  proposals by the contacted agents
• Selection of the best proposals / establishment
  of the contract(s) / reception of the result(s)
• Selection / construction of final result

43
The Contract Net
44
Cooperation styles

• Three cooperation styles may be distinguished
  Hoc 96
• Confrontative cooperation a task is performed
  by agents with heteregoneous competencies or
  viewpoints, operating on the same data set the
  result is obtained by fusion the emphasis is on
  competence distribution
• Augmentative cooperation a task is performed by
  agents with similar competencies or viewpoints,
  operating on disjoint subsets of data the
  result is obtained as a collection of partial
  results the emphasis is on data distribution
• Integrative cooperation a task is decomposed
  into sub-tasks performed by agents operating in a
  coordinated way  the result is obtained upon
  execution completion the emphasis is on goal
  distribution

45
Confrontative cooperation
46
Augmentative cooperation
Agent 1
47
Integrative cooperation
48
Coordination of actions

• How to plan and coordinate the actions of several
  agents in order to reach a common goal?
• Two main modes
• Planning (centralized or distributed)
• Opportunistic problem solving

49
Planning

• Centralized planning
• A centralized manager distributes the plans to
  every agent, having the knowledge of their
  competences competencies in task decomposition
  
• Easiest way to maintain consistency of problem
  solving but not too far from classical planning
• Distributed planning
• Each agent produces partial plans and communicate
  them to the other agents or to a mediator
• Issues fuse/synchronize the plans in a
  consistent way avoid duplication of efforts
  conflicts dynamic planning?
• Heavy communication load, high complexity

50
Opportunistic problem solving

• The system  simply  chooses a next action at
  each step, as the one that will allow the best
  progress toward the solution, given the curent
  situation (ie the available data and the
  intermediate state of problem solvng)
• Strongly data-directed, allow rapid refocusing
  (at each control cycle)
• Implies some knowledge of action cost and utility
  

51
Resolution of conflicts

• Several solutions
• Authoritary a supervising agent has the
  authority and knowledge to take a decision
• Mediation a mediator agent knows the various
  viewpoints and tries to solve the conflict
• Negotiation the conflicting agents try to find
  a solution through several negotiation steps

52
The negotiation process

• Main negotiation steps
• 1. A makes a proposal
• 2. B evaluates this proposal, determines the
  resulting satisfaction according to his own goals
  
• 3. if B is satisfied, then STOP
• otherwise B elaborates a counter-proposal based
  on his own goals and constraints
• 4. Go to step 2 with A  and B roles exchanged

53
4. - The blackboard architecture

• A group of human experts is working cooperatively
  to solve a problem, using a blackboard as the
  workplace to develop the solution
• Problem solving starts when the problem and
  initial data are written on the blackboard
• The experts watch the blackboard, looking for an
  opportunity to apply their expertise to the
  developing solution
• When an expert finds sufficient information to
  make a contribution, he records the contribution
  on the blackboard, hopefully enabling other
  experts to apply their expertise
• This process continues until the problem has been
  solved

54
The blackboard architecture
Level N
Solution
Hypotheses
Level 2
Level 1
Data
Blackboard
55
Knowledge sources


56
KS Knowledge Sources / Specialists

• Each KS is a specialist at solving certain
  aspects of the overall problem the KSs are all
  independent once a KS finds the information it
  needs on the blackboard, it can proceed without
  any assistance from others
• Additional KSs can be added, poorer performing
  KSs can be enhanced, and inappropriate KSs can be
  removed, without changing any other KSs
• It does not matter whether a KS implements
  rule-based inferencing, a neural network,
  linear-programming, or a procedural simulation
  program. Each of these diverse approaches can
  make its contributions within the blackboard
  framework each KS is hidden from direct view,
  and seen as a black box from the outside

57
Organizing the BB

• When the problem at hand is complex, there is a
  growing number of contributions made on the
  blackboard, so that quickly locating pertinent
  information may become a problem 
• A common solution is to subdivide the blackboard
  into regions, each corresponding to a particular
  kind or level of information
• Other criteria like information relevance,
  criticality or recency can be used

58
Event-based activation

• The KS do not interact directly they  watch 
  the blackboard, looking for an opportunity to
  contribute to the solution
• Such opportunities arise when an event occurs (a
  change is made to the blackboard) that match the
  KS condition part some specialists may also
  respond to external events, such as the ones
  produced by perceptual units
• In practice, rather than having each KS scan the
  blackboard, each KS informs the system about the
  kind of events in which it is interested the
  system records this information and directly
  considers the KS for activation whenever that
  kind of event occurs

59
Incremental / opportunistic problem solving

• Blackboard systems operate incrementally KSs
  contribute to the solution as appropriate,
  sometimes refining, sometimes contradicting, and
  sometimes initiating a new line of reasoning 
• Blackboard systems are particularly effective
  when there are many steps toward the solution and
  many potential paths involving those steps
• By opportunistically exploring the paths that are
  most effective in solving the particular problem,
  a blackboard system can significantly outperform
  a problem solver that uses a predetermined
  approach to generating a solution

60
Control

• A control component that is separate from the
  individual KSs is responsible for managing the
  course of problem solving
• The control component can be viewed as a
  specialist in directing problem solving, by
  considering the overall benefit of the
  contributions that would be made by triggered KSs
  
• When the currently executing KS activation
  completes, the control component selects the most
  appropriate pending KS activation for execution

61
The agenda-based control mechanism

• Every time a KS action is executed, the changes
  to the BB are described in terms of BB event
  types these event descriptions are passed to
  the BB monitor, which identifies the KSs that
  should be trigggered (the ones that declared
  interested in this type of event)
• If a KS precondition is found to be satisfied,
  the KS is said to be activated and its action
  component placed in the agenda
• All possible actions are placed onto the agenda
  on each cycle the actions are rated and the most
  highly rated is chosen for execution
• In addition, focus decisions may be used to rate
  and schedule KS activation 

62
Focus of attention

• In the simple blackboard model, the scheduler
  chooses a KS and then the KS executes using the
  context (BB elements) appropriate for it
• In some systems, instead of directly choosing a
  KS, the scheduler chooses first a context
  (location in the BB) only then the KSs for
  which that context is appropriate are considered
  enabled and are executed
• Control decisions thus operate on the condition
  and action parts of the KSs
• Typically, the focus of attention will be an
  event chosen from the event list

63
BB control at a glance
Knowledge Sources

Events
KSIs
64
(No Transcript)
65
Adding a control blackboard

• The BB1 blackboard framework introduced the
  notion of control blackboard. The key idea was to
  store the control strategy on the blackboard
  itself and to build KSs capable of modifying it.
  In this way the system could adapt its activity
  selection to better suit the current situation
• Example control plan
• Prefer KS whose actions occur at successive
  domain levels
• Start with KS whose action occur at a given
  outcome level
• Prefer KS triggered on recent problem-solving
  cycles
• At this step, prefer this KS

66
Adding a control blackboard

• The purpose is to build a control plan on the
  control BB the solution elements for this
  control problem are decisions about what actions
  are desirable, feasible, and actually performed
• To this end, control KS exploit, generate and
  modify the solution elements placed on the
  control blackboard, under control of a scheduling
  mechanism
• The potential activities of every domain and
  control KS are recorded on the same agenda, so
  that the most prioritary activity can be chosen
  by the scheduler

67
Systèmes Multi-Agents

• Conception
• Application

68
Part II - Application

• Patient Monitoring
• 1. Monitoring as a physically distributed problem
  
• 2. Monitoring as a (distributed) cognition issue
• 3. Monitoring as a negotiation problem

69
1. - Monitoring as a physically distributed
problem

• Dimitrios G. Katehakis et al. A Distributed,
  Agent-Based Architecture for the Acquisition,
  Management, Archiving and Display of Real-Time
  Monitoring Data in the Intensive Care Unit,
  Technical Report FORTH-ICS / TR-261
• http//www.ics.forth.gr/ICS/acti/cmi_hta/publicati
  ons/technical_reports/tr261/ICU.html
  

70
Intensive Care Unitsa physically distributed
environment
71
Intensive Care Unitsa physically distributed
environment

• Many variables
• Continuous measurements of electrocardiogram,
  central venous pressure, systemic arterial
  pressures, cardiac output, urine output,
  pulmonary arterial pressures, blood gases, and
  mixed venous saturation
• Measurements made by the ventilator itself
  respiratory rate, tidal volume, peak inspiratory
  pressure, average airway pressure, spontaneous
  minute volume, lung mechanics, oxygen
  consumption, and metabolic rate
• Many interaction / monitoring needs

72
(No Transcript)
73
(No Transcript)
74
System Architecture

• Two types of agents the acquisition agent and
  the monitoring agent. Acquisition agents perform
  data acquisition and feed data to monitoring
  agents, who facilitate data visualization and
  storage

75
Agent s role

• The acquisition agent collects data either from
  patient connected sensors or from clinical
  information systems
• Acquired data are kept temporarily on a local
  data store, until they are transmitted to the
  appropriate monitoring agents
• An acquisition agent may have a number of input
  and output channels, each of which can be
  dedicated to a different monitoring agent. The
  acquisition agent is therefore communicating with
  several monitoring agents simultaneously
• The monitoring agents receive data, which are
  stored temporarily in a data repository and are
  visualized through a Graphical User Interface
  (GUI)

76
Introducing cognition agents
77
2. - Monitoring as a (distributed) cognitive issue

• GUARDIAN -- A prototype intelligent agent for
  monitoring and therapeutics in intensive care
• Barbara Hayes-Roth
• Knowledge System, Laboratory at Stanford
  University
• http//ai.eecs.umich.edu/cogarch2/authors/bhayes-r
  oth.html

78
Monitoring as a (distributed) cognitive issue

• The perception - cognition - action problem
• A compromise to be reached between the quality
  and rapidity of reasoning
• Sacrifice the quality of a solution for one that
  meets the deadline
• A quick action of less quality will push off the
  deadline far enough so that a quality solution
  can be found
• Interleave reactive and cognitive behaviours

79
Architecture

• Three independent sub-systems cognition,
  perception and action, which
• Operate concurrently and asynchronously
• Communicate through a globally accessable
  communication interface which asynchronously
  relays data among limited size I/O buffers
• the system interact concurrently with subsets of
  the environment,
• thereby increasing performance,
• and reducing the overall complexity each
  subsystem must be able to deal with

80
Cognitive system
Perceptual input / Cognitive events
Action parameter/filters
Perceptual input/filters
Fast reflex arcs
Perception agents
Action agents
Perceptual input
Action parameter
Interactive displays
Sensors
Actuators
I/O
Patient, ventilator, human users,
81
Perception agent s role interpretation

• The sensor agent s role is to acquire a given
  type of signals, transduce it into an internal
  representation, and holds the results in its I/O
  buffer
• The perception agent s role is to retrieve the
  sensor agent information, to analyze and
  interpret it and transmit the results to the
  Communication Interface
• Example peak inspiratory pressure
• value  high 
• trend  rising 
• relevance to ongoing reasoning tasks  not
  relevant  
• priority  high 

82
Perception agent s role focus of attention

• The perception agent s retrieves the sensor
  agent information at some given rate, according
  to given filters
• Rate and filter information is transmitted by the
  cognitive system, according to the current
  reasoning state
• The system is therefore provided with focus of
  attention capabilities
• The more important the data, the more often the
  system will see it
• Conversely, as the buffer size is limited, if the
  cognitive system does not look at the buffer
  often enough, perceptual information may be lost

83
Action agent s role action

• The action agent s role is to
• Monitor its input buffer, retrieve intended
  actions, and translate them into executable
  programs of actuator commands
• Control the execution of these programs by
  sending successive commands to its actuator at
  appropriate times 
• Example action dynamically adjust the
  ventilator s settings
• The action agents relieve the reasoning system of
  the computational burden of managing the
  low-level details of action execution

84
Action agent s role user interaction

• The action agent s role is also to communicate
  with the external users, in order to
• Recommend other interventions to correct
  diagnosed problems or avoid predicted problems
• Give explanations about the system s current
  monitoring strategy, its reasoning about some
  particular problem, and the biological and
  physical phenomena underlying the patient s
  condition.

85
Coordination between perception, reasoning and
action

• The perception, reasoning, and action systems
  work concurrently, in a continuous way
• Information from the environment is perceived
  continuously, even while the system is engaged in
  computationally expensive reasoning tasks
• The system is guaranteed to perceive any critical
  events that occur
• Conversely, the system reasons continuously,
  regardless of the rate of incoming events. Thus,
  it is guaranteed to complete a critical reasoning
  task without interruption, unless it decides to
  attend to a more critical new event

86
Coordination between perception/reasoning and
action

• Fast reflex reactions occur across
  perception-action arcs and allow perception to
  drive action directly
• For example, Guardian might automatically sound
  an alarm and deliver a simple explanation
  whenever perceived values of key physiological
  parameters enter critical ranges
• Comparatively slow cognitive reactions involve
  all three systems, with cognition mediating
  actions in response to perception

87
Cognitive system s role

• The role of the cognitive system is twofold
• To interpret perceived information from the
  environment, perform the needed reasoning tasks
  and decide what actions to perform (several
  reasoning agents)
• To construct and modify dynamic control plans to
  coordinate the perception, reasoning, and action
  tasks (one control agent)
• These tasks are performed by agents operating
  according to the blackboard model of control

88
Cognitive system s Architecture
89
Control agent

• The control agent comprises 3 components that run
  sequentially
• The agenda manager uses current perceptual and
  cognitive events and the current control plan to
  identify and rate possible reasoning operations
  it records them on the agenda buffer
• The scheduler takes information from the agenda
  and uses the control plan to select the operation
  that best matches the current plan it records
  that operation on the next operation buffer it
  may also decide to interrupt the agenda
  management to give priority to critical
  operations
• The executor executes the chosen operation,
  producing changes to the global memory new
  perceptual preprocessing parameters or intended
  actions in output buffers new reasoning results
  for ongoing tasks or new control decisions

90
Controller agent cycle
Scheduler
Agenda Manager
Executor
Reasoning Agents
Perception/Action Agents
91
Cognitive state

• Holds the control information necessary to drive
  control it is comprised of three buffers
• The event buffer holds current asynchronously
  arriving perceptual inputs and cognitive events
  produced by reasoning
• The agenda holds currently executable reasoning
  operations - those whose trigger conditions are
  satisfied
• The next operation holds the reasoning operation
  to be executed next

92
Cognitive state

• All of these buffers have limited capacity, and
  are exploited according to two criteria
• Best-first retrieval (items that score higher are
  retrieved earlier)
• Worst-first overflow (items that score lower
  overflow earlier)
• defined in terms of four orthogonal attributes
• Relevance to Guardian's current reasoning
  activities
• Importance with respect to Guardian's global
  objectives
• Recency of entering the buffer
• Urgency of processing the item in order to have
  the intended effect (e.g., meet deadlines)
• If too many critical events occur simultaneously,
  they will overflow the buffers

93
The control plan

• A control plan is a temporal pattern of control
  decisions, each describing a class of operations
  to be performed, under specified constraints,
  during some time period
• It is used to focus the reasoning cycle given a
  strategy to complete the task at hand
• This includes determining which actions have
  priority on the agenda, when the scheduler should
  interrupt, and what the perceptual filters should
  contain
• The only changes to the control plan are those
  made by the control KS, and hence were determined
  necessary by the control plan and environment at
  that time

94
Cognitive systems Architecture
Control KS
Reasoning KS
Control agent
95
Example Control Plan

• Example plan
• Respond to critical events
• Monitor all parameters for changes
• D 10D 2D
• Time
• According to the first decision, Guardian decides
  to respond to critical events. With the second
  decision, it decides that the perceptual
  preprocessor should send new values for patient
  parameters only when their values change by a
  threshold percentage. These decisions remain in
  effect (with some changes in preprocessing
  threshold) throughout the given period of time

96
Example Control KS

• Name Urgent-Reaction
• Trigger Critical observation, O
• Action Record control decision with
• Prescription Quickly react to O
• Criticality Criticality of O
• Goal Diagnosed problems related to O are
  corrected
• This operation is triggered and its parameter, O,
  is instantiated whenever the perception system
  delivers an observation with high criticality
  (such as high PIP - Peak Inspiratory Pressure)
• When executed, it generates a control decision
  favoring  quick  reasoning operations that
   react to  O, and gives it the same criticality
  as O
• The decision is deactivated when its goal is
  achieved, namely that all diagnosed problems
  related to O have been corrected.

97
Resulting control plan

• Modified plan
• Respond to critical events
• Monitor all parameters for changes
• D 10D 2D
• Quickly react to high PIP
• Time

98
Reasoning knowledge a multispecialist approach

• Reasoning knowledge is distributed among several
  task-dependent specialists
• Diagnosis of observed signs and symptoms
• Prediction of patient condition
• Causal inference of precursors and consequences
  of observations, problems, etc.
• Explanation of underlying causal phenomena
• Each of these tasks may be performed using
  associative or model-based reasoning methods

99
Associative methods

• Associative methods use clinical knowledge, apply
  to familiar problems, and give simple
   answers , with minimal explanation
• For example, Guardian responds to an observed
  rise in PIP by quickly diagnosing a
  hypoventilation problem and increasing the
  patient's ventilation
• Having relieved the symptoms and extended the
  hard deadline, it acquires additional data to
  diagnose and correct the specific underlying
  problem (e.g., pneumothorax)

100
Model-based methods

• Model-based methods use biological and
  first-principles knowledge, apply to familiar and
  unfamiliar problems, and give detailed
   answers  with informative explanations
• For example, Guardian can give a
  pathophysiological explanation of its prediction
  that normal minute ventilation of a cold
  post-operative patient will result in low
  arterial partial pressure of CO2
• The patient's low temperature leads to decreased
  metabolic activity in the cells, this results in
  decreased O2 consumption and decreased CO2
  production in the tissue compartment.

101
Model-based methods

• Another example
• Name Find-Generic-Causes
• Trigger Observe condition C
• where C exemplifies Generic-fault F
• Action Find Generic-fault that can-cause F
• Find-Generic-Causes is triggered when C is
  observed
• Upon execution, the action is to look for
  generic-faults that  can-cause  F
• By recording each such cause in the global
  memory, this operation creates internal events
  that trigger other reasoning operations

102
An illustrative scenario

• A scenario illustrating the system capacity to
• Manage moderately important, slowly evolving
  problems (e.g., low temperature and its
  consequences)
• Manage time-critical problems (e.g., high PIP and
  the underlying pneumothorax)

103
A strategy to investigate a patient s low
temperature

• The system is monitoring all patient parameters
  for value changes of a threshold percentage
• It notices the patient s low temperature, a
  non-critical problem but worth investigating
• It makes control decisions that instantiate an
  abstract strategy for investigating this type of
  problem
• a) Diagnose the low temperature
• b) Infer and correct immediate consequences
• c) Predict changes
• d) Infer and act to avoid expected consequences

104

• a) Diagnose the low temperature
• Attribute the low temperature to the patient s
  immediate post-operative status
• b) Infer and correct immediate consequences
• Infer that the patient's PaCO2 is currently low,
  due to the interaction between low temperature
  and normal breathing rate
• c) Predict changes
• Predict that the temperature will rise to high
  and then fall to normal over several hours
• Predict that the PaCO2 will rise to high and fall
  to normal with temperature
• d) Infer and act to avoid expected consequences
• Decide to lower the breathing rate to correct the
  PaCO2
• Plan a series of rate changes correlated with
  temperature to maintain the PaCO2 within an
  acceptable range

105
An unexpected event

• In the course of this strategy, the system
  observes high, rising PIP, indicating a
  potentially life-threatening condition with a
  deadline for corrective action on the order of
  minutes
• A control decision is made that instantiates an
  abstract strategy for correcting critical
  conditions as quickly as possible
• Fast associative reasoning is favoured to
  diagnose and correct the problem

106
A strategy to correct critical conditions

• a) Consider other patient data to diagnose the
  problem class, hypoventilation problem
• b) Advise increasing ventilation so the patient
  will get enough oxygen
• c) Request diagnostic actions auscultation of
  the chest for asymmetric breathing sounds and
  inspection of chest xrays
• d) Diagnose the underlying problem, a
  pneumothorax
• e) Advise insertion of a chest tube to relieve
  the pressure of accumulated air in the chest
  cavity
• f) Predict and confirm the resulting drop in PIP
  
• g) Advise reduction of the breathing rate as
  increased ventilation is no longer necessary
• h) Request a lab test in twenty minutes to
  confirm that blood gases are normal

107
Discussion

• Knowledge representation is complex, even for
  simple and well-kown situations it is difficult
  to ensure the order and time of execution of the
  system modules
• There is a number of coefficients and variables
  to adjust
• The basic functions of sensing, reasoning and
  acting are distributed among local agents
  sensing and acting may be engaged in a pure
  reactive way but as well be influenced by the
  reasoning process under development
• The ratio of intra-agent computation to
  inter-agent com-munication is relatively high
• Consistency is ensured by a global control plan
  influencing what the agents tackle and
  constraining their internal decisions

108
3. - Monitoring as a negotiation problem

• Anthropic agency a multiagent system for
  physiological processes
• Francesco Amigoni, Marco Dini, Nicola Gatti, and
  Marco Somalvico
• Artificial Intelligence in Medicine Journal,
  Vol. 27, n3, 2003
• Special issue  Software agents in health care 

109
The anthropic agency

• Agency a multiagent system as a single machine
  composed of complex components the agents
• Anthropic from the Greek anthropos, namely man
  the agency is employed to model the
  physiological processes of the human being
• An example application the regulation of the
  glucose-insulin metabolism in diabetic patients,
  a process where partially overlapping models of
  glucose level regulation coexist

110
Diabetic pathology

• Glucose is one of the bodys main sources of
  energy
• The body regulates the processes that control the
  production and storage of glucose by secreting
  the endocrine hormone, insulin, from the
  pancreatic B-cells
• Type 1 diabetes is characterized by a loss of
  pancreatic beta-cell (B-cell) function and an
  absolute insulin deficiency
• Since insulin is the primary anabolic hormone
  that regulates blood glucose level, this results
  in the inability to maintain blood glucose
  concentrations within physiological limits

111
Diabetic pathology

• A long time exposition to very high values of
  blood glucose concentration causes serious
  complications to other body organs
  (cardiovascular and renal system, retina)
• Type 1 diabetics require a continuous supply of
  insulin for survival (multiple daily injections
  or a continuous subcutaneous insulin infusion
  guided by daily blood glucose measurements) in
  order to try and keep the glucose concentration
  under control
• Many factors have to be considered to choose the
  current dose of insulin to inject amount of
  food, current glucose concentration value,
  general physical state

112
Diabetic pathology

• In diabetes, there is an uncoupling of blood
  glucose levels and the concentration of insulin
  that prevents the proper regulation of glycemia.
  Instead of a narrow glycemic range, blood glucose
  deviations can extend from hypoglycemia into
  hyperglycemia

113
Diabetic pathology

• The main problem in the diabetic pathology is the
  insulin response when the person eats because it
  is when the glucose concentration reaches the
  maximum value
• Another issue is the effect of physical activity
  on the insulin level
• There is a need to keep constant the glucose
  level to sustain the physical activity
• Conversely, physical activity helps regulating
  the glucose level and keeping more sensitive to
  insulin, therefore being able to function with
  less insulin

114
Variation of the glucose level when eating
115
Purpose of a monitoring system

• To constantly monitor the patient, eg analyze its
  current physiological state
• To inject isulin when needed
• To adjust the insulin amount in order to keep the
  glucose and insulin concentrations as close as
  possible to the concentrations of a normal person
  

116
System architecture

• Three groups of agents working in an asynchronous
  way knowledge extraction, decision making, and
  plan generation
• Several types of decisional agents with only
  partial views of the phenomenon to be controlled
  and different viewpoints (eg physiological
  models)
• Presence of overlapping decisional models the
  input parameters as well as the output proposed
  decisions may intersect
• A negotiation mechanism to fuse the corresponding
  decisions 

117
Agent s role

• Knowledge extraction agent extract high-level
  information (parameter values) from low-level
  data received from sensors 
• Decisional agents generate a set of decisions
  in terms of desirable new states ( corrected 
  values for the parameters to be monitored)
• Actuator agents generate the sequence of
  actions to perform to reach the desired states
• The agents communication is mediated by two
  dedicated blackboards the parameter blackboard
  and the knowledge blackboard

118
System architecture
Decision Making
Knowledge Extraction
Plan Generation



119
Extractor agents

• An extractor agent
• is connected to sensors,
• from which it acquires signal information,
• that it filters and processes,
• to generate the values of a set of parameters
• The parameters values generated by all the
  extractor agents are placed in the parameters
  blackboard.

120
Implemented extractor agent

• The implemented extractor agent puts in the
  parameters blackboard a vector of parameter
  describing
• The current level of insulin
• The current level of glucose
• The current variation of the glucose
• The current level of the physical activity (as
  provided by piezoelectric crystal sensor for
  example)

121
Decisional agents

• The decisional agents
• Read the parameter information from the
  parameters blackboard 
• Computes a  decision  as a pair (desired target
  value for a parameter, weight)
• The computation of the desirable target value is
  based on the agent internal model, on the current
  values of parameters, and on the effects of past
  decisions
• The weight is a measure of how much the current
  parameter value is away from optimum and, thus,
  of how much the decisional agent  wants  to
  reach the proposed target value for that
  parameter
• Put the result (p,w) in the knowledge blackboard

122
Decisional agent s model

• Each decisional agent embeds the model of a
  particular physiological process aspect
• This model must provide a measure of how far the
  patient s state is from the optimum
• The model must also account for the
  interdependencies between the patient s
  physiological state, pathological state and
  activity
• Given a model, a set of desirable target states
  that minimize the distance to the optimum
  ( potential  values) is computed by means of an
  heuristic gradient descent technique

123
Decisional agent s model

• Samples of the evolution over time of the levels
  of insulin, glucose and glucose variations are
  collected, given a pathology level (in terms of
  the insulin basal secretion level and glucose
  variation sensibility), a food absorbtion curve
  and a physical activity level
• These curves are then sampled, thus providing the
  parameter values corresponding to different
  pathological states, in different life conditions
  these values are the indexes of the matrix
• The matrix values (  potential  values), are
  computed for each set of indexes as the distance
  between the corresponding pathological parameter
  and the one of a normal person smaller values
  correspond to more desirable states

124
(No Transcript)
125
Decisional agents

• Every parameter value of a proposed target point
  has an associated weight, which is the product of
  two factors
• the difference between the potential of the
  current value and the potential of the proposed
  value
• a weight measuring the  importance  of the
  physiological function controlled by the
  decisional agent
• for example by setting the importance weights it
  is possible to give higher priority to the
  control of vital functions than to the control of
  peripheral functions

126
The negotiation mechanism at a glance
Decision Agent 1
Physical cost
Social cost
Agent decision
Actuator Agent 1
Equalizer
Target state
Agent decision
Social cost
Physical cost
Decision Agent 2
127
The implemented decisional agents (1)

• The first decisional agent embeds a simplified
  control model of the glucose-insulin metabolism
  related to food adsorption
• Its role is to compute desired values and weights
  for the level of insulin, level of glucose and
  variation of the glucose, based on their current
  values
• This first decisional agent tries to reduce the
  glucose concentration during food adsorption

128
The implemented decisional agents (2)

• The second decisional agent embeds a simplified
  control model of the glucose-insulin metabolism
  related to physical activity
• Its role is to compute desired values and weights
  for the level of insulin, level of glucose and
  variation of the glucose based on their current
  values and the level of activity
• This second decisional agent tries to keep
  constant the glucose level by limiting the
  exogenous insulin introduction when the physical
  activity is intense.

129
Agent s importance

• The value of the importance is variable according
  to the current state of the system
• The importance of the first decisional agent,
  which is related to the food absorbtion, is
  higher than that of the second decisional agent,
  which is related to physical activity
• This reflects the fact that the main problem of a
  diabetic patient is the insulin response when the
  patient eats

130
The negotiation mechanism

• Different decisional agents can propose different
  variations for the same parameters, generating
  conflicts
• For example, the first decisional agent may
  propose to decrease the glucose level after a
  meal whilst the second may propose to keep
  constant the glucose level because the patient is
  undergoing an intense physical activity
• In the proposed approach, the relations between
  agent s models are not explicitely considered
  and the decisional agents are not aware of the
  presence of the others
• Necessity of an external negotiation mechanism
  (the equalizer, situated in the knowledge
  blackboard)

131
The negotiation mechanism

• To make a final decision, the decisional agent
  has to select a single target state
• For this purpose, it calculates the cost of the
  variation of each parameter from the current
  state to a target state, as the weighted sum of
  the measure variations
• The weights are computed as the sum of two
  elementary costs 
• The actuation cost ( physical  cost)
• The negotiation cost ( social  cost)

132
The negotiation mechanism

• The unitary cost of the parameter is the sum of
  two elements
• The actuation cost is determined by the actuator
  agent that acts on the parameter it measures
  the cost of the physical variation of the
  parameter
• The negotiation cost is determined by the
  equalizer component during the negotiation
  process it measures the difficul