Multi-Agent Systems Lecture 6 University

1
Multi-Agent SystemsLecture 6 University
Politehnica of Bucarest2005 - 2006Adina
Magda Floreaadina_at_cs.pub.rohttp//turing.cs.pub
.ro/blia_06
2
Working togetherLecture outline

• 1 Coordination strategies
• 1 Distributed problem solving
• 2 Distributed planning
• 3 An example Partial global planning

3
1 Coordination strategies

• Coordination the process by which an agent
  reasons about its local actions and the
  (anticipated) actions of others to try to ensure
  the community acts in a coherent manner

Coordination
Self-interested agents own goals
Collectively motivated agents common goals
Coordination for coherent behavior
Cooperation to achieve common goal
Competitive conflicting goals
Neutral to one another disjunctive goals
3
4

• Model
• Protocol
• Communication

• Perfect coordination ???
• Centralized coordination
• Distributed coordination

• Tightly coupled interactions - distributed search
• Cognitive agents DPS (distributed planning,
  task sharing, resource sharing)
• Heterogeneous agents - interaction protocols
  Contract Net, KQML conversations, FIPA protocols
• Dynamic interactions Shared mental states,
  commitments and conventions
• Complex interactions - organizational structure
  to reduce complexity
• Unpredictable interactions - social laws
• Conflict of interests - interaction protocols
  voting, auctions, bargaining, market mechanisms,
  extended Contract Net, coalition formation

Cooperative
Neutral or competitive
4
5
2. Distributed problem solving

• Distributed planning - the problem to be solved
  is to design and execute a plan in a distributed
  manner, by many agents discussed here
• Task and result sharing - an agent has many tasks
  to do and asks other agents to do some of its
  tasks then it should integrate the results
  discussed at negotiation

5
6
3 Distributed planning

• What can be distributed
• The process of coming out with a plan is
  distributed among agents
• Execution is distributed among agents
• Planning
• State representation and plan representation
• Search vs planning
• representation of changes to the world state
• representation of and reasoning about the plan
  (steps/actions)
• Linear planning
• Partial order planning
• Hierarchical planning
• Conditional planning

Planning ? Search
6
7
3.1 Planificare monoagent

• Operatori de plan planificare liniara
• - Actiune.
• - Lista Preconditiilor
• - Lista Adaugarilor
• - Lista Eliminarilor
• Operatori de plan planificare neliniara
• - Actiune.
• - Lista Preconditiilor
• - Lista Postconditiilor

7
8
STRIPS planificare liniara

• Operatori de plan
• STACK(x,y), UNSTACK(x,y), PICKUP(x), PUTDOWN(x)
• Predicate
• ON(x,y), ONTABLE(x), CLEAR(x), HOLD(x), ARMEMTY
• Axiome

8
9
Reprezentarea STRIPS

• Operatori de plan

LP LE CLEAR(y)HOLD(x) LA ON(x,y
)ARMEMPTY LP LE LA PICKU
P(x) LP LE LA PUTDOWN
(x) LP LE LA
9
10
TWEAK planificare neliniaraReprezentare
Actiune Preconditii Postconditii Actiune
PICKUP(x) Preconditii Postconditii
10
11
TWEAK planificare neliniara
Operatii de modificare a planului (1) adaugarea
de pasi este operatia prin care se creaza noi
pasi care se adauga la plan (2) promovarea este
operatia de stabilire a unei ordonari (temporale)
intre doi pasi de plan (3) legarea simpla este
operatia de atribuire de valori variabilelor
pentru a valida preconditiile unui pas de
plan (4) separarea este operatia de impiedicare
a atribuirii anumitor valori unei
variabile (5) eliminarea destructivitatii este
operatia de introducere a unui pas S3 (un pas
deja existent in plan sau un pas nou) intre pasii
S1 si S2, in scopul de a adauga un fapt invalidat
de pasul S1 si necesar in pasul S2.
11
12

• 3.1 Centralized planning for distributed plans
• Operators
• move(b,x,y) ? movetotable(b,x)
• Precond on(b,x) ? clear(b) ? clear(y)
  Precond on(b,x) ? clear(b)
• Postcond on(b,y) ? clear(x) ? Postcond
  on(b,T) ? clear(x) ? ?on(b,x)
• ?on(b,x) ? ?clear(y)

I'm Bill Agent1
I'm Tom Agent2
on(A,B) on(C,D) on(E,F) on(B,T) on(D,T)
on(F,T)
on(B,A) on(F,D) on(A,E) on(D,C) on(E,T)
on(C,T)
1. Given a goal description, a set of
operators, and an initial state
description generate a partial order plan
12
13

• S1 move(B,T,A) To satisfy the preconditions,
  we have
• S2 move(A,B,E) S2 lt S1, S3 lt S4
• S3movetotable(E,F) S6 lt S4, S6 lt S5
• S4 move(F,T,D) Also
• S5 move(D,T,C) S2 threat to S3 ? S3 lt S2
• S6 movetotable(C,D) S4 threat to S5 ? S5 lt S4
• Then the partial ordering is S3 lt S2
  lt S1
• S6 lt S5 lt S4
• S3 lt S4
• S3 movetotable(E,F) S2 move(A,B,E) S1
  move(B,T,A)
• S6 movetotable(C,D) S5 move(D,T,C) S4
  move(F,T,D)
• Any total ordering that satisfies this partial
  ordering is a good plan for Agent1
• What if we have 2 agents?
• DECOMP1
• Subplan1 S3 lt S2 lt S1
• Subplan2 S6 lt S5 lt S4
• and S3 lt S4
• Agent1 S3 lt send(clear(F)) lt S2 lt S1
• Agent2 S6 lt S5 lt wait(clear(F)) lt S4

lt
lt
2. Decompose the plan into subproblems so as to
minimize order relations across plans 3. Insert
synchronization 4. Allocate subplans to agents
13
14

• S3 movetotable(E,F) S2 move(A,B,E) S1
  move(B,T,A)
• S6 movetotable(C,D) S5 move(D,T,C) S4
  move(F,T,D)
• DECOMP2
• Subplan1 S3 lt S5 lt S4
• Subplan2 S6 lt S2 lt S1
• and S3 lt S2 and S6 lt S5
• Agent1 S3 lt send(on(E,T)) , wait(clear(D)) lt S5 lt
  S4
• Agent2 S6 lt send(clear(D)), wait(on(E,T)) lt S2 lt
  S1
• Obviously, DECOMP2 has more order relations among
  subplans than DECOMP1
• Therefore, we choose DECOMP1
• S3 lt send(clear(F)) lt S2 lt S1
• S6 lt S5 lt wait(clear(F)) lt S4
• But
• then back to DECOMP2

lt
lt
4. If failure to allocate subplans then redo
decomposition (2) If failure to allocate
subplans with any decomposition then redo
generate plan (1) 5. Execute and monitor subplans
I know how to move only D, E, F
I know how to move only A, B, C
14
15

• 2.2 Distributed planning for centralized plans
• Each of the planning agents generate a partial
  plan in parallel then merge these plans into a
  global plan
• parallel to result sharing
• may involve negotiation
• Agent 1 - is specialized in doing
  movetotable(b,x)
• Agent 2 - is specialized in doing move(b,x,y)
• Agent 1 - based on Sf it comes out with the
  partial plan
• PAgent1 S3 movetotable(E,F) satisfies
  on(E,T)
• S6 movetotable(C,D) satisfies on(C,T)
• no ordering
• Agent 2 - based on Sf it comes out with the
  partial plan
• PAgent 2 S1 move(B,T,A), S2
  move(A,B,E) satisfies on(B,A) ? on(A,E)
• S4 move(F,T,D), S5 move(D,T,C) satisfies
  on(F,D) ? on(D,C)
• ordering S2 lt S1 and S5 lt S4
• Merge PAgent1 with PAgent2 by checking
  preconditons and threats
• Establish thus order S3 lt S2, S6 lt S5, S3 lt S4
  order of PAgent2
• Then give any instance of this partial plan to an
  execution agent to carry it out

15
16

• The problem is decomposed and distributed among
  various planning specialists, each of which
  proceeds then to generate its portion of the plan
• similar to task sharing
• may involve backtracking
• Agent 1 - knows only how to deal with 2-block
  stacks
• Agent 2 - knows only how to deal with 3-block
  stacks

16
17

• 2.3 Distributed planning for distributed plans
• a) Plan merging
• Agents formulate local plans to satisfy their
  goals
• Local plans are exchanged
• Local plans are combined analyzing for positive
  and negative interaction
• Add messages and/or timing commitments to resolve
  negative plan interactions and to exploit
  positive plan interactions
• Interacting situations
• Positive interactions between plans
• redundant actions
• static detection sequencing
• favour actions
• dynamic detection incorporation
• Negative interactions between plans
• harmful actions
• exclusive actions
• incompatible actions

17
18

• movehigh(b,x,y)
• Precond have_lifter ? clear(b) ? clear(y) ?
  on(y,z) ? z? T
• Postcond on(b,y) ? clear(x) ? ?on(b,x) ? ?
  clear(y) ? free_lifter
• pick_lifter
• Precond free_lifter
• Postcond have_lifter ? ?free_lifter
• Agent1 S1move(B,T,A) lt S2 pick_lifter lt S3
  movehigh(E,T,B)
• Agent2 R1move(C,T,D) lt R2 pick_lifter lt R3
  movehigh(F,T,C)

Negative interactions what type?
R1
S1
need_l
S2
S3
Sf1
free_l
R2
R3
18
19

• b) Iterative plan formation
• build all feasible plans
• build partial order plans to facilitate plan
  merging
• build abstract plans to be iteratively refined
• - see next section and PGP section

19
20

• c) Hierarchical distributed planning
• Design plans on several levels of abstraction
• Use abstract plans
• Abstract operator - a kind of macro-operator
  sequence of applicable operators

Write paper
Edit content
Read references
Organize ideas
Edit text
..
Find editor
Check for errors
Edit figures
20
21
Hierarchical behavior-space search
algorithm 1. Level ? 0, Agent_List Agent1, ,
AgentN 2. for every Agenti in Agent_List
do 2.1 Agenti sends description of Gi and Pi
to every Agentj, j1,N, j?i 2.2 Agenti gets
Gj, Pj from Agentj, j1,N, j?i 2.3 if Pi is
compatible with Pj, j1,N, j?i then
Agenti removes itself from Agent_List 3. if
Agent_list then exit 4. Be N the new number
of agents in Agent_List 5. Sort agents in
Agent_List 6. for i1,N-1, cf. ordering
do 6.1 make Agenti the current superior 6.2
Agenti determines conflicts between Pi 6.3
if conflicts to be resolved at a lower level
then (a) Level ? Level 1 (b)
Agent_List Agenti1, , AgentN (c) go
to step 2 6.4 send Pi to each Agentj, ji1,
N 6.5 for ji1, N do - Agentj checks
compatibility of Pj with Pi and replan, if nec.

• A kind of CSP
• Ordering
• - what heuristic?

Add exit condition for no solution
21
22

• 2.4 Distributed planning and execution
• Real world incomplete and incorrect information
• a) Contingency planning
• Conditional planning - deals with incomplete
  information by constructing a conditional plan
  that accounts for each possible situation or
  contingency that could arrive
• sensing actions
• a context of a plan step, i.e., a union of
  conditions on the environment that must hold in
  order for a step to be executed ? introduces
  disjunctive steps conditional links among plan
  steps

Start
on(A,B)?clear(C)?clear(A)
Checkarm(Ag1)
Ask Ag2 to move(A,B,C)
?armbroken(Ag1)
armbroken(Ag1)
move(A,B,C)
Context ?armbroken(Ag1)
Negotiate with Ag2 for it to achieve move
Plan to achieve on(B,A)
Finish
on(B,A)?on(A,C)
22
23

• b) Execution monitoring
• The agent does not execute the plan with "its
  eyes closed" - It monitors what is happening
  while it executes the plan and it can do
  replanning to achieve a goal in a new situation
• Conditional planning thinks before to several
  alternatives
• Monitoring and replanning defers the job I
  shall see what to do if new conditions occur
• c) Social laws
• What actions are legal to be executed in a
  certain context
• Find conflicting situations, analyze what
  concurrent actions lead to these situations and
  prohibit such concurrent actions by social laws
• It is fit, in general, for loosely coupled
  subproblems / subplans

23
24
3 Partial Global Planning

• Initially applied in the Distributed Monitoring
  Vehicle (DVM) Testbed
• Extended to be domain independent
• Integrates planning and execution
• Coordination by means of partial plans exchange
• Partial plans abstract plans partial ordering
  ? plan merging
• The domain - unpredictable, unreliable
  information
• The tasks are inherently distributed each agent
  performs its own task
• The agents are not aware of the global state of
  the system however there is a common goal
  converge on a consistent map of vehicle movements
  by integrating the partial tracks formed by
  different agents into a single complete map or
  into a consistent set of local maps distributed
  among agents
• Cooperative agents (collectively motivated)

24
25

• 3.1 Aircraft monitoring scenario
• each type of aircraft produces a characteristic
  spectrum of acoustic frequencies
• signals may be improperly sensed, there is
  ghosting and environmental noise
• there are two agents A and B whose regions of
  interest overlap each agent receives data only
  about its own region, from its acoustic sensor
• the goal is to identify any aircraft that is
  moving through the region of interest, determine
  their types and track them through regions
• converge on a consistent map of vehicle movements
  by integrating the partial tracks formed by
  different agents into a single complete map or
  into a consistent set of local maps distributed
  among agents

Final solution
Data input
25
26

• 3.2 Agent functioning
• 1. Represent its own expected activity by a set
  of local (tentative) plans, at two levels higher
  level (abstract plans) and detailed level local
  plans may involve alternative actions depending
  on the result of previous actions and changes in
  the environment
• ? conditional plans hierarchical plans
• 2. Communicate abstract local plans to the other
  agents and get from them such plans ? another
  form of communication
• 3. Model collective activity of the agents by
  forming Partial Global Plans and finding out how
  they can be improved for better coordination
• identify when the goals of one or more agents can
  be considered subgoals of a single global goal ?
  partial global goal
• construct a PGP and identify opportunities for
  improved coordination
• search for an improved PGP
• 4. Based on 3, propose changes to one or more
  agents' plans
• ? negotiation
• 5. Modify its local plan according to the
  proposal and plan what and when results will be
  communicated to the other agents

26
27

• 2 types of problem-solving activities
• task-level activities - build a map of vehicle
  movements
• meta-level activities - decide how and with whom
  to coordinate
• Result sharing - agents exchange appropriate
  results at the right time
• Task sharing - allow agents to propose potential
  plans that involve the transfer of tasks among
  them

A Process 1/3 data
B Process 1/3 data
Who? Process 1/3 data
27
28

• 3.3 Plan representation
• A plan represents future activity at two levels
  of detail
• at the higher level it outlines the major steps
  it expects to take to achieve its goal - abstract
  plan
• at a detailed level it specifies primitive
  actions to achieve the next step in the abstract
  plan as the plan is executed, new details are
  added incrementally
• action
• Prec preconditions for the action
• Post results of the action
• D - the set of data to be processed by the
  action
• P - the set of procedures to be applied to the
  data
• Tstart - the estimated start time of the action
• Tend - the estimated end time of the action
• abres - an estimate of the characteristics of
  and confidence in the abstract partial result
  that will be developed as conclusion of action

28
29

• 3.4 PGP formation and coordination
• (1) Task decomposition
• (2) Local plan formation
• (3) Local plan abstraction
• (4) Communication about local abstract plans
• Meta-Level Organization specifies roles and
  controls communication
• For each agent, the MLO specifies
• - the agents it has authority over
• - the agents that have authority over it
• - the agents that have equal authority
• (5) Partial global goal identification
• Set of operators that generate global goals based
  on local goals

29
30

• (6) Partial global plan construction and
  modification
• partial global goal
• plan-activity-map plan actions to be executed
  concurrently by itself and the other agents,
  including costs and expected results of actions
  PGP
• PGP hill-climbing algorithm
• Criteria for rating the actions (eval function)
• the action extends a partial result (vehicle
  tracking hypothesis)
• the action produces a partial result that might
  help some other agents in forming partial results
• how long the action is expected to take

30
31

• Algorithm for PGP plan step reordering
• For the current ordering, rate individual actions
  and sum the rating
• For each action, examine the later actions for
  the same agent and find the most highly rated
  one. If it is higher rated, then swap the actions
• If the new ordering is more highly rated than the
  current one, then replace the current ordering
  with the new one and Go to step 2
• Return the current ordering

31
32

• (7) Communication planning
• From the modified plan-activity-map, the agent
  builds a solution-construction-graph how the
  agents should interact, including specifications
  about what partial results to exchange and when
  to exchange them
• (8) Translate to local level the activities in
  the revised plan
• (9) If authority, send PGP to the other agents

32
33

• References
• E.H. Durfee. Distributed problem solving and
  planning. In Multiagent Systems - A Modern
  Approach to Distributed Artficial Intelligence,
  G. Weiss (Ed.), The MIT Press, 2001, p.121-164.
• V.R. Lesser. A retrospective view of FA/C
  distributed problem solving. IEEE Trans. On
  Systems, Man, and Cybernetics, 21(6), Nov/Dec
  1991, p.1347-1362.
• E.D. Durfee, V.R. Lesser Partial global planning
  A coordination framework for distributed
  hypothesis formation. IEEE Trans. On Systems,
  Man, and Cybernetics, 21(5), Sept. 1991,
  p.1167-1183.
• K.S. Decker, V.R. Lesser. Generalizing the
  partial global planning algorithm. International
  Journal of Intelligent Cooperative Information
  Systems, 1(2), 1992, p. 319-346.
• S. Russell, P. Norvig. Artificial Intelligence A
  Modern Approach. Prentice hall, 1995, Ch. 11, 12,
  13.

33