Lecture 7: Reaching Agreements

1
Reaching Agreements Voting
2
Voting

• Truthful voters vote for the candidate they think
  is best.
• Why would you vote for something you didnt want?
  (run off election want to pick competition)
  (more than two canddiates, figure your candidate
  doesnt have a chance)
• We vote in awarding scholarships, teacher of the
  year, person to hire.
• Rank feasible social outcomes based on agents'
  individual ranking of those outcomes
• A - set of n agents
• O - set of m feasible outcomes
• Each agent i has a preference relation gti O x
  O, asymmetric and transitive

2
3

• Social choice rule (good for society)
• Input the agent preference relations (gt1, , gtn)
• Output elements of O sorted according the input
  - gives the social preference relation lt of the
  agent group
• In other words creates ordering for the group

3
4

• Desirable properties of the social choice rule
• A social preference ordering gt should exist for
  all possible inputs (Note, I am using gt to mean
  is preferred to.)
• gt should be defined for every pair (o, o')?O
• gt should be asymmetric and transitive over O
• The outcomes should be Pareto efficient
• if ??i ?A, o gti o' then o gt o (not misorder
  if all agree)
• The scheme should be independent of irrelevant
  alternatives (if all agree on relative ranking of
  two, should retain ranking in social choice)
• No agent should be a dictator in the sense that
• o gti o' implies o gt o' for all preferences of
  the other agents

4
5

• Arrow's impossibility theorem
• No social choice rule satisfies all of the six
  conditions
• Must relax desired attributes
• May not require gt to always be defined
• We may not require that gt is asymmetic and
  transitive
• Use plurality protocol all votes are cast
  simultaneously and highest vote count wins.
• Introducing an irrelevant alternative may split
  the majority causing the old majority and the new
  irrelevant to drop out of favor (The Ross Perot
  effect).
• A binary protocol involves voting pairwise
  single elimination
• The order of the pairing can totally change the
  results

6
One voter ranks c gt d gt b gt a One voter ranks a gt
c gt d gt b One voter ranks b gt a gt c gt d Notice,
just rotates preferences.
winner (c, (winner (a, winner(b,d)))a
winner (d, (winner (b, winner(c,a)))d
winner (d, (winner (c, winner(a,b)))c
winner (b, (winner (d, winner(c,a)))b
surprisingly, order of pairing yields different
winner!
7

• Borda protocol (used if binary protocol is too
  slow) assigns an alternative O points for the
  highest preference, O-1 points for the second,
  and so on
• The counts are summed across the voters and the
  alternative with the highest count becomes the
  social choice
• Winner turns loser and loser turns winner if the
  lowest ranked alternative is removed (does this
  surprise you?)

7
8
Borda Paradox remove loser, winner
changes(notice, c is always ahead of removed
item)

• a gt b gt c
• b gt c gta
• c gt a gt b
• a gt b gt c
• b gt c gt a
• c gt a gtb
• a ltb ltc
• a15,b14, c13

• a gt b gt c gtd
• b gt c gt d gta
• c gt d gt a gt b
• a gt b gt c gt d
• b gt c gt dgt a
• c gtd gt a gtb
• a ltb ltc lt d
• a18, b19, c20, d13

When loser is removed, next loser becomes winner!

9
Strategic (insincere) voters

• Suppose your choice will likely come in second
  place. If you rank the first choice of rest of
  group very low, you may lower that choice enough
  so yours is first.
• True story. Deans selection. Each committee
  member told they had 5 points to award and could
  spread out any way among the candidates. The
  recipient of the most points wins. I put all my
  points on one candidate. Most split their points.
  I swung the vote! What was my gamble?
• Want to get the results as if truthful voting
  were done.

10
Typical Competition Mechanisms

• Auction allocate goods or tasks to agents
  through market. Need a richer technique for
  reaching agreements
• Negotiation reach agreements through
  interaction.
• Argumentation resolve confliction through
  debates.

11
Reaching Agreements Voting
12
Negotiation

• May involve
• Exchange of information
• Relaxation of initial goals
• Mutual concession

13
Mechanisms, Protocols, Strategies

• Negotiation is governed by a mechanism or a
  protocol
• defines the rules of encounter between the
  agents
• the public rules by which the agents will come to
  agreements.
• The deals that can be made
• The sequence of offers and counter-offers that
  can be made
• Given a particular protocol, how can a particular
  strategy be designed that individual agents can
  use?

14
Negotiation Mechanism
Negotiation is the process of reaching agreements
on matters of common interest. It usually
proceeds in a series of rounds, with every agent
making a proposal at every round.

• Issues in negotiation process
• Negotiation Space All possible deals that
  agents can make, i.e., the set of candidate
  deals.
• Negotiation Protocol A rule that determines
  the process of a negotiation how and when a
  proposal can be made, when a deal has been
  struck, when the negotiation should be
  terminated, and so.
• Negotiation Strategy When and what proposals
  should be made.

15
Protocol

• Means kinds of deals that can be made
• Means sequence of offers and counter-offers
• Protocol is like rules of chess game, whereas
  strategy is way in which player decides which
  move to make
• Do we even understand what is up for grabs? We
  may ask for a raise without considering bigger
  office, different appointment, 4-day work week,
  etc.

16
Negotiation Protocol

• Who begins
• Take turns
• Build off previous offers
• Give feed back (or not).
• Tell what utility is (or not)
• Obligations requirements for later
• Privacy
• Allowed proposals you can make as a result of
  negotiation history

17
Negotiation Process 1

• Negotiation usually proceeds in a series of
  rounds, with every agent making a proposal at
  every round.
• Communication during negotiation

18
Negotiation Process 2

• Another way of looking at the negotiation process
  is (can talk about 50/50 or 90/10 depending on
  who moves the farthest)

19
Jointly Improving Direction method

• Iterate over
• Mediator helps players criticize a tentative
  agreement (could be status quo)
• Generates a compromise direction (where each of
  the k issues is a direction in k-space)
• Mediator helps players to find a jointly
  preferred outcome along the compromise direction,
  and then proposes a new tentative agreement.

20
Example list of things to be done. Assigned to
individuals already

• Who does what
• What is order to do
• What is paid for tasks

21
Goals of Negotiation (in many cases)

• Efficiency not waste utility. Pareto Opt
• Stability no agent have incentive to deviate
  from agreed-upon strategy (as in one-shot
  negotiation).
• Simplicity low computational demands on agents
• Distribution interaction rules not require a
  central decision maker
• Symmetry (in some cases) may not want agents to
  play different roles.

22
Example

• Planes need to be assigned landing time.
• Rule could be that airplanes with less fuel land
  first. Any disadvantage?

23
Slotted Blocks world

• Like blocks world, only a fixed number of slots
  on table.
• Forces need to coordinate
• Ex Need to share car. Has side effects
• Ex Schedule classes/professors no side effect.

24
Various Domains
Worth Oriented Domain
State Oriented Domain
Task Oriented Domain
25
Typical Negotiation Problems

• Task-Oriented Domains(TOD) an agent's activity
  can be defined in terms of a set of tasks that it
  has to achieve. The target of a negotiation is to
  minimize the cost of completing the tasks.
• State Oriented Domains(SOD) each agent is
  concerned with moving the world from an initial
  state into one of a set of goal states. The
  target of a negotiation is to achieve a common
  goal. Main attribute actions have side effects
  (positive/negative). TOD is a subset of SOD.
  Most classical AI domains are instances of SOD.
• Main attribute of SOD actions have side
  effects. Agents can unintentionally achieve one
  anothers goals. Negative interactions can also
  occur.
• Worth Oriented Domains(WOD) agents assign a
  worth to each potential state, which captures its
  desirability for the agent. The target of a
  negotiation is to maximize mutual worth (rather
  than worth to individual). Superset of SOD.
• Rates the acceptability of final states.
  Allows agents to compromise on their goals.

26
The simplest plan to achieve On(White,Gray) has
the side effect of achieving Clear(black)
27
Single issue negotiation

• Like money
• Symmetric (If roles were reversed, I would
  benefit the same way you would)
• If one task requires less travel, both would
  benefit equally by having less travel
• utility for a task is experienced the same way by
  whomever is assigned to that task.
• Non-symmetric we would benefit differently if
  roles were reversed
• if you delivered the picnic table, you could just
  throw it in the back of your van. If I delivered
  it, I would have to rent a U-haul to transport it
  (as my car is small).

28
Multiple Issue negotiation

• Could be hundreds of issues (cost, delivery date,
  size, quality)
• Some may be inter-related (as size goes down,
  cost goes down, quality goes up?)
• Not clear what a true concession is (larger may
  be cheaper, but harder to store or spoils before
  can be used)
• May not even be clear what is up for negotiation
  (I didnt realize not having any test was an
  option) (on the jobAsk for stock options,
  bigger office, work from home.)

29
How many agents are involved?

• One to one
• One to many (auction is an example of one seller
  and many buyers)
• Many to many (could be divided into buyers and
  sellers, or all could be identical in role)
• n(n-1)/2 number of pairs

30
Negotiation DomainsTask-oriented

• Domains in which an agents activity can be
  defined in terms of a set of tasks that it has to
  achieve, (Rosenschein Zlotkin, 1994)
• An agent can carry out the tasks without
  interference (or help) from other agents such
  as who will deliver the mail
• All resources are available to the agent
• Tasks redistributed for the benefit of all agents

31
Task-oriented Domain Definition

• How can an agent evaluate the utility of a
  specific deal?
• Utility represents how much an agent has to gain
  from the deal. (it is always based on change from
  original allocation)
• Since an agent can achieve the goal on its own,
  it can compare the cost of achieving the goal on
  its own to the cost of its part of the deal.
• If utilitylt0, it is worse off than performing
  tasks on its own.
• Conflict deal (stay with status quo) if agents
  fail to reach an agreement
• where no agent agrees to execute tasks other than
  its own.
• utlity 0

32
Formalization of TOD

• A Task Oriented Domain(TOD) is a triple ltT, Ag,
  cgt
• where
• T is a finite set of all possible tasks
• AgA1, A2,, An is a list of participant
  agents
• c?(T)?R defines cost of executing each subset
  of tasks.

• Assumptions on cost function
• c(?) 0.
• The cost of a subset of tasks does not depend on
  who carries out them. (Idealized situation)
• Cost function is monotonic, which means that
  more tasks, more cost. (It cant cost less to
  take on more tasks.)
• T1 ? T2 implies c(T1) ? c(T2)

33
Redistribution of Tasks

• Given a TOD ltT, A1,A2, cgt, T is original
  assignment, output is D assignment after the
  deal
• An encounter (instance) within the TOD is an
  ordered list (T1, T2) such that for all k, Tk ?
  T. This is an original allocation of tasks that
  they might want to reallocate.
• A pure deal on an encounter is the redistribution
  of tasks among agents (D1, D2), such that all
  tasks are reassigned
• D1? D2 T1? T2
• Specifically, (D1, D2)(T1, T2) is called
  the conflict deal.
• For each deal ?(D1, D2), the cost of such a deal
  to agent k is Costk(?)c(Dk) (i.e, cost to
  k of deal ? is cost of Dk, ks part of deal)

34
Examples of TOD

• Parcel Delivery
• Several couriers have to deliver sets of parcels
  to different cities. The target of negotiation is
  to reallocate deliveries so that the cost of
  travel to each courier is minimal.
• Database Queries
• Several agents have access to a common database,
  and each has to carry out a set of queries. The
  target of negotiation is to arrange queries so as
  to maximize efficiency of database operations
  (Join, Projection, Union, Intersection, ) . You
  are doing a join as part of another operation, so
  please save the results for me.

35
Possible Deals

• Consider an encounter from the Parcel Delivery
  Domain. Suppose we have two agents. Both agents
  have parcels to deliver to city a and only agent
  2 has parcels to deliver to city b. There are
  nine distinct pure deals in this encounter
• (a, b)
• (b, a)
• (a,b, ?)
• (?, a,b)
• (a, a,b)

1. (b, a,b)
2. (a,b, a)
3. (a,b, b)
4. (a,b, a,b)

the conflict deal
36
Figure deals knowing union must be ab

• Choices for first agent a b ab
• Second agent must pick up the slack
• a for agent 1 ?bab (for agent 2)
• b for agent 1?aab
• ab for agent 1 ?aabb
• for agent 1 ?ab

37
Utility Function for Agents

• Given an encounter (T1, T2), the utility function
  for each agent is just the difference of costs
  and is defined as follow
• Utilityk(?)c(Tk)-Costk(?) c(Tk)- c(Dk)
• where
• ?(D1, D2) is a deal
• c(Tk) is the stand-alone cost to agent k (the
  cost of achieving its original goal with no help)
• Costk(?) is the cost of its part of the deal.
• Note that the utility of the conflict deal is
  always 0.

38
Parcel Delivery Domain (assuming do not have to
return home like Uhaul)
Distribution Point
Cost function c(?)0 c(a)1 c(b)1 c(a,b)3
1
1
city a
city b
2

• Utility for agent 1 (org a)
• Utility1(a, b) 0
• Utility1(b, a) 0
• Utility1(a, b, ?) -2
• Utility1(?, a, b) 1

• Utility for agent 2 (org ab)
• Utility2(a, b) 2
• Utility2(b, a) 2
• Utility2(a, b, ?) 3
• Utility2(?, a, b) 0

39
Dominant Deals

• Deal ? dominates deal ?' if ? is better for at
  least one agent and not worse for the other,
  i.e.,
• ? is at least as good for every agent as ?'
• ?k?1,2, Utilityk(?)? Utilityk(?')
• ? is better for some agent than ?'
• ?k?1,2, Utilityk(?)gt Utilityk(?')
• Deal ? weakly dominates deal ?' if at least the
  first condition holds (deal isnt worse for
  anyone).
• Any reasonable agent would prefer (or go along
  with) ? over ?' if ? dominates or weakly
  dominates ?'.

40
Negotiation Set Space of Negotiation

• A deal ? is called individual rational if ?
  weakly dominates the conflict deal. (no worse
  than what you have already)
• A deal ? is called Pareto optimal if there does
  not exist another deal ?' that dominates ?.
  (best deal for x without disadvantaging y)
• The set of all deals that are individual rational
  and Pareto optimal is called the negotiation set
  (NS).

41
Utility Function for Agents (example from
previous slide)

1. Utility1(a, b) 0
2. Utility1(b, a)0
3. Utility1(a,b, ?)-2
4. Utility1(?, a,b)1
5. Utility1(a, a,b)0
6. Utility1(b, a,b)0
7. Utility1(a,b, a)-2
8. Utility1(a,b, b)-2
9. Utility1(a,b, a,b)-2

1. Utility2(a, b) 2
2. Utility2 (b, a)2
3. Utility2 (a,b, ?)3
4. Utility2 (?, a,b)0
5. Utility2 (a, a,b)0
6. Utility2 (b, a,b)0
7. Utility2 (a,b, a)2
8. Utility2 (a,b, b)2
9. Utility2 (a,b, a,b)0

42
Individual Rational for Both(eliminate any
choices that are negative for either)

• (a, b)
• (b, a)
• (a,b, ?)
• (?, a,b)
• (a, a,b)
• (b, a,b)
• (a,b, a)
• (a,b, b)
• (a,b, a,b)

(a, b) (b, a) (?, a,b) (a,
a,b) (b, a,b)
individual rational
43
Pareto Optimal Deals

• (a, b)
• (b, a)
• (a,b, ?)
• (?, a,b)
• (a, a,b)
• (b, a,b)
• (a,b, a)
• (a,b, b)
• (a,b, a,b)

is (-2,3), but nothing beats 3 for agent 2
(a, b) (b, a) (a,b, ?) (?, a,b)
Pareto Optimal
Beaten by (ab) deal
44
Negotiation Set
Individual Rational Deals (a, b) (b,
a) (?, a,b) (a, a,b) (b, a,b)

• Pareto Optimal Deals
• (a, b)
• (b, a)
• (a,b, ?)
• (?, a,b)

Negotiation Set (a, b) (b, a)
(?, a,b)
45
Negotiation Set illustrated

• Create a scatter plot of the utility for i over
  the utility for j
• Only those where both is positive are
  individually rational (for both) (origin is
  conflict deal)
• Which are pareto optimal?

Utility for i
Utility for j
46
Negotiation Set in Task-oriented Domains
Utility for agent i
Negotiation set (pareto optimal Individual
rational)
B
A
C
Utility of conflict Deal for agent i
The circle delimits the space of all possible
deals
E
D
Conflict deal
Utility for agent j
Utility of conflict Deal for agent j
47
Negotiation Protocol

• P(d) Product of the two agent utilities from d
• product maximizing negotiation protocol One step
  protocol
• Concession protocol
• At time t gt 0, A offers d(A,t) and B offers
  d(B,t), such that
• Both deals are from the negotiation set
• "i e A,B and "t gt0, Utilityi(d(i,t)) lt
  Utilityi(d(i,t-1))
• I propose something less desirable for me
• Negotiation ending
• Conflict - Utilityi(d(i,t)) Utilityi(d(i,t-1))
• Agreement, j !i e A,B, Utilityj(d(i,t)) gt
  Utilityj(d(j,t))
• Only A gt agree d(B,t) either agrees with
  proposal of other
• Only B gt agree d(A,t) either agrees with
  proposal of other
• Both A,B gt agree d(k,t) such that
  P(d(k))maxP(d(A)),P(d(B))
• Both A,B and P(d(A))P(d(B)) gt flip a coin
  (product is the same, but may not be the same for
  each agent flip coin to decide which deal to
  use)

Pure deals
Mixed deal
48
The Monotonic Concession Protocol One
direction, move towards middle

• Rules of this protocol are as follows. . .
• Negotiation proceeds in rounds.
• On round 1, agents simultaneously propose a deal
  from the negotiation set individually rational,
  pareto optimal). Can re-propose same deal.
• Agreement is reached if one agent finds that the
  deal proposed by the other is at least as good or
  better than its proposal.
• If no agreement is reached, then negotiation
  proceeds to another round of simultaneous
  proposals.
• An agent is not allowed to offer the other agent
  less (in term of utility ) than it did in the
  previous round. It can either stand still or make
  a concession. Assumes we know what the other
  agent values.
• If neither agent makes a concession in some
  round, then negotiation terminates, with the
  conflict deal.
• Meta data may be present explanation or critique
  of deal.

49
Condition to Consent an Agreement
If both of the agents finds that the deal
proposed by the other is at least as good or
better than the proposal it made. Utility1(?2)?
Utility1(?1) and Utility2(?1)? Utility2(?2)
50
The Monotonic Concession Protocol

• Advantages
• Symmetrically distributed (no agent plays a
  special role)
• Ensures convergence
• It will not go on indefinitely
• Disadvantages
• Agents can run into conflicts
• Inefficient no quarantee that an agreement will
  be reached quickly

51
Negotiation Strategy

• Given the negotiation space and the Monotonic
  Concession Protocol, a strategy of negotiation is
  an answer to the following questions
• What should an agents first proposal be?
• On any given round, who should concede?
• If an agent concedes, then how much should it
  concede?

52
The Zeuthen Strategy a refinement of monotonic
protocol

• Q What should my first proposal be?
• A the best deal for you among all possible deals
  in the negotiation set. (Is a way of telling
  others what you value.)

agent 2's best deal
Agent 1's best deal
53
Example of Zeuthan

• In interviewing for Womens center director, the
  candidate we were most interested in was
  approached.
• She started by asking for
• 10K more money
• Job for husband
• Tenured full professor in academic department
• Gold parking pass for terrace

54
What was her strategy?

• Clearly Zeuthan
• Advantages she had something to concede and we
  knew what she valued
• Disadvantage could be thought of as too much so
  that the committee removes her from the pool.
• Have had students make initial request that
  backfired as seemed totally off-base.
• If you realize someone is using this strategy,
  you might NOT be offended.

55
The Zeuthen Strategy

• Q I make a proposal in every round, but may be
  the same as last time. Do I need to make a
  concession in this round?
• A If you are not willing to risk a conflict, you
  should make a concession.

How much am I willing to risk a conflict?
How much am I willing to risk a conflict?
Agent 1's best deal
agent 2's best deal
56
Willingness to Risk Conflict

• Suppose you have conceded a lot. Then
• You have lost some of your expected utility
  (closer to zero).
• In case conflict occurs, you are not much worse
  off.
• An agent will be more willing to risk conflict
  if the difference in utility between the loss in
  making an concession is greater than the loss in
  taking a conflict deal with respect to the
  current offer.
• If both are equally willing to risk, both concede.

57
Risk Evaluation

• You have to calculate?
• How much you will lose if you make a concession
  and accept your opponent's offer?
• How much you will lose if you stand still which
  causes a conflict?

utility agent i loses by conceding and accepting
agent j's offer

• riski

utility agent i loses (from current offer, not
ideal) if conflict
Utilityi (?i )-Utilityi (?j )


Utilityi (?i )
where ?i and ?i are the current offer of agent i
and j, respectively. risk is willingness to risk
conflict (1 is perfectly willing to risk)
58
Risk Ratio -
59
Risk Ratio
60
Risk Evaluation

• risk measures the fraction you have left to gain.
  If it is close to one, you have gained little
  (and are more willing to risk).
• This assumes you know what others utility is.
  Which is not always the case. If we have such
  perfect knowledge, we can compute the deal
  immediately. More likely, we base our decisions
  on what we THINK the others utility is.
• You may not agree with their concept of utility
  as they claim certain concessions or
  expectations.
• In the car example, your initial price would be
  important.
• What one sets as initial goal affects risk. If I
  set an impossible goal, my willingness to risk is
  always higher.

61
The Zeuthen Strategy

• Q If I concedes, then how much should I concede?
• A Enough to change the balance of risk (who has
  more to lose). (Otherwise, it will just be your
  turn to concede again at the next round) Not so
  much that you give up more than you needed to
• Q What if both have equal risk?
• A Both concede.

62
About MCP and Zeuthen Strategies

• Advantages
• Simple and reflects the way human negotiations
  work.
• Stability in Nash equilibrium if one agent is
  using the strategy, then the other can do no
  better than using it him/herself.
• Disadvantages
• Computationally expensive players need to
  compute the entire negotiation set.
• Communication burden negotiation process may
  involve several steps.

63
A one-shot Negotiation Protocol(like dividing a
candy bar)

• Protocol both agents suggest an agreement the
  one giving a
• higher product of utilities wins (flip a coin in
  case of a tie)

64
A one-shot Negotiation Protocol(like dividing a
candy bar)

• Protocol both agents suggest an agreement the
  one giving a
• higher product of utilities wins (flip a coin in
  case of a tie)
• Obvious strategy amongst the set of agreements
  with maximal
• product of utilities, propose the one that is
  best for you
• Properties This mechanism is
• efficient outcomes have maximal Nash
  product and are Pareto optimal (like MCP with
  Zeuthen Strategy)
• stable no agent has an incentive to
  deviate from the strategy (like MCP with extended
  Zeuthen Strategy)
• In addition, the one-shot protocol is also
• simple only one round is required
• But why should anyone accept to use such a
  protocol? (There is no motivation to be less
  than fair.)

65
Recap How did we get to this point?

• Both agents making several small concessions
  until an
• agreement is reached is the most intuitive
  approach to
• one-to-one negotiation.
• The Monotonic Concession Protocol (MCP) is a
  straightforward formalization of the above
  intuition. Both propose at every round.
• The Zeuthen Strategy is also motivated by
  intuition (willingness to risk conflict) and
  constitutes a stable and (almost) efficient
  strategy for the MCP.
• The one-shot protocol (together with the obvious
  strategy)
• produces similar outcomes as MCP/Zeuthen, but it
  is a much
• simpler mechanism.

66
Parcel Delivery Domain Example 2 (dont return
to dist point)
Distribution Point
Conflict Deal (a,b,c,d, a,b,c,d)
7
7
All choices are IR, as cant do worse (acbd)
is dominated by (abcd)
1
1
1
a
d
c
Negotiation Set (a,b,c,d, ?) (a,b,c),
d) (a,b, c,d) (a, b,c,d) (?,
a,b,c,d)
b
Cost function c(?)0 c(a)c(d)7 c(b)c(c
)c(a,b)c(c,d)8 c(b,c)c(a,b,c)c(b,c,d
)9 c(a,d)c(a,b,d)c(a,c,d)c(a,b,c,d)1
0
67
Parcel Delivery Domain Example 2 (Zeuthen works
here both concede on equal risk)
No Pure Deal Agent 1's Utility Agent 2's Utility
5 (a,b,c,d, ?) 0 10
4 (a,b,c), d) 1 3
3 (a,b, c,d) 2 2
2 (a, b,c,d) 3 1
1 (?, a,b,c,d) 10 0
Conflict deal 0 0
agent 1
agent 2
5
4
2
1
3
68
What bothers you about the previous agreement?

• Decide to both get (2,2) utility, rather than the
  expected utility of (0,10) for another choice.
• Is there a solution?
• Fair versus higher global utility.
• Restrictions of this method (no promises for
  future or sharing of utility)

69
State Oriented Domain

• Goals are acceptable final states (superset of
  TOD)
• Have side effects - agent doing one action might
  hinder or help another agent. Example in blocks
  world, on(white,gray) has side effect of
  clear(black).
• Negotiation develop joint plans (what they each
  do) and schedules for the agents, to help and not
  hinder other agents
• Example Slotted blocks world -blocks cannot go
  anywhere on table only in slots (restricted
  resource)
• Note how this simple change (slots) makes it so
  two workers get in each others way even if goals
  are unrelated.

70
Assumptions of SOD

• Agents will maximize expected utility (will
  prefer 51 chance of getting 100 than a sure
  50)
• Agent cannot commit himself (as part of current
  negotiation) to behavior in future negotiation.
• No explicit utility transfer (no money that can
  be used to compensate one agent for a
  disadvantageous agreement)
• Interagent comparison of utility common utility
  units
• Symmetric abilities (all can perform tasks, and
  cost is same regardless of agent performing)
• Binding commitments

71
Achievement of Final State

• Goal of each agent is represented as a set of
  states that they would be happy with.
• Looking for a state in intersection of goals
• Possibilities
• (GREAT) Both can be achieved, at gain to both
  (e.g. travel to same location and split cost)
• (IMPOSSIBLE) Goals may contradict, so no mutually
  acceptable state (e.g., both need a car)
• (NEED ALT) Can find common state, but perhaps it
  cannot be reached with the primitive operations
  in the domain (could both travel together, but
  may need to know how to pickup another)
• (NOT WORTH IT) Might be a reachable state which
  satisfies both, but may be too expensive
  unwilling to expend effort (i.e., we could save a
  bit if we car-pooled, but is too complicated for
  so little gain).

72
Examples CooperativeEach is helped by joint plan

• Slotted blocks world initially white block is at
  1 and black block at 2. Agent 1 wants black in
  1. Agent 2 wants white in 2. (Both goals are
  compatible.)
• Assume pick up is cost 1 and set down is one.
• Mutually beneficial each can pick up at the
  same time, costing each 2 Win as didnt have
  to move other block out of the way!
• If done by one, cost would be four so utility
  to each is 2.

?
73
Examples CompromiseBoth succeed, but worse for
both than if other agent gone

• Slotted blocks world initially white block is at
  1 and black block at 2, two gray blocks at 3.
  Agent 1 wants black in 1, but not on table. Agent
  2 wants white in 2, but not directly on table.
• Alone, agent 1 could just pick up black and place
  on white. Similarly, for agent 2. But would undo
  others goal.
• But together, all blocks must be picked up and
  put down. Best plan one agent picks up black,
  while other agent rearranges (cost 6 for one, 2
  for other)
• Can both be happy, but unequal roles.

?
74
Example conflict

• I want black on white (in slot 1)
• You want white on black (in slot 1)
• Cant both win. Could flip a coin to decide who
  wins. Better than both losing. Weightings on
  coin neednt be 50-50.
• May make sense to have person with highest worth
  get his way as utility is greater. (Would
  accomplish his goal alone) Efficient but not
  fair?
• What if we could transfer half of the gained
  utility to the other agent? This is not normally
  allowed, but could work out well.

75
Negotiation Domains Worth-oriented

• Domains where agents assign a worth to each
  potential state (of the environment), which
  captures its desirability for the agent,
  (Rosenschein Zlotkin, 1994)
• agents goal is to bring about the state of the
  environment with highest value
• we assume that the collection of agents have
  available a set of joint plans a joint plan is
  executed by several different agents
• Note not all or nothing but how close you
  got to goal.

76
Worth Oriented Domain

• Rates the acceptability of final states
• Allows partially completed goals
• Negotiation a joint plan, schedules, and goal
  relaxation. May reach a state that might be a
  little worse that the ultimate objective
• Example Multi-agent Tile world (like airport
  shuttle) isnt just a specific state, but the
  value of work accomplished

77
How can we calculate Utility?

• Weighting each attribute
• Utility Price60 quality15 support25
• Rating/ranking each attribute
• Price 1, quality 2, support 3
• Using constraints on an attribute
• Price5,100, quality0-10, support1-5
• Try to find the pareto optimum

78
What if choices dont benefit others fairly?

• Suppose there are two states that satisfy both
  agents.
• State 1 one has a utility of 6 for one agent and
  3 for the other.
• State 2 utility of both agents 4.
• State 1 is better (overall), but state 2 is more
  equal. How can we get cooperation (as why should
  one agent agree to do more)?

79
Mixed Deal

• If ? (J1, J2p) is a deal, then
• utili(?) putil(J)i (1-p)util(J)k where k
  is is opponent -the role i plays with (1-p)
  probability
• An all or nothing form of a mixed deal simply
  means one set of tasks is everything.

80
Parcel Delivery Domain (assuming noreturn)
Distribution Point
Cost function c(?)0 c(a)1 c(b)1 c(a,b)3
1
1
city a
city b
2

• Utility for agent 1 (org a)
• Utility1(a, b) 0
• Utility1(b, a) 0
• Utility1(a, b, ?) -2
• Utility1(?, a, b) 1

• Utility for agent 2 (org ab)
• Utility2(a, b) 2
• Utility2(b, a) 2
• Utility2(a, b, ?) 3
• Utility2(?, a, b) 0

81
At seats

• For the parcel delivery example above, show what
  a mixed deal does for the following deals
• 1. deal 1, each does one
• 2. deal 3, all or nothing (notice a pure
  deal3 is not even individually rational)

82
Consider deal 3 with probability

• (,ab)p means agent 1 does with p
  probabilty and ab with (1-p) probabilty.
• What should p be to be fair to both (equal
  utility)
• (1-p)(-2) p1 utility for agent 1
• (1-p)(3) p0 utility for agent 2
• (1-p)(-2) p1 (1-p)(3) p0
• -22pp 3-3p gt p5/6
• If agent 1 does no deliveries 5/6 of the time, it
  is fair.

83
Try again with other choice in negotiation
(Deal 1) Utility1(a, b) 0

• (a,b)p means agent 1 does a with p
  probabilty and b with (1-p) probabilty.
• What should p be to be fair to both (equal
  utility)
• (1-p)(0) p0 utility for agent 1
• (1-p)(2) p2 utility for agent 2
• 02 no solution
• Can you see why we cant use a p to make this
  fair?

84
Incomplete Information

• Dont know tasks of others in TOD.
• Solution
• Exchange missing information
• Penalty for lie
• Possible lies (notice reduce possibilities in
  order to be able to solve)
• False information
• Hiding letters (dont admit part of your job)
• Lie about letters (claim work that isnt
  required)
• decoy produce if needed
• phantom cant produce, caught in lie
• Not carry out a commitment

85
Subadditive Task Oriented DomainCost of whole is
cost of parts

• for finite X,Y in T, c(X U Y) lt c(X) c(Y)).
• Example of subadditive
• Deliver to one, saves distance to other (in a
  tree arrangement)
• Example of subadditive TOD ( rather than lt)
• deliver in opposite directions
• doing both saves nothing
• Not subadditive doing both actually costs more
  than the sum of the pieces. Say electrical power
  costs, where I get above a threshold and have to
  buy new equipment.

86
Decoy task

• We call producible phantom tasks decoy tasks (no
  risk of being discovered). Only unproducible
  phantom tasks are called phantom tasks.
• Example Need to pick something up at store.
  (Can think of something for them to pick up, but
  if you are the one assigned, you wont bother to
  make the trip.)
• Need to deliver empty letter (no good, but
  deliverer wont discover lie)

87
Incentive compatible MechanismAre the rules (in
terms of allowable deals) we establish sufficient
to produce truth telling?

• L ?there exists a beneficial lie in some
  encounter
• T ? There exists no beneficial lie.
• T/P ? Truth is dominant if the penalty for lying
  is stiff enough.

Example indicates a case where lying helps. Can
you see it? Who lies? What is lie?
88
Explanation of arrow

• If it is never beneficial in a mixed deal
  encounter to use a phantom lie (with penalties),
  then it is certainly never beneficial to do so in
  an all-or-nothing mixed deal encounter (which is
  just a subset of the mixed deal encounters).

89
Concave Task Oriented Domain

• We have 2 tasks X and Y, where X is a subset of Y
• Another set of task Z is introduced
• c(YU Z) c(Y) c(XU Z) c(X)

90
Modularity c(X U Y) c(X) c(Y) - c(X
?Y). Notice modular encourages truth telling,
more than others

• Concave
• c(YU Z) c(Y) c(XU Z) c(X)
• The cost of tasks Z adds to set of tasks Y
  cannot be greater than the cost Z add to a subset
  of Y
• Expect it to add more to subset (as is smaller)
• At seats is postmen doman concave (no, unless
  restricted to trees)
• Example Y is in pacman shape, X is nodes in
  polygon.
• adding Z adds 0 to X (as was going that way
  anyway) but adds 2 to its superset Y (as was
  going around loop)
• Concavity implies sub-additivity
• Modularity implies concavity

y
91
Explanation of Previous Chart

• Arrows show reasons we know this fact (diagonal
  arrows are between domains). For example, What is
  true of a phantom task, may be true for a decoy
  task in same domain as a phantom is just a decoy
  task we dont have to create.
• Similarly, what is true for a mixed deal may be
  true for an all or nothing deal (in the same
  domain) as a mixed deal is an all or nothing deal
  where one choice is empty. The direction of the
  relationship may depend on truth (never helps) or
  lie (sometimes helps).
• The relationships can also go between domains as
  sub-additive is a superclass of concave and a
  super class of modular.

92
Modular TOD

• c(X U Y) c(X) c(Y) - c(X ?Y).
• Notice modular encourages truth telling, more
  than others

93
Implied relationship between cells Implied
relationship between domains (slanted arrows).L
means lying may be beneficialT means telling the
truth is always beneficialT/P Truth telling is
beneficial if penalty for being caught is great
94
Attributes-Modularity

• c(XU Y) c(X) c(Y) c(XnY)
• The cost of the combination of 2 sets of tasks
  is exactly the sum of their individual costs
  minus the cost of their intersection
• Only Fax Domain is modular (as costs are
  independent)
• Modularity implies concavity

95
Incentive Compatible Facts (return home)

• Fact1 in SubadditiveTOD, any Optimal Negotiation
  Mechanism (ONM) over A-or-N deals, hiding lies
  are not beneficial
• ExA1hides letter to c, his utility doesnt
  increase.
• If he tells truth p1/2
• Expected util (abc)1/2 5
• Lie p1/2 (as utility is same)
• Expected util (for 1) (abc)1/2 ½(0) ½(2)
  1 (as has to deliver the lie)

1
4
4
1
96

• Fact2 in SubadditiveTOD, any ONM over Mixed
  deals, every phantom lie has a positive
  probability of being discovered. (as if other
  person delivers phantom, you are found out)
• Fact3 in Concave TOD, any ONM over Mixed deals,
  no decoy lie is beneficial. (as less increased
  cost is assumed so probabilities would be
  assigned to reflect the assumed extra work)
• Fact4 in Modular TOD, any ONM over Pure deals, no
  decoy lie is beneficial. (modular tends to add
  exact cost hard to win)

97
Fact4 Modular, all or nothing, decoy
1 U(1) 2 U(2) Seems U(2) (act)
eb 0 ebc 0 0
ebc -2 ? 8 6
? 6 ebc 0 0
Both deliver to e and b. Suppose agent 2 lies
about having a delivery to c. Under Lie
benefits are shown ? Under Truth, p would be
1/2 If we assign p (ebc, ?) p agent 1
utility -2p 6(1-p) Agent 2 (under lie)
8p0(1-p) -2p 6(1-p) 8p0(1-p) -8p6 8p
p6/16 (so 2 is worse off)
98

• Fact5 in Concave TOD, any ONM over Pure deals,
  Phantom lies can be beneficial.
• Example from next slideA1creates Phantom letter
  at node c, his utility has risen from 3 to 4
• Truth p ½ so utility for agent 1 is (ab) ½
  ½(4) ½(2) 3
• Lie (bca) is logical division as no percent
• Util for agent 1 is 6 (org cost) 2(deal cost)
  4

99

• Fact6 in SubadditiveTOD, any ONM over A-or-N
  deals, Decoy lies can be beneficial (not
  harmful). (as it changes the probability. If you
  deliver, I make you deliver to h)
• Ex2 (from next slide)A1lies with decoy letter to
  h (trying to make agent 2 think picking up bc is
  worse for agent 1 than it is), his utility has
  rised from 1.5 to 1.72. (If I deliver, I dont
  deliver h)
• If tells truth, p (of agent 1 delivering all)
  9/14 as
• p(-1) (1-p)6 p(4) (1-p)(-3) ? 14p9
• If invents task h, p11/18 as
• p(-3) (1-p)6 p(4) (1-p)(-5)
• Utility(p9/14) is p(-1) (1-p)6 -9/14 30/14
  21/14 1.5
• Utility(p11/18) is p(-1) (1-p)6 -11/18
  42/18 31/18 1.72
• SO lying helped!

100
Postmen return to postoffice
Concave
Phantom
Subadditive (h is decoy)
101

• Fact7 in Modular TOD, any ONM over Pure deals,
  Hide lie can be beneficial. (as you think I
  have less, so increase load will cost more than
  it realy does)
• Ex3 (from next slide) A1 hides his letter node b
• (eb) utility for A1 (under lie) is 0
• utility for A2 (under lie) is 4
  UNFAIR (under lie)
• (be) utility for A1 (under lie) is 2
• utility for A2 (under lie) is 2
• So I get sent to b, but I really needed to go
  there anyway, so my utility is actually 4. (as I
  dont go to e)

102

• Fact8in Modular TOD, any ONM over Mixed deals,
  Hide lies can be beneficial.
• A1 hides his letter to node a
• A1s Utility is 4.5 gt 4 (Utility of telling the
  truth)
• Under truth Util(faebcd)1/2 4 (save going
  to two)
• Under lie divide as (efdcab)p (you always win
  and I always lose. Since work is same, swapping
  cannot help. In a mixed deal, the choices must
  be unbalanced.
• Try again, under lie (abcdef)p
• p(4) (1-p)(0) p(2) (1-p)(6)
• 4p -4p 6
• p 3/4
• Utility is actually
• 3/4(6) 1/4(0) 4.5
• Note, when I get assigned cdef ¼ of the time, I
  STILL have to deliver to node a (after completing
  by agreed upon deliveries). So I end up going 5
  places (which is what I was assigned originally).
  Zero utility to that.

103
Conclusion

• In order to use Negotiation Protocols, it is
  necessary to know when protocols are appropriate
• TODs cover an important set of Multi-agent
  interaction

104
MAS Compromise Negotiation process for
conflicting goals

• Identify potential interactions
• Modify intentions to avoid harmful interactions
  or create cooperative situations
• Techniques required
• Representing and maintaining belief models
• Reasoning about other agents beliefs
• Influencing other agents intentions and beliefs

105
PERSUADER case study

• Program to resolve problems in labor relations
  domain
• Agents
• Company
• Union
• Mediator
• Tasks
• Generation of proposal
• Generation of counter proposal based on feedback
  from dissenting party
• Persuasive argumentation

106
Negotiation Methods Case Based Reasoning

• Uses past negotiation experiences as guides to
  present negotiation (like in court of law cite
  previous decisions)
• Process
• Retrieve appropriate precedent cases from memory
• Select the most appropriate case
• Construct an appropriate solution
• Evaluate solution for applicability to current
  case
• Modify the solution appropriately

107
Case Based Reasoning

• Cases organized and retrieved according to
  conceptual similarities.
• Advantages
• Minimizes need for information exchange
• Avoids problems by reasoning from past failures.
  Intentional reminding.
• Repair for past failure is used. Reduces
  computation.

108
Negotiation Methods Preference Analysis

• From scratch planning method
• Based on multi attribute utility theory
• Gets a overall utility curve out of individual
  ones.
• Expresses the tradeoffs an agent is willing to
  make.
• Property of the proposed compromise
• Maximizes joint payoff
• Minimizes payoff difference

109
Persuasive argumentation

• Argumentation goals
• Ways that an agents beliefs and behaviors can be
  affected by an argument
• Increasing payoff
• Change importance attached to an issue
• Changing utility value of an issue

110
Narrowing differences

• Gets feedback from rejecting party
• Objectionable issues
• Reason for rejection
• Importance attached to issues
• Increases payoff of rejecting party by greater
  amount than reducing payoff for agreed parties.

111
Experiments

• Without Memory 30 more proposals
• Without argumentation fewer proposals and
  better solutions
• No failure avoidance more proposals with
  objections
• No preference analysis Oscillatory condition
• No feedback communication overhead increased by
  23

112
Multiple Attribute Example

• 2 agents are trying to set up a meeting. The
  first agent wishes to meet later in the day while
  the second wishes to meet earlier in the day.
  Both prefer today to tomorrow. While the first
  agent assigns highest worth to a meeting at
  1600hrs, s/he also assigns progressively smaller
  worths to a meeting at 1500hrs, 1400hrs.
• By showing flexibility and accepting a
  sub-optimal time, an agent can accept a lower
  worth which may have other payoffs, (e.g. reduced
  travel costs).

Worth function for first agent
Ref Rosenschein Zlotkin, 1994
113
Utility Graphs - convergence

• Each agent concedes in every round of negotiation
• Eventually reach an agreement

114
Utility Graphs - no agreement

• No agreement

115
Argumentation

• The process of attempting to convince others of
  something.
• Why argument-based negotiationsgame-theoretic
  approaches have limitations
• Positions cannot be justified Why did the agent
  pay so much for the car?
• Positions cannot be changed Initially I wanted
  a car with a sun roof. But I changed preference
  during the buying process.

116

• 4 modes of argument (Gilbert 1994)
• Logical - If you accept A and accept A implies
  B, then you must accept that B
• Emotional - How would you feel if it happened to
  you?
• Visceral - participant stamps their feet and show
  the strength of their feelings
• Kisceral - Appeals to the intuitive doesnt
  this seem reasonable

117
Logic Based Argumentation

• Basic form of argumentation
• Database (Sentence,Grounds)
• Where
• Database is a (possibly inconsistent) set of
  logical formulae
• Sentence is a logical formula know as the
  conclusion
• Grounds is a set of logical formula
• grounds ? database
• sentence can be proved from grounds
• (we give reason for our conclusions)

118
Attacking Arguments

• Milk is good for you
• Cheese is made from milk
• Cheese is good for you
• Two fundamental kinds of attack
• Undercut (invalidate premise) milk isnt good
  for you if fatty
• Rebut (contradict conclusion) Cheese is bad for
  bones

119
Attacking arguments

• Derived notions of attack used in Literature
• A attacks B A u B or A r B
• A defeats B A u B or (A r B and not B u A)
• A strongly attacks B A a B and not B u A
• A strongly undercuts B A u B and not B u A

120
Proposition Hierarchy of attacks
Attacks a u ? r
Defeats d u ? ( r - u -1)
Undercuts u
Strongly attacks sa (u ? r ) - u -1
Strongly undercuts su u - u -1
121
Abstract Argumentation

• Concerned with the overall structure of the
  argument (rather than internals of arguments)
• Write x ? y indicates
• argument x attacks argument y
• x is a counterexample of y
• x is an attacker of y
• where we are not actually concerned as to what x,
  y are
• An abstract argument system is a collection or
  arguments together with a relation ? saying
  what attacks what
• An argument is out if it has an undefeated
  attacker, and in if all its attackers are
  defeated.
• Assumption true unless proven false

122
Admissible Arguments mutually defensible

1. argument x is attacked if no member attacks y and
   y?x
2. argument x is acceptable if every attacker of x
   is attacked
3. argument set is conflict free if none attack each
   other
4. set is admissible if conflict free and each
   argument is acceptable (any attackers are
   attacked)

123
a
d
c
b
Which sets of arguments can be true? c is always
attacked. d is always acceptable
124
An Example Abstract Argument System