Modeling Complex Multi-Issue Negotiations Using Utility Graphs

1
Modeling Complex Multi-Issue Negotiations Using
Utility Graphs

• Valentin Robu, Koye Somefun, Han La Poutré
• CWI, Center for Mathematics and Computer Science,
  Amsterdam, The Netherlands

2
Multi-issue (multi-item) negotiation

• Negotiation method of competitive (or partially
  cooperative) allocation of goods, resources,
  tasks between agents
• Applications
• E-commerce Bundling can be an effective method
  to increase sales (use in recommender systems)
• High degree of customization possible through
  negotiations
• Logistics mechanism for task allocation
• Many deals are negotiated bilaterally or in
  closed groups of companies (e.g. transportation
  contracts)
• Utility functions are not (or partially) revealed
  gt indirect revelation mechanism
• Search with incomplete information

3
Utility functions for multi-issue negotiations

• Linearly additive
• Linear combination of issue utilities
• Search space is structured -gt more accesible to
  heuristics Faratin Sierra Jennings. 2002,
  Jonker Robu 2004, Coehoorn Jennings 2004
  Gerding La Poutre, 2004
• Auction-type XOR of ANDs
• K-additive
• Captures local substitutability/complementarity
  effects between k issues
• Finding optimal allocation can become hard even
  for the 2-additive case
• Exiting solutions assume a trusted mediator,
  computationally expensive (3000-5000 bids for 50
  issues)
• Klein, Faratin, Sayama Bar-Yam, 2003 Lin
  2004

4
Utility graphs basic ideas

• Inspiration probabilistic graphical models
• Each node one issue under negotiation (or item
  in a bundle)
• Nodes grouped into clusters of connected nodes
• Cost of representation
• Exponential in size of the cluster
• Linear in the number of clusters
• Use in negotiation
• Opponent modelling seller maintains updates a
  model of buyers preferences

5
Utility graphs an example

• Global utility is a sum of utility over clusters,
  rather than individual issues
• Buyer - cluster potentials
• u(I1) 7, u(I2) 5, u(I3) 0
• u(I4) 0, u(I1, I2) - 5,
• u(I2, I3)4, u(I2, I4)4
• Seller - all items have cost 2.
• uBUYER(I11, I20, I31, I40) 7
• Gains from Trade Buyer_utility Seller_Cost
• Optimal combination?

GT(I10, I21, I31, I41)13 - 32 7
6
Utility graphs Use in negotiation

• Bundles with maximal G.T. ? Pareto-optimal
  bundles Somefun, Klos La Poutré 2004
• Seller keeps a model of the utility graph of the
  buyer and aims for a bundle with maximal GT
• After each counter-offer, he updates this model
  (true graph of the buyer remains hidden)
• Seller knows a super-graph of possible buyer
  utility graphs (qualitative assumption)

7
Partitioning a utility graph

• Q How to select the bundle with a maximal GT,
  with respect to a utility graph learned so far?
• A1 (Brute force answer) generate all possible
  bundles and select the best one.
• Complexity for 50 issues 250 gt 1015 bundles
• A2 Partition the graph into sub-graphs
• Nodes belonging to more than 1 subgraph cutset
  nodes
• For all possible instantiations of cutset nodes,
  compute local sub-bundle combination
• Merge them, such that a local optimum is achieved

8
Partitioning a utility graph (2)

• Complexity of exploring all bundles 2c (2p
  2q)
• Partitions can be found in polynomial time
  (always for graphs of tree-width 2)

9
Learning in utility graphs (1)

• Seller has a super-graph for possible inter-
  dependencies in the buyer population
• This graph contains tables for each cluster, with
  size 2 at the power of size of the cluster
• Initial values proportional to the Hamming
  distance
• Values are adjusted as follows

, for the combination induced from
buyers bid , for all other combinations
10
Learning a simple example

• Two complementary issues I1 and I2

I1 I2 time t t1 t2
0 0 0 0 0
0 1 7 8.4 10
1 0 5 4 3.2
1 1 17 13.6 10.9
Buyer asks, for several rounds I10, I21 This
combination gets updated with (1a), the others
with (1-a)

• Supposing costs are c(I1)c(I2)3, a0.2 the
  bundle with maximal GT changes from (1,1) to
  (0,1) after 2 steps

11
Learning in utility graphs (2)

• The cluster update factor is clique-specific
• C total number of cliques a, ß learning
  parameters
• Where the clique Gains from Trade Ratio is
  defined as ratio of local (per clique) vs.
  total (bundle-wide) GT
• We adjust the model more towards the others
  value for clusters which are less important, and
  less for the others

12
Experimental validation set-up

• Graph with 50 issues, 28 clusters 3 of size 4,
  16 of size 3, 6 of size 2, 3 of size 1
• Costs and strength of interdependencies drawn
  from a independent, normal distributions
  (i.i.d-s)
• Means around 1(Hamming Distance)
• Spreads between 0 and 5
• gt highly non-linear search space
• Results averaged for 100 tests/configuration

13
Experimental results
14
Negotiation part Conclusions

• It is possible to reach Pareto-efficient outcomes
  reasonably fast, by exploiting the decomposable
  structure of utility functions
• Consequence
• We can handle complex negotiations even in time
  constrained domains / with buyer impatience
• Assumption A structure of the super-graph for
  the population of likely buyers
• Solution collaborative filtering past
  negotiation data

15
Structure of the initial utility graph

• Preferences of buyers are in some way clustered
• Class (population) of buyers with similar
  preference structures gt largely overlapping
  utility graphs
• Can we estimate which items can be potentially
  complementary/substitutable by looking at
  previous buying patterns?
• Collaborative filtering asks the same questions !
• Not all relationships hold for all users only a
  super-graph of these relationships is required

16
Architecture simulation model view
17
Collaborative filtering Overview

• Output recommendations to buyers, based on
  previous buy instances
• User-based for each user, select a neighbourhood
  of users with a similar preferences
• Item-based identify relationships between items,
  based on previous buying patterns
• In our case, recommendation step is completely
  replaced by negotiation gt more customization
  possible

18
Step 1 Data preparation
4 Item-item matrixes
Negotiation outcomes matrix
Item pairs I1 I2 IK... I50
I1 N 134 220
I2 134 N
IK
I50 220 N
Items Items Items Items
Previous negotiations I1 I2 IK... I50
Neg. 1 0 1 1 0
Neg. 2 1 1 0 1
Neg. N (eg. N2000) 1 1 0 0

• 1-1 pairs Ni,j(1,1)
• 1-0 pairs Ni,j(0,1)
• 0-1 pairs Ni,j(1,0)
• 0-0 pairs Ni,j(0,0)

Total no. buys (out of N) N1(1) N2(1) NK(1).. N50(1)
Total no. buys (out of N) 260 130 50
19
Step 2 Data analysis (1)

• Compute item-item similarity, based on the
  appearance data

4 Item-item matrixes
Cosine / correlation matrix
Item I1 IK... I50
I1 N 220
IK
I50 220 N
Item I1 IK... I50
I1 1 0.84
IK 0.23
I50 0.84 1
Total number buys/item

• 2 matrixes for cosine-based similarity
• 1 matrix for correlation- based similarity

Number Buys / item N1(1) N50(1)
Number Buys / item 260 50
20
Criteria 1 Cosine-based similarity

• Measure of distance between the buying vectors
  for two items i, j
• Intuitive, but not so precise
• Complementarity effect
• Substitutability effect

21
Criteria 2 Correlation-based similarity

• Average buys per item
• Similarity between items i and j

22
Results Correlation-based similarity
23
Conclusions discussion

• Utility graphs efficient way to guide online
  learning of buyer preferences in electronic
  negotiations
• Learning a starting structure of these graphs
  possible through collaborative filtering
• By combining the two techniques gt relatively
  short negotiations (around 20 steps/50 issues)
• Intuition we explicitly utilize the clustering
  effect between utility functions of typical
  buyers
• Personalization techniques used in collaborative
  filtering can be successfully combined with
  personalization through agent-mediated negotiation

24
Questions

• Thank you very much for your attention!
• Full paper(s) available from
• homepages.cwi.nl/robu