Structured Models for Multi-Agent Interactions

1
Structured Models for Multi-Agent Interactions

• Daphne Koller
• Stanford University
• Joint work with David Vickrey

2
Nash Equilibrium

• Strategy profile ? strategy for every player
• Regret?(pi) pis gain by changing strategy ?i
• Nash equilibrium ? s.t. each agent has 0 regret
• Theorem (Nash) Every game has at least one Nash
  equilibrium

-

Best?
Actual?
Regret?
3
Finding Nash Equilibria

• Nash equilibria difficult to compute in games
  with more than 2 agents
• Best current game solving package (GAMBIT)
• 2 hrs 30 min for 6-player 3-action game
• Game representation exponential in of agents
• Algorithms inherently centralized

Our approach

• Structured game representation
• Find approximate equilibria

Fast, decentralized algorithms
4
Graphical Games

• Agent pis utility depends on only ki other
  agents
• Represented as directed cyclic graph
• pis utility depends on Parents(i)

Example property development along a road
Koller Milch Kearns, Littman Singh
5
Approximate Equilibria
e-optimal Nash equilibrium each agents
regret ? e
6
Constraint Satisfaction

• Constraint each agent has zero regret
• Constraints are local since regret is local
• Each involves only node and its parents

7
Solving the CSP

• Problem Strategies are continuous
• Solution
• Discretize strategy space of each player
• Constraint regret of agent ? ? ? ? in table
• Produces ?-approximate equilibria
• The finer the discretization, the lower the ?

8
Variable Elimination

• Key algorithm for both CSPs and BN inference
• Idea Eliminate variables one by one
• Entry in new table is ? iff
• there exists strategy for eliminated variable T
  s.t.
• for all tables containing T the matching entry
  is?

9
Cost Minimization

• Problem Discretization too sparse ? no
  ?-equilibrium
• Solution replace ? in constraint tables with
  regrets
• To eliminate T
• entry in new table is min over all strategies of
  T
• of max over all tables
• Finds best ?-equilibrium for given discretization

10
Experimental Results
Outer ring size 20
Ring of Rings
Execution Time (sec)
Equilibrium quality (?)
60
0.035
0.03
50
0.025
40
0.02
30
0.015
20
0.01
10
0.005
0
0
0
2
4
6
8
10
12
14
16
18
20
0
2
4
6
8
10
12
14
16
18
20
Nodes in Internal Ring
Compare to 2 hrs 30 min for 6-player 3-action
game
11
Conclusion

• Finding equilibria is hard
• Computationally complex
• Requires centralized solution
• Used graphical language, similar to BNs, to
  represent structure
• Adapted graphical algorithm to find equilibria
• Significantly more efficiently
• In a decentralized way