The Computational Complexity of Finding Nash Equilibria

1
The Computational Complexityof Finding Nash
Equilibria

• Edith Elkind
• Intelligence, Agents, Multimedia group (IAM)
• School of Electronics and CS
• U. of Southampton

2
Games and Strategies

• Games strategic interactions between rational
  entities
• Solution concepts whats going to happen?
• dominant strategies
• Nash equilibrium
• .
• Can it be computed?
• if your computer cannot find it, the market
  probably cannot either

3
Matrix (Normal Form) Games

• finite set of players 1, , n
• each player has k actions
• (pure strategies) 1, , k
• payoffs of the ith player Pi 1, , kn ? R

Row player
Column player
4
Nash Equilibrium

• Nash equilibrium a strategy profile such that
• noone wants to deviate given other players
  strategies, i.e., each players strategy is a
  best response to others strategies
• (0, 0) and (1, 1) are both NE

Row player
Column player
5
Pure vs. Mixed Strategies

• NE in pure strategies may not exist!
• matching pennies
• Mixed strategy a probability distribution over
  actions
• 50 tail, 50 head

Row player
Column player
6
Existence of NE

• Theorem (Nash 1951)
  any n-player k-action game
  in normal form has an equilibrium
  in mixed strategies
• can we find one in poly-time?

7
Plan of the Talk
2 players, k actions
8
2 (rconst) players, k actions

• Input representation
• 2 players two k x k matrices
• r players r k x k x x k matrices
• poly-size for constant r
• Output representation
• for 2 players all NE are in Q
• but not for 3 and more players
• Checking for pure NE easy
• at most k2 strategy profiles

9
2 players, k actions
mixed NE

• Naïve approaches exp(k)
• Simplex-like approach
  (Lemke-Howson algorithm)
• works well in practice
• exp(k) in the worst case (2004)
• Is it time to give up?
• maybe the problem is NP-hard?

10
Is Finding NE NP-hard?

• Reminder a problem P is NP-hard if you can
  reduce 3-SAT to it
• yes-instance 3-SAT ? yes-instance of P
• no-instance 3-SAT ? no-instance of P
• Problem each instance of NASH is
  a yes-instance!
• every game has a NE
• Formally if NASH is NP-hard then NP coNP
• Need complexity theory for
  total search problems

11
Reducibility Among Search Problems
S X Y
T X Y

• S associates x in X with a solution set S(x)
• Total search problem for any x, S(x) is not empty

If T is easy, so is S
12
Completeness Results?

• Can we prove that any total search problem is
  reducible to r-NASH?
• Not really the class T of all total search
  problems is a semantic class
• not known how to find complete problems for these
• Want to pick a large subclass S of T s.t.
• S includes some natural problems
• there are problems that are complete for S
• in particular, r-NASH is complete for S

13
END OF THE LINE

• Input Boolean circuits
  S (Successor), P
  (Predecessor)
• n inputs, n outputs
• S(0n) ? 0n, P(0n) 0n
• Output x ? 0n s.t.
• S(P(x)) ? x or P(S(x)) ? x
• Intuition G(V, E)
• V Sn
• E (x,y) yS(x), xP(y)

00000
11001
01011
01011
14
PPAD

• PPAD Polynomial Parity Argument, Directed
  version
• PPAD is the class of all search problems that are
  reducible to END OF THE LINE

search problem solution
g
f
circuits S, T end of the
line
15
r-NASH is in PPAD

• Proof on Nashs theorem
• existence of NE reduces to Brouwers fixpoint
  theorem
• Brouwers fixpoint theorem reduces to Sperners
  lemma
• Sperners lemma is proven by a parity argument
  (similar to END OF THE LINE)
• Reduction of r-NASH to END OF THE LINE can be
  extracted from these proofs (Papadimitriou 94)

16
Brouwers Fixpoint Theorem

• Brouwers Theorem Any continuous mapping from
  the simplex to itself has a
  fixpoint.
• Nash ? Brouwer proof sketch
• set of all strategy profiles ? simplex
• mapping (s1, , sn) ? (s1d1, , sndn), where
  di is a shift in the direction of
  best response to (s1, , si-1, si1, , sn)
• NE is a point where noone wants to deviate, i.e.,
  a fixpoint

17
Sperners Lemma

• Proper coloring
• vertices on BC are not blue
• vertices on AC are not green
• vertices on AB are not yellow
• Sperners Lemma
  there exists a trichromatic
  triangle
• Brouwers theorem ? Sperners Lemma
• x is blue if the grad(F) at x points away from A,
  etc.
• trichromatic triangle has no direction
• repeat at increased resolution

18
Reductions (Papadimitriou 1994)

• END OF THE LINE is PPAD-complete
• TRICHROMATIC TRIANGLE is PPAD-complete
• 3D-BROUWER is PPAD-complete
• r-NASH is in PPAD

19
r-NASH vs 3D BROUWER

• Existence of NE follows from Brouwers fixpoint
  theorem
• NE are special cases of Brouwers fixpoints
• just how special?
• Can any fixpoint be represented
  as a NE of a game?
• Is there a reduction
  from 3D BROUWER to r-NASH?

20
Hardness Reductions the Timeline

• 3D-BROUWER is PPAD-complete (Papadimitriou, 1994)
• 4-NASH is PPAD-complete (Daskalakis,Goldberg,
  Papadimitriou, Sep 2005)
• 3-NASH is PPAD-complete
  (Daskalakis, Papadimitriou, Oct 2005,
  Chen, Deng, Nov 2005)
• 2-NASH is PPAD-complete !!!
  (Chen, Deng, Dec 2005)

21
n players, 2 actions

• representation payoffs to each player for every
  action profile (vector in 0, 1n) n2n numbers
• graphical games
• players are vertices of a graph
• Vs payoff depends on actions of W in N(V) U V
• n players, max degree d gt n2d1 numbers

t0, u0, v0, w0 12 t1, u0, v0, w0 31
. t1, u1, v1, w1 -6
W
Ws payoffs (16 cases)
T
V
U
22
Algorithms What Was Known

• Bounded-degree trees
• Exp-time algorithm/poly-time approximation
  algorithm to find all NE (Kearns, Littman, Singh,
  UAI 2001)
• ??? poly-time algorithm to find a single NE
  (Kearns, Littman, Singh, NIPS2001)
• Heuristics for graphs with cycles

23
Our Results
(E., Goldberg, Goldberg06)

• Algorithm in NIPS01 paper is incorrect (does not
  always output a NE)
• We fix the NIPS01 algorithm, but
• our algorithm runs in poly-time on paths
• with a trick, also on cycles
• There is a graph of pathwidth 2 on which our
  algorithm runs in exp time
• true for all algorithms that use the basic
  approach of the UAI01 paper

24
Warm-up 2-player 2-action games
Row player
Column player
BR(C)
Suppose R plays 1 w.p. r EP(C) from playing 0
(1-r)1 EP(C) from playing 1 r3 1-r gt 3r
iff r lt ¼
Suppose C plays 1 w.p. c EP(R) from playing 0
(1-c)2 EP(R) from playing 1 c1 (1-c)2 gt c
iff c lt 2/3
c
1
r
1
mixed NE r1/4, c2/3
25
Algorithm for Trees (KLS01)

• Potential best response v is a PBR to w
  iff when W plays w, there is a NE
  for TV in which V
  plays v.
• upstream pass construct PBRV(w)
  from PBRU1(v), PBRU2(v) and PBRU3(v)
• downstream pass root selects its strategy based
  on the childrens PBRs propagates to leaves

W
V
v
TV
U3
U1
w
26
Computing PBR on a Path

• E0 EP(V) from playing 0
  
  (1-u)(1-w)v000(1-u)wv001u(1-w)v100uwv101
  auwbucwd
• E1 EP(V) from playing 1
  (1-u)(1-w)v010(1-u)wv011u(1-w)
  v110uwv111 auwbucwd
• E0 E1 iff w (AuB)/(CuD) f(u)

v
u
1
1
(v, u) ? (f(u), v)
.5
PBRU(v)
PBRV(w)
.5
1
w
v
1
.1
.9
27
Trees too many segments
W
v
u
t
u2
t2
v1
V
v2
u1
t1
v
v
w
v1
v2
v1
v2
T
U
(v,t), (v,u) ? (f(u,t), v)
Incorrect!
KLS (NIPS01) can trim
PBR
28
Solutions?

• Solution 1 (for paths) algorithm of UAI01
  paper, careful analysis
• the number of segments/rectangles in each PBR is
  O(n2)
• running time O(n3)
• Solution 2 (for paths) can pick a subset of each
  PBR consisting of O(n) segments
• O(n2) running time

29
Extension to trees?
V0
V1
V2
Vn-1
Vn
U1
Un-1
U2
Un
T1
T2
Tn-1
Tn
30
Graphical games hardness results

• NP-hard?
• no total search problem
• PPAD-hard?
• yes!
• in fact, this is how the hardness result for
  4-player games was obtained
  (Goldberg, Papadimitriou, Aug 2005)

31
Equivalences GP05
deg d graphical game G NE of G
d2-player game G NE of G
32
Combining Reductions GP05
33
PPAD-hardness missing details

• 3D-Brouwer is PPAD-complete
  (Papadimitriou, 1994)
• 4-NASH is as hard as deg 3-GG
  (Goldberg, Papadimitriou, Aug 2005)
• deg 3-GG is PPAD-complete and hence
• 4-NASH is PPAD-complete
  (Daskalakis,Goldberg, Papadimitriou, Sep 2005)
• 3-NASH is PPAD-complete
  (Daskalakis, Papadimitriou, Oct 2005,
  Chen, Deng, Nov 2005)
• 2-NASH is PPAD-complete !!!
  (Chen, Deng, Dec 2005)

34
NE with special properties

• Pure NE
• easy for constant number of players
• NP-hard for general graphical games
• even if max degree 3
• NP vs. PPAD pure NE may not exist!
• poly-time on trees (KLS algorithm)
• also on graphs with bounded treewidth

35
Welfare-Maximizing NE
Row player
Column player

• Nash equilibria
• (0, 0) total payoff is 3
• (1, 1) total payoff is 4
• (1/4, 2/3) total payoff is 17/12
• not all NE are created equal

36
Algorithms for Good NE

• 2-player games
  checking for NE with total payoff gt T
  is NP-hard
  (Gilboa Zemel 89, Conitzer,
  Sandholm 03)
• Graphical games
• - for any algebraic a, deg(a) n, there is a
  GG
• with int payoffs on a path of length O(n)
• in which in the best NE player 1 plays a
• - approximation algorithms for any e
• (E., Goldberg, Goldberg 07)

37
Approximate NE

• e-Nash equilibrium a strategy profile such that
  noone can gain gt e by deviating
• Graphical games on trees poly-time
  algorithms for any e (KLS01)
• 2-player games ( utilities in 0, 1 )
• PPAD-complete for eO(1/n)
• Approximation for constant e
• 0.5 WINE06 (Dec 2006)
• 0.382 ( 1-1/f ) ACM EC07 (June 2007)
• 0.364 WINE07 (Dec 2007)
• 0.339 WINE07 (Dec 2007)

38
Conclusions

• Computational aspects of game-theoretic questions
  are crucial
• Lots of cool open problems
• computing NE in graphical games on trees
• finding e-Nash in 2-player games for small e
• A rich set of techniques
• Talk to me if you want to know more

39
Mixed strategies and payoffs

• Payoff matrices
• the row player plays a (a1, , an)
• the column player plays b (b1, , bn)
• expected payoff of R when playing i (Ri, , b)
• expected payoff of C when playing i (C, j, a)

R11 R12 R1n R21 R22 R2n Rn1
Rn2 Rnn
C11 C12 C1n C21 C22 C2n Cn1
Cn2 Cnn
R
C
40
2 players, k actions
support guessing

• if 1st players strategy a supported on I ? N
  ai ? 0 iff i ? I
• 2nd players strategy b supported on J ?
  N bj ? 0 iff j ? J
• then I ? BR(b) (b, Ri, ) (b, Rk, ) for all
  i? I, k? N
• J ? BR(a) (a, C, j) (a, C, k)
  for all j? J, k? N
• LP on variables a1, , an, b1, , bn
• solutions to LP ? Nash equilibria
• running time 22kpoly(k)

linear inequalities!
41
Reminder 2-player 2-action games
Row player
Column player
BR(C)
Suppose R plays 1 w.p. r EP(C) from playing 0
(1-r)1 EP(C) from playing 1 r3 1-r gt 3r
iff r lt ¼
Suppose C plays 1 w.p. c EP(R) from playing 0
(1-c)2 EP(R) from playing 1 c1 (1-c)2 gt c
iff c lt 2/3
c
1
r
1
mixed NE r1/4, c2/3
42
Computing PBR on a path

• f(u) (aub)/(cud)
• a, b, c, d are determined by Vs payoffs

v
u
1
1
.5
PBRU(v)
PBRV(w)
.5
1
w
v
1
.1
.9