Settling Complexity of NASH equilibrium

1

• Settling Complexity of NASH equilibrium
• Joint work with CHEN Xi
• and
• Extensions with CHEN Xi and Shanghua TENG

2
Importance of complexity issue of NASH

• Papadimitriou STOC 2001 invited talk
• Fortnow Computational Complexity Column
• We know only a few natural problems in NP that
  are not known to be NP-complete or in P. The two
  most often named are factoring and graph
  isomorphism. Another that has come to forefront
  is Nash Equilibrium.
• http//weblog.fortnow.com/2005/02/complexity-of-na
  sh-equilibrium.html

3

• Outline of Story
• Nash Equilibrium (Mathematicians)
• Algorithmic Solutions (OR and Economists)
• Complexity (Computer Scientists)

4

• Nash Equilibrium

5
Zero Sum Game

• N players
• Each chooses one strategy from a set of
  strategies (which is in general different for
  different players)
• Once each player fixes its strategy, the game has
  an outcome for each player (which again is in
  general different for different players) which
  sums up to zero.

6
Notations

• N players i1,2,,n
• Let Si be the set of strategies of player i
• Let si be the strategy chosen by player i
• Let ui(s1, s2,, sn) be the payoff to player i
• Zero sum game is one such that it always hold
  that
• Sumui(s1, s2,, sn)i1,2,,n0

7
An Example Matching Pennies

• Input
• Two players Evan and Odette
• The protocol
• Each flips a coin
• The winner will be decided by the outcome of the
  flipped coins 0 for the number and 1 for the
  picture.
• If coins sum to odd, Oddete wins
• Otherwise, Evan wins.
• Loser pays winner M coins

8
Evans Payoff Matrix
9
Odettes Payoff Matrix
10
Joint Payoff Matrixnumbers in each cell sum to
zero
11
von Neuman min-max Theorem

• Pure strategy game may not always have an
  equilibrium
• Any two player zero sum game has a mixed strategy
  equilibrium.
• John von Neumann (1928), zur Theorie der
  Gesellshaftsspiele, Mathematische Annalen 100,
  pp.295-320.

12
Pure vs Mixed Strategies

• Pure strategy Each player fixes its decision
  on its decided strategy such as always choosing
  the picture for the outcome of the coin.
• Mixed strategy equilibrium Each player chooses
  each side of the coin with a certain probability.
• Flipping it is usually regard as assign a
  probability of ½ for each side of the coin.

13
pure strategy equilibirum
When both choose betrayal, none of them can do
better by shifting away from its strategy of
betrayal. Therefore, it is an equilibrium point.
14
Non-existence of pure strategy equilibrium
At any pair of strategies of the two Players, one
of them can do better By shifting away from its
strategy.
15
Mixed strategy equilibrium
Each chooses number or picture with probability ½.
16
Evans payoff when Both choose ½ and ½.
When Odette chooses half and half, the payoff to
Evan is always zero. Therefore, no matter what
other strategy Evan shifts into, it cannot do
better than the strategy of ½ and ½. The same
argument also hold for Odette.
17
Proof of von Neuman min-max Theorem via Linear
Programming

• Any two player zero sum game has a mixed strategy
  equilibrium.
• Let the payoff matrix of row player be A, an m by
  n matrix.
• The payoff matrix of the column player is A.
• Row player chooses its pure strategies with
  probability x1, x2, , xn
• Column player chooses its pure strategies with
  probability y1, y2, , ym

18
Von Neumanns Maximin Approach

• The row player wants to choose a mixed strategy
  to guarantee a payoff such that no matter
  whatever pure strategy the column player choose
• The column player wants to do the same thing.
• Luckily, those two strategies can be achieved
  simultaneously, by linear program duality.

19
Non-zero-sum Game
20
Another Game

• Cournot game
• Between two firms.
• They produce the same good
• The price of good is a decreasing function of the
  total quantity of goods they jointly produce
• The function is known to both firms.
• Strategy The quantity one firm chooses to
  produce.
• Example 200 pieces will have price 5, 300 piece
  price 3, 400 piece price 1

21
Joint Payoff Matrix
22
Reduction to zero sum game

• Reduce to three player zero sum game
• Introduce a third person
• With only one strategy
• The payoff is the negation of the sum of the
  other two players.
• Similar argument holds for games of more players.

23
Solutions?

• We can still define a maximin solution for every
  player.
• However, we dont have a minmax theorem as in the
  two player case.

24
von Neumanns Proposal

• Study co-operative game behavior for multiple
  players games.
• The von Neumann-Morgenstern solution

25
Nashs approach of Equilibrium

• Nash proved that an equilibrium point exists for
  any number of players in a non-cooperative
  setting.
• Nash, J. F. "Non-Cooperative Games." Ann. Math.
  54, 286-295, 1951.
• His trick was the use of best-response functions
• a recent theorem that had just emerged -
  Kakutani's fixed point-theorem.
• Won nobel prize in 1994

26
An intuition for a proof

• x is the best response to y
• Payoff1(x,y) maximizes Payoff1(x,y)
• y is the best response to x
• Payoff2(x,y) maximizes Payoff2(x,y)
• Best response function
• BR(x,y) (x', y') (maps unit square to itself)
• Its fixed point
• BR(x,y) (x, y)
• is NASH equilibrium by definition.

27
Previous Work of Nash

• As an undergraduate, he had inadvertently (and
  independently) proved Brouwer's fixed point
  theorem.
• Later on, he went on to break one of Riemann's
  most perplexing mathematical conundrums.
• Source
• http//cepa.newschool.edu/het/profiles/nash.htm

28
Extensions

• General equilibrium in Economics proven by
  Arrow-Debreu using Kakutanis fixed point
  theorem.
• Arrow, K.J., and G. Debreu (1954). Existence of
  an equilibrium for a competitive economy.
  Econometrica 22 (July) 26590.
• Both are Nobel prize winners (1972, 1982).

29
Algorithmic Solutions
30
Relationships between the two models

• Two player zero sum game can be solved by linear
  programming
• The next easiest problem is two player non-zero
  sum game.
• N player non-zero sum game can solve n player
  zero sum game (2 and more)
• N1 player zero-sum game can solve n player
  non-zero sum game (3 and more)

31
Linear Complementary Problem

• uMvb0
• v0
• ltu,vgt 0
• It is complementary since either u_i0 or v_i0
  for each i.

32
Mathematical Programming of Players

• Let A and B be the payoff matrices to the two
  players.
• The players problems are thus
• Max xtAy xte1, x 0, y fixed
• MaxxtBy yte1, y0, x fixed

33
Dual LPs

• The dual of Max xtAy xe1, x 0, y fixed
• Min z etz Ay, z 0
• In LCP form xt(ez-Ay)0
• MaxxtBy ety 1, y0, x fixed
• Min w we xtB, w 0
• In LCP form (we-xtB)y0

34
Rescale the equations

• Divide by z on each side and set ylt-y/z
• Qy et Ay, y0
• xt(e-Ay)0
• Divide by w on each side and set xlt-x/w
• P x e xtB, x 0
• (e-xtB)y0

35
Nash Equilibrium as LCP

• v(xt,yt)t
• uMvb 0
• be
• The matrix M is gt
• The equations are
• vt (Mve)0
• v 0

36
LCP formulation of NASH

• Formulation in LCP
• Cottle, RW and Dantzig, GD (1962-3),
  Complementary Pivot Theory of MathematicalProgram
  ming, Linear Algebra and its Applications 1,
  103-125, 1968.
• Algorithmic Solution
• C.E. Lemke and J.T. Howson, Jr., Equilibrium
  Points in Bimatrix Games, J. SIAM 12, pp.418-423,
  1964
• Review Article
• http//www.informs.org/History/dantzig/LinearProgr
  amming_article.pdf

37
More on Lemke and Howson

• The underlying logic, involving motions on the
  edges of an appropriate polyhedron, has been the
  starting point of the path-following methodology
• More details on significance and relevance, see
• 1978 John von Neumann Theory Prize Winning
  Citations for John F. Nash and Carlton E. Lemke
• http//www.informs.org/Prizes/vonNeumannDetails.ht
  ml

38
Main Idea of Lemke and Howson algorithm

• Start with v0
• Choose an index k to be dropped
• That is, vk0 no longer holds
• As the complementary condition
• vt (Mve)0
• Requires we set Mkv10
• vk increases
• until Miv10 for some i
• If ik, we have a non-trivial solution, i.e., a
  Nash equilibrium.
• Otherwise, vi can be increased afterward.
• Change both vk and vi simultaneously such that
  Miv10
• while keep other vs zero.

39
Main Idea of Lemke and Howson algorithm

• NOTE move along edges of polytope with (n-1)
  constraints.
• There is only one choice in the non-degenerate
  case.

40
Extensions

• Scarf developed a path-following approach to
  solve the fixed point problem, approximately and
  to solve the general equilibrium problem.
• Equivalently an algorithm for n person games.
• von Neumann Award (1983).

41
Lower Bound on Lemke and Howson

• An example is constructed such that an
  exponential number of steps is necessary no
  matter which initial index is chosen
• R. Savani and B. von Stengel (2005),
  Hard-to-Solve Bimatrix Games. Econometrica, to
  appear.
• http//www.maths.lse.ac.uk/Personal/stengel/bvs-pu
  bl.html

42

• Complexity

43
Combinatorial Nature of NASH

• Though the problem is defined on continuous
  variables, its proof of existence depended on
• Fixed point theorem
• Dependent on Sperners lemma
• Path following algorithm
• Relied on Combinatorial structures of polytopes.

44
The complexity issue

• The complexity of Nash Equilibrium is therefore
  well defined in the standard computational
  complexity theory.
• However, it has been an open problem whether
  there is a polynomial time algorithm since the
  early work of Lemke-Howson, of forty years ago.

45
PPAD

• Polynomial Parity Argument, Directed Version
• Characterize the proof technique employed in the
  proof of many mathematical problems
• Papadimitriou. On the complexity of the parity
  argument and other inefficient proofs of
  existence. JCSS 48, pp.498-532, 1994.

46
Define PPAD end of line

• A graph of exponential size
• 2n nodes
• Each node has at most one outgoing edge and at
  most one incoming edge
• A polynomial time Turing machine computing the
  successor of every node.
• Node 1 has no incoming edge and has one outgoing
  edge.
• Output Requirement find another node with
  exactly one edge (incoming or outgoing).

47
Problems in PPAD

• Sperner Lemma, Fixed Point, NASH
• In addition, Sperner Lemma and Fixed Point are
  PPAD-Complete
• That means, if they can be solved in polynomial
  time, any problem in PPAD can be solved in
  polynomial time.

48
NASH remains open

• NASH is in PPAD by Lemke and Howson algorithm
• It is not known to be PPAD complete

49
A Big Breakthrough

• The complexity of computing a Nash equilibrium   
  
• Constantinos Daskalakis, Paul W. Goldberg,
  Christos H. Papadimitriou
• which proves 4 player NASH is PPAD-Complete.
• October 10, 2005, in ECCC
• http//www.eccc.uni-trier.de/eccc-reports/2005/TR0
  5-115/index.html
• Conjecture
• 3 player game is hard
• 2 player game is polynomial solvable

50
The Next Step

• 3-NASH is PPAD-Complete
• Xi Chen and Xiaotie Deng
• ECCC, Nov 18, 2005
•   
• Three-Player games are hard
• Constantinos Daskalakis, Christos H.
  Papadimitriou
• Nov 29, 2005, in ECCC

51
Importance of Complexity of 2NASH

• At the frontier of EASY problem Two player zero
  sum game is easy (LP), 2NASH is the next easiest
  problem.
• At the frontier of HARD problem Lemke-Howson
  algorithm, started as a solution for 2NASH, has
  been the original of many related problems that
  employed the path following methodology. Most of
  the important ones are PPAD-hard, among them are
• Fixed point problem
• Many person NASH Equilibrium
• Sperner problem

52
The result

• Xi Chen and Xiaotie Deng
• Settling the Complexity of Two Player Nash
  Equilibrium
• Dec 4, 2005 in ECCC

53
The main structure of proof

• Reduce end of line to
• 3D SPERNER, which is reduced to
• 3D BROUWER FIXED POINT, discrete version
• R player NASH Equilibrium
• Degree 3 graphic NASH equilibrium, approximate
  version
• 4 player NASH, approximate version

54
Our simplified structure of proof

• Reduce end of line to
• 2D SPERNER, which is reduced to
• 2D BROUWER FIXED POINT, discrete version
• 2 player NASH, exact version

55
2D discrete fixed point problem
56
Problems in PPAD

• Sperner Lemma, Fixed Point, NASH
• In addition, 3D Sperner Lemma and 3D Fixed Point
  were PPAD-Complete
• That means, if they can be solved in polynomial
  time, any problem in PPAD can be solved in
  polynomial time.

57
SPERNER Lemma

• Given a triangulated triangle,
• Given a proper labeling of its nodes,
• Labels 0,1,2
• A condition on labels on the boundaries.
• There must be a triangle with all three labels
  appear in its three nodes.

58
Triangulation of a triangle
Ai is colored with i Any point on the boundary
is labeled with labels on the two
endpoints. Labels on the interior points Are not
restricted. SPERNER LEMMA There is a triangle
with all Three labels.
59
Other reasons why 2NASH could not be hard?

• 2D Sperner was open
• 2D fixed point was open

60
The results

• Xi Chen and Xiaotie Deng
• On Complexity of 2D fixed point
• ICALP 2006

61
Relationship of 2D Sperner with 2NASH

• Affirms the possibility that 2NASH could be
  harder
• Simplifies the proof structure

62
Differences from NP-hard Proofs

• Problems of exponential size search space
• Local properties of the input/output structure
• Which can be verified by a polynomial time
  algorithm also as an input parameter.
• Reductions should reduce the polynomial
  algorithms for the local properties to each other

63
Define PPAD Another End

• A graph of exponential size
• 2n nodes
• Each node has at most one outgoing edge and at
  most one incoming edge
• A polynomial time Turing machine computing the
  successor of every node.
• Node 1 has no incoming edge and has one outgoing
  edge.
• Output Requirement find another node with
  exactly one edge (incoming or outgoing).

64
2D Sperner

• A triangle of exponential size
• 2n nodes in the base and in the height
• 22n triangles
• The nodes are properly labeled
• A polynomial time Turing machine computing the
  label of each node
• Output Requirement find a small triangle with
  all three labels.

65
Outline of Proof

• Planar embedded version of Another End
• Even though Another End is planar, its direct
  embedding is in general not planar and an
  efficient embedding is not known.
• A carefully designed labeling processing to
  construct an instance of the Sperners problem.

66
The structure for a planar embedding

• Denote N2n
• Embed a complete graph to a planar grid of size
  3NN, 6N with about NN/2 crossings
• This can be done by a polynomial time algorithm
  locally, i.e., report the edges enters and leaves
  any node in the grid.
• Use the edges in the planar embedding of the
  complete graphs for the embedding of Another
  End
• Difficulty make sure each node has at most one
  incoming and at most one outgoing edge.

67
The complete graph embedding
68
Four types of gadgets at crossing
69
The line graph embedding

• Denote N2n
• Use the edges in the planar embedding of the
  complete graphs for the embedding of Another
  End
• Trick turn left or right on crossings to keep
  the property of single incoming edge and single
  outgoing edge.
• It does not preserve the graph, but preserves all
  the ends of lines.

70
Detours at Crossings
71
Properties

• Properties of the reduction
• The new graph is larger than the original by an
  exponential factor.
• At each node of the grid, it can be calculated in
  polynomial time
• Whether an embedded line goes through it, and in
  which direction.
• Whether two embedded lines are at it or close
  enough to change its incoming/outgoing edges.
• The set of the ends of the lines is exactly the
  same as before the reduction

72
Embedding of a complete graph of three nodes
73
Embedding of a Subgraph (02,21)
74
Theorem

• Another End of Lines embedded on 2D is
  PPAD-complete
• Xi Chen, Xiaotie Deng, On the Complexity of 2D
  Discrete Fixed Point Problem, ICALP 2006.

75
Outline of 2D SPERNER

• Start with Another End of Lines embedded on 2D
• Encode it so that
• We can triangulate a triangle and properly label
  it.
• Do it locally according to the coordinates.
• A completely labeled triangle correspond to an
  end except the start.
• 0 on path, 1 on left, 2 on right and everywhere
• The end of path will have a triangle of three
  labels.

76
A direct line/cycle graph
77
Corresponding triangulation
78
2D Brower Fixed Point

• Input Exponential size grid
• A function of three values (0,1), (1,0),
  (-1,-1).
• Output A unit square on vertices of which the
  function values have all three possibilities

79
2D Brower Fixed Point simplex version

• Input Exponential size grid
• A function of three values (0,1), (1,0),
  (-1,-1).
• Output A triangle on a unit square on vertices
  of which the function values have all three
  possibilities

80
Reduction to Bimatrix Game
81
Special Features as a Natural Problem

• Input two m by n matrices A and B
• Output two vectors x and y as Nash equilibrium
• The input size is no longer exponential
• The input data are explicitly given
• The solution is exact

82
Reduction from 2D Fixed Point

• Code logic operations (and arithmetic operations)
  by Games of Two Players
• Use them to encode polynomial algorithms to
  calculate the function values from input variable
  values
• Make the output function value equal to the
  unspecified input value (fixed point)
• Nash equilibrium obtains the result which derive
  the fixed point.

83
A two player game for addition

• Player one
• Input Nodes a, b and
• Output node c
• Player two intermediate node d
• Pure strategies s1lt(a, 1), (d, 1)gt s2lt(b, 1),
  (d, 1)gt s3lt(c, 1), (d, 0)gts4lt(c, 1), (d,
  1)gts5lt(c, 0), (d, 0)gt
• Payoffs
• For Player one one for s4, s5 zero otherwise
• For Player two one for s1, s2, s3 zero
  otherwise.

84
Proof addition gate

• Pure strategies s1lt(a, 1), (d, 1)gt s2lt(b, 1),
  (d, 1)gt s3lt(c, 1), (d, 0)gts4lt(c, 1), (d,
  1)gts5lt(c, 0), (d, 0)gt
• Payoffs
• For Player one one for s4, s5 zero otherwise
• For Player two one for s1, s2, s3 zero
  otherwise.
• Equilibrium probability
• Player one x on a, y on b, z on c
• Player two w on d
• At equilibrium player two has the same utility
  choosing zero or one
• Player two xyz
• Player one w(1-w)

85
Other operations

• Can be implemented by a two player game in a
  similar manner.
• Leave them as homework for students.

86
Main difficulties

• Comparison operator (lt or gt) cannot be
  implemented by Nash equilibrium exactly nor
  approximately.
• Each operation can be done by two players but it
  is hard to combine all the operations to a two
  player game.
• The dependence of an operation on its
  input/output must not interference with other
  operations
• This part is the trickiest part of the Chen/Deng
  paper.

87
Summary Discussion
88
Mile Stones

• Existence
• Von Neuman zero sum game 1928
• Nash non-zero sum game 1951
• Algorithms
• Cottle and Dantzig, LCP Model, 1962-3
• Lemke and Howson, path-following, 1964
• R. Savani and B. von Stengel, Exponential lower
  bound of LH algorithm, 2004
• Complexity
• Papadimitriou, NASH is in PPAD,1991
• Daskalakis, Goldberg, Papadimitriou, 4-NASH is
  PPAD-complete, Oct 10, 2005
• ChenDeng, DaskalakisPapadimitriou, 3-NASH is
  PPAD-complete Nov18, 2005, Nov29 2005
• Chen and Deng, 2 player Nash Equilibrium is
  PPAD-complete Dec 04, 2005

89
The end brings us back to the beginning

• Lemkes algorithm solves the two player nash
  equilibrium developed a new combinatorial
  algorithmic paradigm in computation of a large
  body of problems in continuous variables.
• In comparison, Dantzigs simplex algorithm.
• Problems include nash, fixed points, general
  equilibrium
• In the end, the solution structure of Lemke to
  bimatrix game that started the history is proven
  to be reduced to any solution to the bimatrix
  game.
• But still Lemke-Howsons algorithm is considered
  the most practical one (with a proven convergence
  theorem) for Nash.

90
A New Start

• Chen Deng Teng
• No FPTAS for two player nash equilibrium
• No polynomial time smoothed algorithm.
• and provides a first nontrivial lower bound in
  smoothed analysis, for a central problem in
  computing.

91
Past related work

• Michael D. Hirsch, Christos H. Papadimitriou,
  Stephen A. Vavasis
• Exponential lower bounds for finding Brouwer fix
  points. J. Complexity 5(4) 379-416 (1989)
• C. Papadimitriou
• The Complexity of the Parity Argument and Other
  Inefficient proofs of Existence (1991)

92
Past related work

• Xi Chen, Xiaotie Deng
• On algorithms for discrete and approximate
  brouwer fixed points. STOC 2005 323-330
• On the Complexity of 2D Discrete Fixed Point
  Problem, ICALP 2006.

93
Concepts of Discrete Fixed Point

• IIMura (2002) Fixed point on lattice point for
  direction preserving functions.
• Chen and Deng (STOC 2005) Fixed point algorithms
  on lattice for functions with 2n values in n
  coordinates (, -) in each coordinate,
  introducing the concept of bad cubes.
• Chen and Deng (COCOON 2006) bad simplex.
• Daskalakis, Goldberg, Papadimitriou (STOC 2006)
  A new discrete definition of the fixed point set
  (on 238 points in 3D, closely related to bad
  cubes), and consider functions with n1 values,
  one is each coordinate and one diagonal ray, with
  no other restrictions.
• Chen/Deng/Teng (ECCC 2006) A new definition of
  the discrete fixed point set (dependent on
  function values on n1 points in an n-dimensional
  space, related to bad simplex).

94
Related Progress

• IIMura (2002) A fixed point theorem on lattice
  points for direction preserving functions.
• Chen and Deng (STOC 2005) Fixed point algorithms
  with oracle model, closing the gap by improving
  both upper and lower bound.
• Daskalakis, Goldberg, Papadimitriou (STOC 2006)
  4NASH is PPAD complete
• Chen and Deng (ECCC2005) PPAD-complete result for
  bimatrix NASH
• Chen and Deng (ICALP 2006) Hardness of 2D
  SPERNER, 2D fixed points.
• Chen/Deng/Teng (ECCC2006) A high dimension fixed
  point hardness result, for a high dimension cube
  with a constant side length, and NFPTS results
  for NASH, and no polynomial time smoothed
  algorithm for NASH.

95
Open problems

• Could PPAD still in P?
• Or it is NP-hard
• How would we place it with other complexity
  classes?

96
Open problems

• Several other related problems discussed in PPAD
  class by Papadimitriou. Could we improve those
  results?

97
Open problems

• Oracle results
• Fixed point problem
• Sperners Lemma
• What about other problems?
• Is there an oracle version of NASH?

98
Open problems

• The negative result for the smoothed analysis
  derived later with Chen and Teng is the first
  lower bound for a non-trivial problem. Could the
  idea be of use for other problems?