Algorithmic Mechanism Design

1
FOURTH PART
Algorithmic Issues in Strategic Distributed
Systems
2
Suggested readings

• Algorithmic Game Theory, Edited by Noam Nisan,
  Tim Roughgarden, Eva Tardos, and Vijay V.
  Vazirani, Cambridge University Press.
• Blog by Noam Nisan http//agtb.wordpress.com/

3
Two Research Traditions

• Theory of Algorithms computational issues
• What can be feasibly computed?
• How long does it take to compute a solution?
• Which is the quality of a computed solution?
• Centralized or distributed computational models
• Game Theory interaction between self-interested
  individuals
• What is the outcome of the interaction?
• Which social goals are compatible with
  selfishness?

4
Different Assumptions

• Theory of Algorithms (in distributed systems)
• Processors are obedient, faulty (i.e., crash),
  adversarial (i.e., Byzantine), or they compete
  without being strategic (e.g., concurrent
  systems)
• Large systems, limited computational resources
• Game Theory
• Players are strategic (selfish)
• Small systems, unlimited computational resources

5
The Internet World

• Users often selfish
• Have their own individual goals
• Own network components
• Internet scale
• Massive systems
• Limited communication/computational resources
• ? Both strategic and computational issues!

6
Fundamental question

• How the computational aspects of a strategic
  distributed system should be addressed?

Theory of Algorithms
Game Theory
Algorithmic Game Theory


7
Basics of Game Theory

• A game consists of
• A set of players (or agents)
• A specification of the information available to
  each player
• A set of rules of encounter Who should act when,
  and what are the possible actions (strategies)
• A specification of payoffs for each possible
  outcome (combination of strategies) of the game
• Game Theory attempts to predict the final outcome
  (or solution) of the game by taking into account
  the individual behavior of the players

8
Solution concept

• How do we establish that an outcome is a
  solution? Among the possible outcomes of a game,
  those enjoying the following property play a
  fundamental role
• Equilibrium solution strategy combination in
  which players are not willing to change their
  state. This is quite informal when a player does
  not want to change his state? In the Homo
  Economicus model, this makes sense when he has
  selected a strategy that maximizes his individual
  payoff, knowing that other players are also doing
  the same.

9
Roadmap

• We will focus on two types of equilibria Nash
  Equilibria (NE) and Dominant Strategy Equilibria
  (DSE)
• Computational Aspects of Nash Equilibria
• Does a NE always exist?
• Can a NE be feasibly computed, once it exists?
• What about the quality of a NE?
• Case study Network Flow Game (i.e., selfish
  routing in Internet), Network Connection Games
• (Algorithmic) Mechanism Design
• Which social goals can be (efficiently)
  implemented in a strategic distributed system?
• Strategy-proof mechanisms in DSE VCG-mechanisms
• Case study Mechanism design for the Shortest
  Path Game

10

• FIRST PART
• (Nash)
• Equilibria

11
(Some) Types of games

• Cooperative/Non-cooperative
• Symmetric/Asymmetric (for 2-player games)
• Zero sum/Non-zero sum
• Simultaneous/Sequential
• Perfect information/Imperfect information
• One-shot/Repeated

12
Games in Normal-Form
We start by considering simultaneous,
perfect-information and non-cooperative games.
These games are usually represented explicitly by
listing all possible strategies and corresponding
payoffs of all players (this is the so-called
normalform) more formally, we have

• A set of N rational players
• For each player i, a strategy set Si
• A payoff matrix for each strategy combination
  (s1, s2, , sN), where si?Si, a corresponding
  payoff vector (p1, p2, , pN)
• ? S1?S2? ?SN payoff matrix

13
A famous game the Prisoners Dilemma
Non-cooperative, symmetric, non-zero sum,
simultaneous, perfect information, one-shot,
2-player game
Strategy Set
Prisoner I Prisoner II Prisoner II Prisoner II
Prisoner I Dont Implicate Implicate
Prisoner I Dont Implicate 1, 1 6, 0
Prisoner I Implicate 0, 6 5, 5
Payoffs
Strategy Set
14
Prisoner Is decision

• Prisoner Is decision
• If II chooses Dont Implicate then it is best to
  Implicate
• If II chooses Implicate then it is best to
  Implicate
• It is best to Implicate for I, regardless of what
  II does Dominant Strategy

Prisoner I Prisoner II Prisoner II Prisoner II
Prisoner I Dont Implicate Implicate
Prisoner I Dont Implicate 1, 1 6, 0
Prisoner I Implicate 0, 6 5, 5
15
Prisoner IIs decision

• Prisoner IIs decision
• If I chooses Dont Implicate then it is best to
  Implicate
• If I chooses Implicate then it is best to
  Implicate
• It is best to Implicate for II, regardless of
  what I does Dominant Strategy

Prisoner I Prisoner II Prisoner II Prisoner II
Prisoner I Dont Implicate Implicate
Prisoner I Dont Implicate 1, 1 6, 0
Prisoner I Implicate 0, 6 5, 5
16
Hence
Prisoner I Prisoner II Prisoner II Prisoner II
Prisoner I Dont Implicate Implicate
Prisoner I Dont Implicate 1, 1 6, 0
Prisoner I Implicate 0, 6 5, 5

• It is best for both to implicate regardless of
  what the other one does
• Implicate is a Dominant Strategy for both
• (Implicate, Implicate) becomes the Dominant
  Strategy Equilibrium
• Note If they might collude, then its beneficial
  for both to Not Implicate, but its not an
  equilibrium as both have incentive to deviate

17
Dominant Strategy Equilibrium

• Dominant Strategy Equilibrium is a strategy
  combination s (s1, s2, , sN), such that si
  is a dominant strategy for each i, namely, for
  any possible alternative strategy profile s (s1,
  s2, , si , , sN)
• pi (s1, s2, , si, , sN) pi (s1, s2, , si,
  , sN)
• Dominant Strategy is the best response to any
  strategy of other players
• If a game has a DSE, then players will
  immediately converge to it
• Of course, not all games (only very few in the
  practice!) have a dominant strategy equilibrium

18
A more relaxed solution concept Nash
Equilibrium 1951

• Nash Equilibrium is a strategy combination
• s (s1, s2, , sN) such that for each i, si
  is a best response to (s1, ,si-1,si1,,
  sN), namely, for any possible alternative
  strategy si of player i
• pi (s1, s2, , si, , sN) pi (s1, s2,
  , si, , sN)

19
Nash Equilibrium

• In a NE no agent can unilaterally deviate from
  his strategy given others strategies as fixed
• Each agent has to take into consideration the
  strategies of the other agents
• If the game is played repeatedly and players
  converge to a solution, then it has to be a NE
• But if a game has one or more NE, players need
  not to converge to it
• Dominant Strategy Equilibrium ? Nash Equilibrium
  (but the converse is not true)

20
Nash Equilibrium The Battle of the Sexes
(coordination game)
Man Woman Woman Woman
Man Stadium Cinema
Man Stadium 2, 1 0, 0
Man Cinema 0, 0 1, 2

• (Stadium, Stadium) is a NE Best responses to
  each other
• (Cinema, Cinema) is a NE Best responses to each
  other
• ? but they are not Dominant Strategy Equilibria
  are we really sure they will eventually go out
  together????

21
A crucial issue in game theory the existence of
a NE

• Unfortunately, for pure strategies games (as
  those seen so far, in which each player, for each
  possible situation of the game, selects his
  action deterministically), it is easy to see that
  we cannot have a general result of existence
• In other words, there may be no, one, or many NE,
  depending on the game

22
A conflictual game Head or Tail
Player I Player II Player II Player II
Player I Head Tail
Player I Head 1,-1 -1,1
Player I Tail -1,1 1,-1

• Player I (row) prefers to do what Player II does,
  while Player II prefer to do the opposite of what
  Player I does!
• ? In any configuration, one of the players
  prefers to change his strategy, and so on and so
  forththus, there are no NE!

23
On the existence of a NE

• However, when a player can select his strategy
  randomly by using a probability distribution
  over his set of possible pure strategies (mixed
  strategy), then the following general result
  holds
• Theorem (Nash, 1951) Any game with a finite set
  of players and a finite set of strategies has a
  NE of mixed strategies (i.e., there exists a
  profile of probability distributions for the
  players such that the expected payoff of each
  player cannot be improved by changing
  unilaterally the selected probability
  distribution).
• Head or Tail game if each player sets
  p(Head)p(Tail)1/2, then the expected payoff of
  each player is 0, and this is a NE, since no
  player can improve on this by choosing
  unilaterally a different randomization!

24
Fundamental computational issues concerned with NE

1. Finding a NE in mixed/pure (if any) strategies
2. Establishing the quality of a NE, as compared to
   a cooperative system, namely a system in which
   agents can collaborate (recall the Prisoners
   Dilemma)
3. In a repeated game, establishing whether and in
   how many steps the system will eventually
   converge to a NE (recall the Battle of the Sexes)
4. Verifying a NE, approximating a NE, NE in
   resource (e.g., time, space, message size)
   constrained settings, breaking a NE by colluding,
   etc...

(interested in a PhD?)
25
Finding a NE in mixed strategies

• How do we select the correct probability
  distribution? It looks like a problem in the
  continuous
• but its not, actually! It can be shown that
  such a distribution can be found by selecting for
  each player a best possible subset of pure
  strategies (so-called best support), over which
  the probability distribution can actually be
  found by solving a system of algebraic equations!
• ? In the practice, the problem can be solved by
  a simplex-like technique called the LemkeHowson
  algorithm, which however is exponential in the
  worst case

26
Is finding a NE NP-hard?

• In pure strategies, yes, for many games of
  interest
• What about mixed strategies? W.l.o.g., we
  restrict ourself to 2-player games Then, we
  wonder whether 2-NASH is NP-hard.
• Recall a problem P is NP-hard if one can
  Turing-reduce in polynomial time any NP-complete
  problem P to it (this means, P can be solved in
  polynomial time by an oracle machine with an
  oracle for P)
• Recall also a problem P is in NP (resp., in
  coNP) if all its "yes"-instances (resp.,
  no-instances) can be decided in polynomial time
  by a Non-Deterministic Turing Machine (NDTM).
• But 2-NASH can be solved in polynomial-time by a
  NDTM (by enumerating all the supports) moreover,
  every instance of 2-NASH is a yes-instance
  (since every game has a NE), and so we could
  certificate in polynomial-time on a NDTM both
  yes and no-instances of any NP-complete
  problem
• if 2-NASH is NP-hard then NP coNP (hard to
  believe!)

27
The complexity class PPAD

• Definition (Papadimitriou, 1994) PPAD
  (Polynomial Parity Argument Directed case) is a
  subclass of TFNP (Total Function Nondeterministic
  Polynomial), where existence of a solution is
  guaranteed by a parity argument. Roughly
  speaking, PPAD contains all problems whose
  solution space can be set up as the (non-empty)
  set of all sinks in a suitable directed graph
  (generated by the input instance), having an
  exponential number of vertices in the size of the
  input, though.
• Breakthrough 2-NASH is PPAD-complete!!!
  (Chen Deng, FOCS06)
• Remark It could very well be that PPADP?NP, but
  several PPAD-complete problems are resisting for
  decades to poly-time attacks (e.g., finding
  Brouwer fixed points)

28
Finding a NE in pure strategies

• By definition, it is easy to see that an entry
  (p1,,pN) of the payoff matrix is a NE if and
  only if pi is the maximum ith element of the row
  (p1,,pi-1, p(s)s?Si ,pi1,,pN), for each
  i1,,N.
• Notice that, with N players, an explicit (i.e.,
  in normal-form) representation of the payoff
  functions is exponential in N ? brute-force
  (i.e., enumerative) search for pure NE is then
  exponential in the number of players (even if it
  is still polynomial in the input size, but the
  normal-form representation needs not be a
  minimal-space representation of the input!)
• ? Alternative cheaper methods are sought for
  many games of interest, a NE can be found in
  poly-time w.r.t. to the number of players (e.g.,
  using the powerful potential method)

29
On the quality of a NE

• How inefficient is a NE in comparison to an
  idealized situation in which the players would
  strive to collaborate selflessly with the common
  goal of maximizing the social welfare?
• Recall in the Prisoners Dilemma (PD) game, the
  DSE (and NE) incurs a total of 10 years in jail
  for the players. However, if they would not
  implicate reciprocally, then they would stay a
  total of only 2 years in jail!

30
A worst-case perspective the Price of Anarchy
(PoA)

• Definition (Koutsopias Papadimitriou, 1999)
  Given a game G and a social-choice function C
  which depends on the payoff of all the players,
  let S be the set of all NE. If the payoff
  represents a cost (resp., a utility) for a
  player, let OPT be the outcome of G minimizing
  (resp., maximizing) C. Then, the Price of Anarchy
  (PoA) of G w.r.t. C is
• Example in the PD game, PoAPD(C)10/25

PoAG(C)
31
A case study for the existence and quality of a
NE selfish routing on Internet

• Internet components are made up of heterogeneous
  nodes and links, and the network architecture is
  open-based and dynamic
• Internet users behave selfishly they generate
  traffic, and their only goal is to
  download/upload data as fast as possible!
• But the more a link is used, the more is slower,
  and there is no central authority optimizing
  the data flow
• So, why does Internet eventually work is such a
  jungle???

32
Modelling the flow problem

• Internet can be modelled by using game theory it
  is a (congestion) game in which
• players users
• strategies paths
  over which users can route their traffic
• Non-atomic Selfish Routing
• There is a large number of (selfish) users
• All the traffic of a user is routed over a single
  path simultaneously
• Every user controls an infinitesimal fraction of
  the traffic.

33
Mathematical model (multicommodity flow network)

• A directed graph G (V,E) and a set of N players
• A set of commodities, i.e., sourcesink pairs
  (si,ti), for i1,..,k (each of the N players is
  associated with a commodity)
• An amount (or rate) 0 ri 1 of traffic between
  si and ti for each i1,..,k, with ?i1,,k ri
  1
• A set Pi of paths between si and ti for each
  i1,..,k
• The set of all paths ?Ui1,,k Pi
• A flow vector f specifying a traffic routing
• fP rate of traffic routed on path P (notice that
  0 fP 1 )
• A flow is feasible if for every i1,..,k we have
  ?P?Pi fP ri

34
Mathematical model (2)

• For each e?E, the amount of flow absorbed by e
  w.r.t. f is fe?Pe?P fP
• For each edge e, a real-value latency function
  le(x) of its absorbed flow x (this is a
  monotonically increasing function which expresses
  how e gets congested when a fraction 0x1 of the
  total flow f uses e)
• Cost (or latency) of a path P c(P)?e?P le(fe)
• Cost (or total latency) of a flow f
  (social-choice function) C(f)?P?? fP c(P)
  ?e?E fe le(fe)
• Observation Notice that the game is not given in
  normal form!

35
Flows and NE

• Definition A flow f is a Nash flow if no player
  can improve its cost (i.e., the cost of its used
  path) by changing unilaterally its path.
• QUESTION Given an instance (G,r(r1,,rk),l(le1,
  , lem)) of the non-atomic selfish routing game,
  does it admit a Nash flow? And in the positive
  case, what is the PoA of such Nash flow?

36
Example Pigous game 1920

• Latency depends on the congestion (x is the
  fraction of flow using the edge)

le1(x)x
Total amount of flow 1
s
t
Latency is fixed
le2(x)1

• What is the (only) NE of this game? Trivial all
  the fraction of flow tends to travel on the upper
  edge ? the cost of the flow is C(f) 1le1(1)
  0le2(0) 11 01 1
• What is the PoA of this NE? The optimal solution
  is the minimum of C(x)xx (1-x)1 ? C(x)2x-1
  ? OPT1/2 ? C(OPT)1/21/2(1-1/2)10.75 ?
  PoA(C) 1/0.75 4/3

37
The Braesss paradox

• Does it help adding edges to improve the PoA?
• NO! Lets have a look at the Braess Paradox (1968)

Cost of each path x11/21 1.5
v
1
x
1/2
s
t
1/2
x
1
Cost of the flow 2(1.51/2)1.5 (notice this a
NE and it is also an optimal flow)
w
38
The Braesss paradox (2)
To reduce the cost of the flow, we try to add a
no-latency road between v and w. Intuitively,
this should not worse things!
v
1
x
0
s
t
x
1
w
39
The Braesss paradox (3)
However, each user is tempted to change its route
now, since the route s?v?w?t has less cost
(indeed, x1)
If only a single user changes its route, then its
cost decreases from 1.5 to approximately 1,
i.e. c(s?v?w?t) x0x 0.5 0.5 1
v
x
1
0
s
t
x
1
But the problem is that all the users will decide
to change!
w
40
The Braesss paradox (4)

• So, the cost of the flow f that now entirely uses
  the path s?v?w?t is
• C(f) 1110112gt1.5
• Even worse, this is a NE (the cost of the path
  s?v?w?t is 2, and the cost of the two paths not
  using (v,w) is also 2)!
• The optimal min-cost flow is equal to that we had
  before adding the new road and so, the PoA is

Notice it is 4/3, as in the Pigous example
41
Existence of a Nash flow

• Theorem (Beckmann et al., 1956) If for each edge
  e the function xle(x) is convex (i.e., its
  graphic lies below the line segment joining any
  two points of the graphic) and continuously
  differentiable (i.e., its derivative exists at
  each point in its domain and is continuous), then
  the Nash flow of (G,r,l) exists and is unique,
  and is equal to the optimal min-cost flow of the
  following instance
• (G,r, ?(x)? l(t)dt/x).
• Remark The optimal min-cost flow can be computed
  in polynomial time through convex programming
  methods.

x
0
42
Flows and Price of Anarchy

• Theorem 1 In a network with linear latency
  functions, the cost of a Nash flow is at most 4/3
  times that of the min-cost flow ? every instance
  of the non-atomic selfish routing has PoA 4/3.
• Theorem 2 In a network with degree-p polynomials
  latency functions, the cost of a Nash flow is
  O(p/log p) times that of the min-cost flow.
• (Roughgarden Tardos, JACM02)

43
A bad example for non-linear latencies

• Assume pgtgt1

xp
1-?
1
s
t
1
? close to 0
0
A Nash flow (of cost 1) is arbitrarily more
expensive than the optimal flow (of cost close to
0)
44
Convergence towards a NE(in pure strategies
games)

• Ok, we know that selfish routing is not so bad at
  its NE, but are we really sure this point of
  equilibrium will be eventually reached?
• Convergence Time number of moves made by the
  players to reach a NE from an initial arbitrary
  state
• Question Is the convergence time (polynomially)
  bounded in the number of players?

45
The potential function method

• (Rough) Definition A potential function for a
  game (if any) is a real-valued function, defined
  on the set of possible outcomes of the game, such
  that the equilibria of the game are precisely the
  local optima of the potential function.
• Theorem In any finite game admitting a potential
  function, best response dynamics (i.e., each
  player at each step greedily takes a move which
  maximizes its personal utility) always converge
  to a NE of pure strategies.
• But how many steps are needed to reach a NE? It
  depends on the combinatorial structure of the
  players' strategy space

46
Convergence towards the Nash flow

• ? Positive result The non-atomic selfish routing
  game is a potential game, and moreover, for many
  instances (i.e., for prominent graph topologies
  and/or commodity specifications), the convergence
  time is polynomial.
• ? Negative result However, there exist instances
  of the non-atomic selfish routing game for which
  the convergence time is exponential (under some
  mild assumptions).