Inefficiency of Equilibria: POA, POS, SPOA

1
Inefficiency of Equilibria POA, POS, SPOA
2
Inefficiency of equilibria

• Outcome of rational behavior might be inefficient
• How to measure inefficiency?
• E.g., prisoners dilemma
• Define an objective function
• Social welfare ( sum of players payoffs)
  utilitarian
• Maximize mini ui (egalitarian)

0,5 3,3
1,1 5,0
3
Inefficiency of equilibria

• To measure inefficiency we need to specify
• Objective function
• Definition of approximately optimal
• Definition of an equilibrium
• If multiple equilibria exist, which one do we
  consider?

4
Equilibrium Concepts
no regret ???
Strong Nash
correlated eq ???
mixed Nash
pure Nash
best- response dynamics
5
Common measures
Pure
/Mixed
/Strong
Pure
/Mixed
/Strong

• Price of anarchy (POA)cost of worst NE / cost of
  OPT
• Price of stability (POS)cost of best NE / cost
  of OPT
• Approximation ratio Measures price of limited
  computational resources
• Competitive ratio Measures price of not knowing
  future
• Price of anarchy Measures price of lack of
  coordination

Pure
Pure
/Mixed
/Mixed
/Strong
/Strong
Corr POA Pure POA Mixed POA Strong POA
Corr POS Pure POS Mixed POA Strong POA
6
Price of anarchy
Prisoners dilemma
0,5 3,3
1,1 5,0

• Example in prisoners dilemma, POA POA 3
• But can be as large as desired
• Wish to find games in which POS or POA are
  bounded
• NE approximates OPT
• Might explains Internet efficiency.
• Suppose we define POA and POS w.r.t. NE in pure
  strategies
• we first need to prove existence of pure NE

7
Max-cut game

• Given undirected graph G (V,E)
• players are nodes v in V
• An edge (u,v) means u hates v (and vice versa)
• Strategy of node i si ?Black,White
• Utility of node i neighbors of different color
• Lemma for every graph G, corresponding game has
  a pure NE

8
NE 1

• Claim (global) max-cut defines a NE
• Proof
• Define strategies of players by cut (i.e., one
  side is Black, other side is White)
• Suppose a player i wishes to switch strategies
  is benefit from switching improvement in
  value of the cut
• Contradicting optimality of cut

ui1
ui2
9
NE 2

• Algorithm greedy-find-cut (GFC)
• Start with arbitrary partition of nodes into two
  sets
• For a node with more neighbors on other side,
  switch sides (repeat until no such node exists)
• Claim 1 GFC provides a 2-approx. to max-cut, and
  runs in polynomial time
• Proof
• Poly time GFC terminates within at most E
  steps (since every step improves the value of the
  solution by at least 1, and E is a trivial
  upper bound to solution)
• 2-approx. Each node ends up with more ()
  neighbors on other side than in own side, so at
  least E/2 edges are in cut (since edges in
  cut gt edges not in cut)

10
NE 2 (contd)

• Claim 2 cut obtained by GFC defines a NE
• Proof a player stops only if her strategy is the
  best response to the other players strategies
• Conclusions
• the max-cut game has a NE in pure strategies
• Price of anarchy Nash/OPT ratio 2

Settling the complexity of local max-cut (almost)
completely Robert Elsaesser, Tobias
Tscheuschner Research Project??
11
Wardrop Equilibria Traffic Flow the
Mathematical Model

• a directed graph G (V,E)
• k source-destination pairs (s1 ,t1), , (sk ,tk)
• a rate (amount) ri of traffic from si to ti
• for each edge e, a cost function ce()

Example (k,r1)
c(x)x
Flow ½
s1
t1
c(x)1
Flow ½
12
Routings of Traffic

• Traffic and Flows
• fP amount of traffic routed on si-ti path P
• flow vector f routing of traffic
• Selfish routing what are the equilibria?

13
Wardrop Flows

• Special case, assumptions
• agents small relative to network (nonatomic game)
• want to minimize cost of their path
• Def A flow is at Pure Nash equilibrium (or is
  a Nash flow) if all flow is routed on min-cost
  paths given current edge congestion

Example
Flow 1
Flow .5
x
x
s
t
s
t
1
1
Flow .5
Flow 0
14
History Generalizations

• model, defn of Nash flows by Wardrop 52
• Nash flows exist, are (essentially) unique
• due to Beckmann et al. 56
• general nonatomic games Schmeidler 73
• congestion game (payoffs fn of of players)
• defined for atomic games by Rosenthal 73
• previous focus Nash eq in pure strategies exist
• potential game (equilibria as optima)
• defined by Monderer/Shapley 96

15
The Cost of a Flow

• Def the cost C(f) of flow f sum of all costs
  incurred by traffic (avg cost traffic rate)

x
½
s
t
½
1
Cost ½½ ½1 ¾
16
The Cost of a Flow

• Def the cost C(f) of flow f sum of all costs
  incurred by traffic (avg cost traffic rate)
• Formally if cP(f) sum of costs of edges of P
  (w.r.t. the flow f), then
• C(f) ?P fP cP(f)

x
½
s
t
½
1
Cost ½½ ½1 ¾
17
Inefficiency of Nash Flows

• Note Nash flows do not minimize the cost
• observed informally by Pigou 1920
• Cost of Nash flow 11 01 1
• Cost of optimal (min-cost) flow ½½ ½1 ¾
• Price of anarchy Nash/OPT ratio 4/3

x
1
½
s
t
1
0
½
18
Braesss Paradox

• Initial Network

cost 1.5
19
Braesss Paradox

• Initial Network Augmented Network

½
½
x
1
0
s
t
½
½
x
1
cost 1.5
Now what?
20
Braesss Paradox

• Initial Network Augmented Network

x
1
0
s
t
x
1
cost 1.5
cost 2
21
Braesss Paradox

• Initial Network Augmented Network
• All traffic incurs more cost! Braess 68
• see also Cohen/Horowitz 91, Roughgarden 01

x
1
0
s
t
x
1
cost 1.5
cost 2
22
Special Case of routing Equal Machine Load
Balancing Parallel Links

• Two nodes
• m parallel (related) links
• n jobs (communication requests)
• User cost (delay) is proportional to link load
• Global cost (maximum delay) is the maximum link
  load

23
Price of Anarchy

• Price of Anarchy
• The worst possible ratio between
• Objective function in Nash Equilibrium and
• Optimal Objective function
• Objective function total user cost, total user
  utility, maximal/minimal cost, utility, etc.,
  etc.

24
Identical machines

• Main results (objective function maximum load)
• For m identical links, identical jobs (pure) R1
• For m identical links (pure) R2-1/(m1)
• For m identical links (mixed)

Lower bound easy uniformly choose machine
with prob. 1/m Upper bound assume opt 1, opt
max expected 2 in NE (otherwise
not NE, NE expected max log m /
loglog m due to Hoeffding
concentration inequality
25
Identical machines
1
2
2
2
1.5
Highest load machine (1), lowest weight job on
1 (1)
Lowest weight job on highest load machine ½ HL
(3)
Every other machine has load ½ HL
26
Related Work (Cont)

• Main results
• For 2 related links R1.618
• For m related links (pure)
• For m related links (mixed)
• For m links restricted assignment (pure)
• For m links restricted assignment (mixed)

27
Related machines

• m (3) machines
• n (4) jobs
• vi speed of machine i
• wj weight of job j

• Li load on machine i

1 (2)
2 (4)
2 (2)
1 (4)
v1 4
v2 2
v3 1
L1 1
L2 3
L3 2
28
Price of Anarchy Lower Bound
G0
G1
G2
Gi
Gk
k
k-i
k-1
k-2
k! / (k-i)!
k!
1
k
k(k-1)
29
Price of Anarchy Lower Bound
G0
k! m k log m / log log m
G1
G2
Gi
Gk
k
k-i
k-1
k-2
k!
1
k
k(k-1)
k! / (k-i)!
30
Its a Nash Equilibrium
G0
G1
G2
2
1
1
Gi
Gk
k
k-i
k-1
k-2
k!
1
k
k(k-1)
k! / (k-i)!
31
Its a Nash Equilibrium
G0
G1
G2
4
2
1
Gi
Gk
k
k-i
k-1
k-2
k!
1
k
k(k-1)
k! / (k-i)!
32
The social optimum
G0
G1
G2
Gi
Gk
k
k-i
k-1
k-2
k! / (k-i)!
k!
1
k
k(k-1)
33
The social optimum
G0
G1
G2
Gi
Gk
k
k-i
k-1
k-2
k! / (k-i)!
k!
1
k
k(k-1)
34
The social optimum
G0
G1
G2
Gi
Gk
k
k-i
k-1
k-2
k! / (k-i)!
k!
1
k
k(k-1)
35
Related Machines Price of Anarchy upper bound

• Normalize so that Opt 1
• Sort machines by speed
• The fastest machine (1) has load Z, no machine
  has load greater than Z1 (otherwise some job
  would jump to machine 1)
• We want to give an upper bound on Z

36
Related Machines Price of Anarchy upper bound

• Normalize so that Opt 1
• The fastest machine (1) has load Z, but Opt is
  1, consider all the machines that Opt uses to run
  these jobs.
• These machines must have load Z-1 (otherwise
  job would jump from 1 to this machine)
• There must be at least Z such machines, as they
  need to do work Z

37
Related Machines Price of Anarchy upper bound

• Take the set of all machines up to the last
  machine that opt uses to service the jobs on
  machine 1.
• The jobs on this set of machines have to use
  Z(Z-1) other machines under opt.
• Continue, the bottom line is that n Z!, or that
  Z log m / log log m

38
Restricted Assignment to Machines
l3
OPT
Group 1
Group 2
Group 3
m0
m0
m0
m0
m0
m1
m0
m1
m1
m1
m1
m1
m2
m2
m2
m3
Group 3
NASH
Group 2
Group 1
m0
m0
m0
m0
m0
m1
m0
m1
m1
m1
m1
m1
m2
m2
m2
m3
39
Price of anarchy for unrelated machines

• POA for unrelated machines is unbounded

Machine 1
Machine 2
e 1
1 e
Job 1
Job 2
Social optimum
Nash equilibrium
makespane
makespan1
PoA1/e
40
Allowing Coordination in Equilibrium

• Strong Equilibrium Aumann59
• No coalition can deviate and strictly improve the
  utility of all of its members
• very robust concept
• may be a better prediction of rational behavior
• most games do not admit Strong Eq.
• usually applied to pure Eq with pure deviations

41
Example 1 Prisoners Dilemma
cooperate
defect
3,3 0,5
5,0 1,1
Unique Nash Eq.
cooperate
defect
Strong Eq. ?
Prisoners dilemma does not admit any Strong Eq.
42
Strong Price of Anarchy

• Determining SPoA requires two parts
• Proving existence of Strong Eq
• Bounding the worst ratio
• SE ? NE ? SPoA PoA

Strong Price of Anarchy (SPoA)
43
k-Strong Equilibrium
SS1xxSn

• A joint action s?S is not resilient to a pure
  deviation of a coalition G if there is a pure
  action profile g of G such that ci(s-G ,g)ltci(s)
  for any i? G
• e.g., (defect,defect) in Prisoners dilemma
• A pure Nash Eq s?S is resilient to pure deviation
  of coalitions of size k if there is no coalition
  G of size at most k such that s is not resilient
  to a pure deviation by G
• A k-Strong Equilibrium is a pure Nash Eq that is
  resilient to pure deviation of coalitions of size
  at most k

44
Strong Equilibrium Hierarchy
NE
1-SE
2-SE
SE Aumann
n-SE
45
Related Work

• Existence of Strong Equilibrium
• monotone decreasing congestion games
  HolzmanLev-tov 1997, 2003
• monotone increasing congestion games correlated
  SE RosenfeldTennenholtz 2006
• Related solution concepts
• Coalition-proof Eq. Bernheim 1987
• Group-strategyproof mechanisms MoulinShenker
  2001
• Coalitions with transferable utilities
  Hayrapetyan et al 2006

NE
CPE
SE
46
Existence of Strong Equilibrium in load balancing
games

• Is every Nash Eq. on identical machines also a
  Strong Eq ?
• NO ! (for m 3)

Coalition 5,5,3,3
47
Strong Eq. Existence

• Theorem in any load balancing game, the lex.
  minimal joint action s is a k-SE for any k

48
Recall Lexicographic Order

• Definition a vector (l1,,lm) is smaller than
  (l1,,lm) lexicographically if for some i, li lt
  li and lk lk for all klti
• Definition A joint action s is smaller than s
  lex. (s?s) if the vector of machine loads L(s),
  sorted in non-decreasing order, is smaller lex.
  than L(s)

s
s?s
s
49
Proof of SE existence

• Lemma suppose L(s) and L(s) differ only in the
  loads of machines in a set M? M. if for each Mi
  ? M, Li(s) lt maxkLk(s) Mk?M, then s?s

50
Proof of SE Existence

• Suppose in contradiction that s (lex. minimal) is
  not a SE, and let G be the smallest coalition
  (deviating to s).
• Claim the same set of machines are chosen by G
  in s and in s (denote it M(G))
• If a job migrates TO some machine, another
  jobmigrates FROM it
• else contradicting s is NE
• If a job migrates FROM some machine, another
  jobmigrates TO it
• else contradicting minimality of G
• Since all jobs in G must benefit, all loads of
  M(G) in s must be smaller than max load of M(G)
  in s
• Contradicting minimality of s

51
Price of Anarchy (PoA)

• Recall for unrelated machines, PoA may be
  unbounded

Machine 1
Machine 2
e 1
1 e
Job 1
Objective min makespan
Job 2
Social optimum
Nash equilibrium
Nash equilibrium
PoA1/e
Strong equilibrium
Strong equilibrium
SPoA1
52
Strong Price of Anarchy

• Theorem for any job scheduling game with m
  unrelated machines and n jobs, SPoA m

53
Proof for SpoA m

• Claim 1 L1(s) OPT
• else coalition of all jobs to OPT

L1(s)
L1(s)
OPT
OPT
M1
Mm
Mi
Mi-1
M1
Mm
Mi
Mi-1
54
Proof for SpoA m

• Claim 1 L1(s) OPT
• else coalition of all jobs to OPT
• Claim 2 ? i Li(s)-Li-1(s) OPT
• else consider s, where all jobs on machines
  i..m go to OPT. For all J ? G
• cJ(s) gt Li-1(s) OPT
• cJ(s) Li-1(s) OPT (since all J ? G together
  add at most OPT)

Li(s)
gt OPT
OPT
Li-1(s)
L1(s)
M1
Mm
Mi
Mi-1
M1
Mm
Mi
Mi-1
Lm(s) m OPT
55
Lower Bound (m machines)

• Theorem there exists a job scheduling game with
  m unrelated machines for which SPoA m
• Proof

M1 M2 M3 M4 Mm
J1 1 1
J2 1 2
J3 1 3
J4 1 4



Jm 1 m
?
OPT 1
SE
makespanm
?
56
Identical Machines

• Theorem there exists a job scheduling game with
  m identical machines and n jobs, such that

1 2 m-1 m
J1

Jm
Jm1


J2m
OPT
11/m
1
2
SE
1/m
m-1
m
1
m-2
57
Results - machines

• Objective function maximum load
• For m identical links, identical jobs (pure) R1
• For m identical links (pure) POA SPOA
  2-1/(m1),
• For m related links (pure)
• For m links restricted assignment (pure)
• For m unrelated machines,