Title: Applications of Game Theory: Part II
1Applications of Game Theory Part II(b)
John C.S. Lui Computer Science Eng. Dept The
Chinese University of Hong Kong
2First course
On the interaction between Overlay Routing and
Underlay Routing Y. Liu, H. Zhang, W. Gong, D.
Towsley INFOCOM 2005
3Motivation Interactions Between Application
Level Network and Physical Network
- physical network control
- routing, congestion control,
- Result?
- interactions?
- controllers mismatch?
4Outline
- Problem Formulation
- Simulation Study
- Game-theoretic Study
- Conclusions
5Routing in Underlay Network
- Routing on physical network level
- Inter-domain BGP, etc.
- Intra-domain OSPF, MPLS, etc.
- determine routes for all source-destination
traffic demand pairs - minimize network-wide delay, cost, etc.
traffic demand pair A-gtB traffic demand pair
A-gtC traffic demand pair C-gtB
6Routing in Overlay Network
- An overlay network choose routes at application
level to minimize its own delay or cost
Overlay demand A-gtB logical routes A-gtC-gtB and
A-gtB
C
A
B
- Overlay
- gains advantage
- better path delay, loss, throughput, etc
- is selfish
- potential performance degradation to other
non-overlay traffic
C
D
E
A
B
demand pair A-gtC
demand pair C-gtB demand pair A-gtB
7Considering overlay and underlay together ?
- How do they interact with each other?
- How does selfish behavior of overlay routing
- affect overall network performance?
- affect non-overlay traffic performance?
- affect its own performance?
8Interactions Between Overlay Routing and
Underlay Routing
Overlay Routing Optimizer To minimize overlay
cost
Underlay Routing Optimizer To minimize overall
network cost
9Approach by authors
- Focusing interaction in a single AS
- Considering two routing models for overlay and
one routing model for underlay - Simulating the interaction dynamic process
- Studying this process in a Game-theoretic
framework
10Routing Models
- Overlay routing model
- Selfish source routing
- Individual user controls infinitesimal amount of
traffic, to minimize its own delay - Optimal overlay routing
- A central entity minimizes the total delay of all
overlay traffic demands
- Underlay routing model
- Optimal underlay routing
- A central entity minimizes the total delay of all
network traffic, e.g. Traffic Engineering MPLS
11Simulation Study Optimal Overlay and Optimal
Underlay
14 node tier-1 POP network (Medina et.al.
2002) bimodal normal model of traffic demand 3
overlay nodes
Node without overlay
Node with overlay
Link
12Simulation Study ( case 1 8 overlay traffic)
Optimal Overlay and Optimal Underlay
- Iterative process
- Underlay takes turn at step 1, 3, 5,
- Overlay takes turn at step 2, 4, 6,
13Simulation Study (case 2 10 overlay traffic)
Optimal Overlay and Optimal Underlay
- Iterative process
- Underlay takes turn at step 1, 3, 5,
- Overlay takes turn at step 2, 4, 6,
14Game-theoretic Study
- Two-player non-zero sum game
Underlay
overlay
X strategy of overlay traffic allocation on
logical links Y strategy of underlay traffic
allocation on physical links
15Game-theoretic Study
- Best-reply dynamics
- Nash Equilibrium
16Optimal Underlay Routing v.s. Optimal Overlay
Routing
- Overlay
- One central entity calculates routes for all
overlay demands, given current underlay routing - Assumption it knows underlay topology and
background traffic
X(k)
1-X(k)
Denote overlays routing decision with a single
variable X(k) overlays flow on path ACB after
round k
17Best-reply Dynamics
- There exists unique Nash equilibrium x,
- x globally stable x(k) ?x, from any initial
x(1)
When x(1)0, overlay performance improves
18Best-reply Dynamics
- There exists unique Nash equilibrium x,
- x globally stable x(k) ?x, from any initial
x(1)
When x(1)0.5, overlay performance degrades
Overlay Delay Evolution
Overlay Routing Evolution
Underlays turn
delay
Overlays turn
x(k)
BAD INTERACTION!
x(k)gtx(k1)gtx
x
x(k)ltx(k1)ltx
Round k
Round k
19Conclusions Open Issues
- Selfish overlay routing can degrade performance
of network as a whole - Interactions between blind optimizations at two
levels may lead to lose-lose situation - Future work
- larger topology analysis/experimentation
- overlay routing and inter-domain routing
- interactions between multiple overlays ()
- implications on design overlay routing
- regulation between overlay and underlay ()
20Second course
On the Interaction of Multiple Overlay
Routings Performance 2005 Joe W.J. Jiang, D.M.
Chiu, John C.S. Lui
21Questions
- These overlays tend to fully utilize available
resource. - So, is there any anarchy?
- How do overlay networks co-exist with each other?
- What is the implication of interactions?
- How to regulate selfish overlay networks via
mechanism design? - Can ISPs take advantage of this?
22Outline
- Motivation
- Mathematical Modeling
- Overlay Routing Game
- Implications of Interaction
- Pricing
- Conclusion
23Motivation
- Overlays provide a feasibility for users to
control their own routing. - Routing, possible multi-path, becomes an
optimization problem. - Interaction occurs (due to same underlay)
- Interaction between one overlay and underlay
traffic engineering, Zhang et al, Infocom05. - Interaction between co-existing overlays ?
24Performance Characteristics
- Objective minimize end-to-end delay (e.g., RON)
- Delay of a physical link e
- Performance Characteristics (Underlay)
25Performance Characteristics
- Objective minimize end-to-end delay
- Delay of a physical link e
- Performance Characteristics (Underlay)
26Performance Characteristics
- Objective minimize end-to-end delay
- Delay of a physical link e
- Performance Characteristics (Underlay)
27System Objectives
- Network Operators
- Min average delay in the whole underlay network
- Overlay Users
- Min average delay experienced by the overlay
28How do Overlays Interact?
- Overlapping physical links.
- Performance dependent on each other.
- Selfish routing optimization.
- Overlays are transparent to each other.
- Lack of information exchange between overlays.
29Contribution
- What is the form of interaction?
- Is there routing instability (oscillation), or
there is an equilibrium ? - Is the routing equilibrium efficient?
- What is the price of anarchy?
- Fairness issues
- Mechanism design can we lead the selfish
behaviors to an efficient equilibrium?
30Mathematical Modeling
- Overlay routing An optimization problem
- Decision variable routing policy
s overlay f flow r path
31Mathematical Modeling
- Overlay routing An optimization problem
- Objective average weighted delay (matrix form)
32Overlay Routing Optimization
33Algorithmic Solution
- Unique optimizer
- Convex programming
- feasible region convex
- delay function continuous, non-decreasing,
strictly convex - Solution
- Apply any convex programming techniques.
- Marginal cost network flow (probabilistic routing
ICNP04). - This is solved in an independent, and distributed
fashion by each overlay.
But will independent optimization leads to system
instability (route flop)?
34Overlay Routing Game
Strategic Game GoverlayltN, (?s), (s)gt
- Nash Routing Game
- Player -- N
- all overlays
- Strategy -- ?s
- feasible routing policy feasible region of
OVERLAY(s) - Preference relation -- s
- low delay players utility function is -delay(s)
35Illustration of Interaction
Aggregate traffic on physical links
Routing decision on logical paths in overlay 1
Delay of logical paths in overlay 1
Overlay 1
Routing decision on logical paths in overlay 2
Delay of logical paths in overlay 2
Overlay 2
Overlay probing
Aggregate overlay traffic
?
Routing Underlay
Underlay (non-overlay) traffic
Overlay n
Routing decision on logical paths in overlay n
Delay of logical paths in overlay n
36Existence of Nash Equilibrium
- Definition Nash equilibrium point (NE)
A feasible strategy profile y(y(1),, y(s),,
y(n))T is a Nash equilibrium in the overlay
routing game if for every overlay s?N,
delay(s)(y(1),y(s),y(n))
delay(s)(y(1),y(s),y(n))for any other
feasible strategy profile y(s) .
37Existence of Nash Equilibrium
Good News NO ROUTE FLOP !!!
In the overlay routing game, there exists a Nash
equilibrium if the delay function
delay(s)(y(s) y(-s)) is continuous,
non-decreasing and convex.
38Fluid Simulation
39Overlay performance
40Overlay routing decisions
41The Price of Anarchy
Global Performance (average delay for all flows)
Efficiency Loss ?
- GOR Global Optimal Routing
- NOR Nash equilibrium for Overlay Routing Game
- NSR Nash equilibrium for Selfish Routing
42Selfish Routing
- (User) selfish routing a single packets
selfishness - Every single packet chooses to route via a
shortest (delay) path. - A flow is at Nash equilibrium if no packet can
improve its delay by changing its route.
43Selfish Routing
- Also a Nash equilibrium of a mixed strategic game
- Player flow f
- Strategy p ? Pf
- Preference low delay
- System Optimization Problem
44Performance Comparison
Overlay One Overlay Two Average Delay
Centralized Global Optimal Routing 2.50 2.38 2.44
NE of Overlay Optimal Routing 2.46 2.53 2.50
NE of Selfish Routing 2.63 2.75 2.69
45Inspiration
- Is the equilibrium point efficient (at least
Pareto optimal) ? - Fairness issues of resource competition between
overlays.
46Example Network
y1
1-y1
y2
1-y2
47Sub-Optimality
physical link delay function de(le)
1-5 1l
3-4 l
2-6 2.5l
y1
y2
Non Pareto-optimal !
Routing (y1, y2) Average Delay (overlay1, overlay2 )
NE (0.5, 1.0) (1.5, 1.5)
Pareto Curve (0.4, 0.9) (1.4, 1.4)
48Fairness Paradox
physical link delay function de(le)
1-5 al
3-4 bl?
2-6 cl
y1
y2
- a, b, c, ? are non-negative parameters
- Everything is symmetric except two private links
a c
49Fairness Paradox
physical link delay function de(le)
1-5 al
3-4 bl?
2-6 cl
y1
y2
a lt c
Overlay 1 has a better private link !
50Fairness Paradox
y1
y2
Unbounded Unfairness
a lt c ? delay1 lt delay2
51War of Resource Competition
1 unit
1 unit
USA
China
poil(y1y2)
pusa(1-y1)
pchn(1-y2)
pusalt pchn
52War of Resource Competition
1 unit
1 unit
y1
USA
China
poil(y1y2)
Min Costchn(y2 y1) y2poil(y1y2)(1-y2)pchn(
1-y2)
pusa(1-y1)
pchn(1-y2)
pusalt pchn
53War of Resource Competition
1 unit
1 unit
USA
poil(y1y2)
China
pusalt pchn ? Costusa gt Costchn
pusa(1-y1)
pchn(1-y2)
54Pricing (opportunity for ISP)
Mechanism Design
Inefficient Nash equilibrium
Desired equilibrium
payment
new Nash equilibrium
55Pricing I Improve Delay
- Objective to achieve global optimality
- NE of overlay routing game
le(s) traffic of overlay s le(-s) traffic
other than overlay s
56Pricing I Improve Delay
- Objective to achieve global optimality
- New NE of overlay routing game
Heterogeneous pricing
57Pricing I Improve Delay
- New NE of overlay routing game
KKT condition
KKT condition
58Pricing II improve fairness
- Cause of unfairness
- Over-utilize good common resources
- Unfair resource (bandwidth) allocation
- Pricing Scheme
price p
ISP
Overlay
Improve performance Reduce cost
maximize profit
routing decision
59Incentive Resource Allocation
new Nash equilibrium ? le
60Revenue Distribution
61Effectiveness of Pricing
62Conclusion
- Study the interaction between multiple
co-existing overlays. - Non-cooperative Nash routing game.
- Prove the existence of NEP.
- Show the anomalies and implications of the NEP.
- Present two distributed pricing schemes to
address the anomalies.
63Third Course
Interaction of ISPs Distributed Resource
Allocation and Revenue Maximization Sam C.M.
Lee, Joe W.J. Jiang, D.M Chiu, John C.S. Lui
64View of ISPs
65Tier-2 ISP
Local ISP
ISP
Peer
Peer
Peer
Peering link
ISP link
ISP
Peer
Peer
Peer
66Optimization problem of peers
Issues to consider
Tier-2 ISP (ISP)
Peer i
1. performance of the link
2. charge of the link
Peer k
Peer j
67Optimization problem of peers
Happiness obtained from sending traffic to peers
Delay cost in ISP link
Payment to ISP
Delay costs in peering links
Payments to peers
68Constraints of peers
1.
2.
3.
4.
69Solution to the peers
- Objective function is strictly concave in every
transmission rate - The optimal transmission rates and maximum
utility are unique and can be found by the
Lagrangian method.
70Problems for an ISP
71Information exchange framework
Next period
Bandwidth allocation
ISP
Bid
peer
Compute resource distribution
Compute optimal rates
72ISP 1 Resource distribution
ISP
Bandwidth 600MBps
?
?
?
peer1
peer2
peer3
Bid 100MBps
Bid 150MBps
Bid 50MBps
73Proportional share algorithm
ISP
Bandwidth 600MBps
100MBps
200MBps
300MBps
peer1
peer2
peer3
Bid 100MBps
Bid 150MBps
Bid 50MBps
74Equal share algorithm
ISP
Bandwidth 600MBps
150MBps
200MBps
250MBps
peer1
peer2
peer3
Bid 100MBps
Bid 150MBps
Bid 50MBps
75Simulations
- When the happiness coefficients of peers are low
PSA
ESA
76Simulation
- When the happiness coefficients of peers are high
PSA
ESA
77ISP 2 Maximization of Revenue
Unit price
Demand by peer i
Total revenue from the peers
Determine the optimal price
78Solution Maximization of revenue
- Estimate the aggregate traffic ( ) from
all - peers in term of the price (P)
79Conclusions
- Utility maximization of a peer
- Resource distribution of ISP
- Revenue maximization of ISP
80Fourth Course
On the Access Pricing Issues of Wireless Mesh
Networks ICDCS 2006
Ray K. Lam Dah-Ming Chiu John C.S. Lui
81WMN Paints a Bright Future
- Wireless mesh network (WMN)
- Wireless nodes
- Multi-hop routing
- Form a wireless mesh
- More access to the Internet
- More people, rich or poor
- More ubiquitous, anywhere, anytime
- More opportunities to everyone
Internet
Internet
82The Critical ThingCooperation
- Multi-hop routing
- Relay packets for each other
- My concerns bandwidth, CPU time, security
- Community network with symmetric traffic
- Help each other gt mutual benefit
- Access network with asymmetric traffic
- Geographically good VS poor
- Why help the poor?
- Incentive system neededpricing
83When AP Meets a Client
- Simple analysis by Musacchio and Walrand 1
- A game with 2 players
- Access point (AP) provides Internet access
- Client buys the service
- One deal per time slot
AP
Client
p1
accept
slot 1
p2
service duration
accept
slot 2
p3
reject
slot 3
p
AP
Client
84A Beautiful Equilibrium
- AP and client each maximizesher gain
- AP guess the right price
- Client compare the price p withservice utility
U - Web browsing utility function
- A beautiful equilibrium
- AP has the same optimal price in every time slot
- Client connects if her per-slot service utility
is greater than slot price (U gt p) - Encourages flat-rate pricing
85To a Multi-hop Scenario
AP
Client
RS
- Adding a relaying node, or reseller (RS)
- RS tries to mark up APs price to the right
level - AP takes note of RSs action when setting her
price - Equilibrium is still flat-rate pricing
- Multi-hop gt multiple RSs
c1
p1
p
accept
accept
c
slot 1
service duration
c2
p2
accept
accept
slot 2
AP
Client
RS
c3
p3
reject
reject
slot 3
86Drawbacks of the Simple Model
- Assuming unlimited network capacity
- 2-player game represents whole system
- Treat every incoming client the same
- Unlimited admission gt unlimited capacity
- Assuming a tree-like network
- 2-hop / multi-hop linear network extension
- Does not consider multiple paths
- Pricing competition may occur
AP
Client
RS
A tree-like network
A graph-like network
87What If Capacity Limited?
- Cannot admit unlimited clients
- Client demands bandwidth guarantee
- AP admission control
- APs system capacity m
- 2-player game not enough
- AP deals with each client differently
- Client arrival model Poisson process
- Like an M/M/m/m/M queuing system
88Flat-rate Pricing Fails
- Failure scenario
- AP is full m clients admitted
- An admitted client a is paying 5/slot
- A new client b arrives
- AP asks b for 6/slot
- If b accepts
- AP raises price for a to 6/slot, OR
- Simply kicks a out
- Flat-rate pricing is not optimal!
89Everybody Loves Flat Rate
- Unrealistic for variable rate
- More practicalfixed-rate, non-interrupted
service - AP charges a client a fixed rate p over time
- AP cannot disconnect a client unilaterally
- AP can still charge different clients at
different fixed rates - How to set the optimal rate on different
occasions?
90Best Strategy in New Service Model
- AP sets price based on remaining capacity
- Raises price when becoming full
- State price at state k, AP charges next
to-be-admitted client at fixed rate pk - Policy of AP characterized by price vector
- Clients best strategy
- Connect AP if service utility per unit time gt
price per unit time (U gt p)
91System Dynamics
- State transition
- Adding a factor P(U gt pk) to regular arrival rate
in M/M/m/m/M model - Reward structure
- Simplification immediate expected profit when a
client connects
? M P(U gt p0)
? (M-1) P(U gt p1)
? (M-m1) P(U gt pm-1)
0
1
2
m
m-1
?
2?
m?
State transition diagram
92System Dynamics
- State transition
- Adding a factor P(U gt pk) to regular arrival rate
in M/M/m/m/M model - Reward structure
- Simplification immediate expected profit when a
client connects
p0/?
p1/?
pm-1/?
0
1
2
m
m-1
0
0
0
Reward Structure
93Finding Optimal Price Vector
- Classical optimization
- Solution for queuing system gives limiting state
probability for each state k, ?k - Gain of AP is a function of price vector
- Complicated to optimize with classical techniques
- Policy-iteration method in Markovian decision
theory - Reduces computational complexity by iterative
algorithm - Guarantees convergence to the best policy
94Numerical Results
- Capacity m5, population M10, departure rate ?1
- Vary arrival rate ? from 0.2 to 10
- Utility U uniformly distributed on 0,10
- U normally distributed with mean 5, s.d.
1.67 - Price increases number of clients in AP and with
?
95Limited Capacity in Multi-hop Case
- Simplification
- Traffic merges at AP
- AP is the bottleneck
- Only AP controlsadmission
- APs policy specified by a price matrix
- At each state, different prices for requests from
different distances - pki price at state k for a client i-hop away
Internet
bandwidth bottleneck
AP
Client
RS
96System Dynamics
- Removing finite population
- Complicates state information
- Different arrival rates for clients at different
distances
?n P(U gt mn(p0,n))
?n P(U gt mn(pm-1,n))
Client n-hop away arrives
Client 2-hop away arrives
Client 1-hop away arrives
?2 P(U gt m2(p0,2))
?2 P(U gt m2(pm-1,2))
?1 P(U gt m1(p0,1))
?1 P(U gt m1(pm-1,1))
0
1
m-1
m
m?
?
State transition diagram
97System Dynamics
- Removing finite population
- Complicates state information
- Different arrival rates for clients at different
distances
p0,n/?
pm-1,n/?
Client n-hop away arrives
Client 2-hop away arrives
Client 1-hop away arrives
p0,2/?
pm-1,2/?
p0,1/?
pm-1,1/?
0
1
m-1
m
0
0
Reward Structure
98Conclusion
- Contributions
- Show that fixed-rate pricing fails with limited
capacity - Generalize unlimited capacity model into limited
capacity model - Devise optimal pricing for fixed-rate,
non-interrupted service with Markovian decision
theory - References
- 1 J. Musacchio and J. Walrand. WiFi access
point pricing as a dynamic game. IEEE/ACM Trans.
Networking. to appear in.