Market Based Control of Complex Computational Systems

1
Market Based Control of Complex Computational
Systems

• Nick Jennings
• nrj_at_ecs.soton.ac.uk

2
The Complex Systems Challenge

• Building software that operates effectively in
  environments that
• Have no centralised control
• Are highly interconnected
• Are in constant state of flux
• Are highly unpredictable
• Involve multiple, individually-motivated actors

3
The Complex Systems Landscape
Semantic Web
Web Services
Service description Service discovery Service
composition
Flexible interoperation reasoning in
heterogeneous environments
Grid Computing
Agent Based Computing
Robust, large scale open systems
Autonomy Rich interactions
4
The Computational Model

• Entities offer services in an institutional
  setting
• Entities connect to services
• Service discovery
• Service composition
• Service procurement
• Entities enact services
• Flexible context sensitive service delivery

5
Agents as Service Providers Consumers

• encapsulated computer system, situated in some
  environment, and capable
• of flexible autonomous action in that environment
  in order to
• meet its objectives

6
Agents as Service Providers Consumers

• encapsulated computer system, situated in some
  environment, and capable
• of flexible autonomous action in that environment
  in order to
• meet its objectives

• control over internal state and over own behaviour

7
Agents as Service Providers Consumers

• encapsulated computer system, situated in some
  environment, and capable
• of flexible autonomous action in that environment
  in order to
• meet its objectives

• control over internal state and over own behaviour

• experiences environment through sensors and acts
  through effectors

8
Agents as Service Providers Consumers

• encapsulated computer system, situated in some
  environment, and capable
• of flexible autonomous action in that environment
  in order to
• meet its objectives

• control over internal state and over own behaviour

• experiences environment through sensors and acts
  through effectors

• reactive respond in timely fashion to
  environmental change
• proactive act in anticipation of future goals

9
Negotiation as de facto Form of Interaction

• Fixed price offerings
• Catalogues
• Dynamic pricing
• Negotiations
• Auctions

• Agree appropriate service contracts
• Service composition
• Service procurement

10
Computational Service Economies
(Dash et al., 2003)
Mechanism Design

• permissible participants
• e.g. buyers, sellers third parties
• interaction states
• e.g. accepting bids, auction closed
• events causing state transitions
• e.g. bid, time out, bid accepted
• valid actions
• bid, ask, propose, accept, reject,
• counter-proposal, critique
• reward structures
• who pays who gets paid for what

rules of the game
11
Computational Service Economies
(Dash et al., 2003)
Mechanism Design
Agent Strategies

• shaped by interaction protocol
• decision making employed to achieve trading
  objectives
• from very simple to very complex
• maximise benefit
• to self (self interest) and/or
• to group (social welfare)

• permissible participants
• e.g. buyers, sellers third parties
• interaction states
• e.g. accepting bids, auction closed
• events causing state transitions
• e.g. bid, time out, bid accepted
• valid actions
• bid, ask, propose, accept, reject,
• counter-proposal, critique
• reward structures
• who pays who gets paid for what

rules of the game
how to succeed in the game
12
Computational Service Economies
(Dash et al., 2003)
Mechanism Design
Agent Strategies

• shaped by interaction protocol
• decision making employed to achieve trading
  objectives
• from very simple to very complex
• maximise benefit
• to self (self interest) and/or
• to group (social welfare)

• permissible participants
• e.g. buyers, sellers third parties
• interaction states
• e.g. accepting bids, auction closed
• events causing state transitions
• e.g. bid, time out, bid accepted
• valid actions
• bid, ask, propose, accept, reject,
• counter-proposal, critique
• reward structures
• who pays who gets paid for what

Game theory analyses interactions to determine
likely outcomes and equilibria
rules of the game
how to succeed in the game
13
The Market-Based Control Project

• Market-Based Control (MBC)
• paradigm for controlling computer systems using
  economically-inspired techniques
• Market mechanisms used to
• generate and predict emerging system properties,
• although decisions are made independently by
  local agents that each have their own aims and
  objectives
• a market is a self-organising system, directed by
  mechanism
• The proposition
• MBC is good for effectively controlling and
  managing complex, adaptive, distributed
  computational systems

14
Objectives

• Develop and evaluate core MBC technologies
• Automated mechanism design
• Automate design of market mechanisms to achieve a
  desired set of global goals
• Adapt to a changing environment and changing
  (priority of) objectives
• Predict and automate design of agent strategies
• Apply MBC solutions to design and manage complex,
  distributed computational systems

15
Project Applications

• Utility data centres
• MBC to allocate computational resources achieve
  a robust, scalable service
• Distributed content delivery within p2p networks
• MBC to regulate sharing of content
• Decentralised control and scheduling of multiple
  robots
• MBC to provide incentives for cooperation and to
  achieve global goals

16
Research Highlights

• Competing sellers in online auctions
• Strategies for bidding in multiple auctions
• Empirical game theory to select mechanisms and
  strategies for complex markets
• Adaptive auctions

17
Research Highlights

• Competing sellers in online auctions
• Strategies for bidding in multiple auctions
• Empirical game theory to select mechanisms and
  strategies for complex markets
• Adaptive auctions

18

• Often strong competition among sellers in online
  auctions
• How many eBay auctions yesterday?
• 10
• 100
• 1000

19

• Often strong competition among sellers in online
  auctions
• Sellers choice of mechanism
  auction parameters affect buyers
  choice of seller
• How should bidder choose between
  auctions/sellers?
• How should a seller set its parameters?
• Focus on sellers reserve price sealed-bid
  auctions

20
Model of Competing Sellers

• Set announce Reserve Price

Seller
Seller
Seller
Mediator
Auction
Auction

• Set announce Auction Fees

Auction
Buyers

• Select seller
• Bid in auctions

21
Shill Bidding

• Competing sellers reduces optimal reserve price
  and expected revenue (compared to isolated
  auctions)
• Avoid by shill bidding
• Seller disguised as buyer to bid in own auction.
• Illegal and undesired, but hard to detect
• But mediator can use auction fees to deter it
• Use Evolutionary Simulation to
• Evaluate effectiveness of different types of
  auction fees in deterring shill bidding
• Measure market efficiency

22
Results with Auction Fees

• Fraction of auctions won by shill bids

Allocative efficiency
CP closing price RD difference between reserve
and closing prices
23
Observations

• Competition among sellers affects choice of
  mechanism and auction parameters
• Important to take competition into account when
  designing mechanisms and bidder strategies
• Sellers can decide to shill bid in order to
  improve profits
• Mediator (such as eBay) can deter shill bidding
  and increase efficiency by setting appropriate
  auction fees

24
International Competition

• Made proposal to have new game in the Trading
  Agents Competition Foundation
• TAC Market Design
• Reverse Trading Agents Competition
• Design mechanisms with varying
• Clearing policy
• Information revelation policy
• Auction fees

25
Research Highlights

• Competing sellers in online auctions
• Strategies for bidding in multiple auctions
• Empirical game theory to select mechanisms and
  strategies for complex markets
• Adaptive auctions

26
Bidding in Multiple Auctions

• Different start/finish times
• Simultaneous, sequential, or hybrid
• Heterogeneous
• N single-unit auctions
• 1st/2nd price sealed bid, English or Dutch
• Each can have different number of bidders
• Multiple items
• Find optimal best response

27
Heuristic Strategies

• Setting too complex to analyse theoretically and
  find optimal strategies
• Heuristic strategies
• Choose the thresholds
• Single auction dominant strategy (DOM)
• Equal threshold (EQT)
• Choose the auction
• Exhaustive search (ES)
• Knapsack utility approximation search (KS)
• Trade-off between speed and complexity

28

• Heuristics close to optimal for this restricted
  case
• EQT better than DOM
• KS much more computationally efficient than ES

29
Research Highlights

• Competing sellers in online auctions
• Strategies for bidding in multiple auctions
• Empirical game theory to select mechanisms and
  strategies for complex markets
• Adaptive auctions

30
Empirical Game Theory

• Game Theory is a mathematical theory which
  underpins auction- and mechanism-design
• very powerful and, at least in theory, can tell
  us what are the optimal mechanism and strategies.
• But some markets too complex to analyse in
  practice using game theory.
• too many participants and too many possible
  moves.
• Evolutionary methods do not always converge on
  robust strategies
• Empirical Game Theory
• emerging field combines principled game-theoretic
  analysis together with computer simulation
  methods.
• amenable to automation, so it may be used by
  agents themselves to decide on market mechanisms.

31
Empirical Game Theory

• Analysing strategies in Double Auctions
• Find payoffs for strategies by repeated
  simulations
• Find mixture of these pure strategies that
  constitute a evolutionary game-theoretic
  equilibrium

32
Research Highlights

• Competing sellers in online auctions
• Strategies for bidding in multiple auctions
• Empirical game theory to select mechanisms and
  strategies for complex markets
• Adaptive auctions

33
Discrete Bid English Auctions
34
Research Questions

• What effect do these discrete bid levels have on
  the auction properties?
• How should the auctioneer determine the discrete
  bid levels to use in any situation in order to
  maximise his revenue?

35
Calculating Auction Revenue
(David et al., 2005)

• We calculate the auction revenue by considering
  the probability of these three cases
• Gives the final result
• We can optimise this expression (analytically or
  numerically) to find the optimal discrete bid
  levels .

36
Optimal Bid Levels

• Uniform bidders valuation distribution

Bid increment decreases
Reserve price increases
37
Optimal Bid Levels

• Increases expected revenue.
• Decreases expected auction duration.
• Increases expected auction efficiency.

38
Learning Auction Parameters

• To calculate optimal discrete bid levels we must
  know
• The bidders valuation distribution.
• The number of participating bidders.
• Typically we do not know these parameters.
• However, we can use Bayesian Machine Learning to
  estimate these parameters online.

39
Learning Auction Parameters
(Rogers et al., 2005)
Prior Knowledge
40
Bayesian Machine Learning

• Bayesian machine learning is attractive for this
  application
• Makes use of our knowledge of how the auction
  closes.
• Allows us to incorporate prior knowledge or
  experience.
• Makes efficient use of the sparse training data
  (observations of auctions).
• Computationally efficient (no need to maximise
  multi-dimensional functions).

41
Learning the Number of Bidders
42
Learning the Number of Bidders
43
Conclusions

• MBC prima facie candidate for controlling
  complex, distributed computational systems with
  autonomous self-interested components
• Computational game theory / Mechanism design
• Evolutionary algorithms / Machine learning
• Decision theory
• Ongoing research and goals
• design of mechanisms and strategies for MBC
• gain understanding of and predict dynamic
  properties of market-based computational systems
• develop formal representation and tools
• Ultimate goal automated mechanism design

44
Partners
http//www.iam.ecs.soton.ac.uk/projects/mbc.html