Using Simulated Annealing to Calculate the Trembles of Trembling Hand Perfection

1
Using Simulated Annealing to Calculate the
Trembles of Trembling Hand Perfection

• Liam Wagner
• Department of Mathematics and
• St Johns College

Stuart McDonald School of Economics
2
Outline of Seminar

• Background to the problem
• Computation of Nash equilibria
• Simulated annealing
• Trembling hand algorithm
• Numerical solution of a three person game
• Conclusion

3
Background to the Problem

• Trembling hand perfection
• Removing non-optimizing behaviour from sub-game
  perfect equilibria
• Uses trembles to remove isolated information
  sets
• Simulated annealing
• Global search algorithm
• Uses trembles to exit local optima

4
Background to the Problem

• Is there a connection ?
• Simulated annealing
• Trembling hand perfection
• Can simulated annealing give an algorithm for
  trembling hand perfection?

5
Computation of Nash Equilibria I

• Well known algorithms
• Lemke and Howson (1964)
• Wilson (1971) and Scarf (1973) for N-person games
• These algorithms utilize the geometric properties
  of games
• Quantal response equilibria of McKelvey and
  Palfrey (1995, 1998)
• Trace algorithms of Harsanyi and Selten (1988)

6
Computation of Nash Equilibria II

• Use of Monte Carlo simulation to solve games
• Ulam (1954) applied to strategic games
• Georgobiani and Torondzadze (1980) applied to
  rectangular games

7
Simulated Annealing I

• Simulated annealing is a form Monte Carlo
  sampling
• Monte Carlo methods sample directly from the
  support of the target distribution
• Simulated annealing draws its samples from a
  Markov chain that converges towards the target
  distribution

8
Simulated Annealing II

• The algorithm operates via an update rule
• Accept j if the energy function E(j) is greater
  than the energy function E(i) on the previously
  sampled value i
• Accept j if exp((E(j)-E(i))/c) gt U0,1, where c
  is the temperature function
• The temperature function is important as it
  establishes the trembles

9
Trembling Hand Algorithm I

• For a Nash equilibrium to satisfy trembling hand
  perfection
• We define for a game G a perturbed game (G,?),
  where ? is a vector of trembles
• If the sequence of Nash equilibria generated by
  each perturbed game converges, then it will
  converge on a perfect equilibrium

10
Trembling Hand Algorithm II

• For a perturbed game (G,?), where ? is a vector
  of trembles
• Each player i's mixed strategy set for (G,?) is
• p is a Nash equilibrium of (G,? ) iff

11
Trembling Hand Algorithm III

• As we are moving along a sequence of perturbed
  games (G,?(k))
• We accept a mixed strategy a if the payoff
  function for each player i
• Otherwise accept a if

12
Trembling Hand Algorithm IV

• Criteria for convergence to proper equilibrium
• Create a sequence of ?-proper equilibria, where
• For a sequence of ?-proper equilibria, if they
  converge, then the point of convergence will be a
  proper equilibrium

13
Trembling Hand Algorithm V

• We accept a mixed strategy a if for each player I
• Otherwise accept a if

14
Trembling Hand Algorithm VI

• Whats going on? We have the following
  collection of neighbourhoods
• Attach the following probabilities

15
Numerical Results I

• The game tree for a three person game (Selten
  1975)

16
Numerical Results II

• Perturbed pay-off functions
• where the mixed strategies are denoted by the as
  and the es are the trembles

17
Numerical Results III

• Simulations of the pay-off functions

18
Conclusion

• Contribution of this paper
• Working algorithm that certainly provides Nash
  equilibria
• The example we have used gives what seems to be
  the perfect equilibrium

19
Conclusion

• Future work must include
• An investigation of the convergence properties
• An investigation of its relation
• Bayesian refinements like Bayes and Markov
  perfection
• Proper equilibrium
• Sequential equilibrium

20
Contact Details

• Stuart McDonald School of Economics,
• s.mcdonald_at_mailbox.uq.edu.au
• Liam Wagner Department of Mathematics,
• LDW_at_maths.uq.edu.au
• http//www.maths.uq.edu.au/ldw