Introduction to Simulated Annealing

1
Introduction to Simulated Annealing

• gw

2
Inspiration Metallurgy

• Solid is heated to melting point
• High-energy, high-entropy state
• Removes defects/irregularities
• Temp is very slowly reduced
• Recrystallization occurs (regular structure)
• New internal state of diffused atoms
• Fast cooling induces fragile structure

3
Simulated Annealing

• Method of sampling search space
• State feasible solution
• High energy many solutions still possible
• Cooling perform local search
• Ground state optimal solution

4
Basic Algorithm
Create initial solution
Modify to create new solution
Evaluate solutions
Stochastically select a solution
Reduce temperature
5
Acceptance Criteria
As a function of Temp
As a function of evaluation difference
6
Cooling Schedule

• Fast

• Slow

http//en.wikipedia.org/wiki/Simulated_annealing
7
Acceptance Characteristics

• At high temperatures, more likely to accept
  worse solutions
• At low evaluation differences, more likely to
  accept worse solution
• Result ? wide search of solution space
• Moves towards a region of the search space
  containing good solutions
• Moves towards low-energy regions
• Moves downhill (gradient descent)

8
Parameters

• Initial solution random
• New solution select - Gaussian distribution
• Temperature schedule
• Initial
• Decay function Ti1 cTi
• Final
• Iterations at temp
• Proportional to neighborhood size

9
Usage

• Search heuristic
• Global optima in large solution space
• Best for discrete-valued solutions
• Traveling Salesman
• VLSI circuit design

10
References / Resources

• Optimization by Simulated Annealing
• Kirkpatrick, S. Gelatt, C. D. and Vecchi, M. P.
• Science 220671-680, 1983.
• Simulated annealing a proof of convergence
• Granville, V., Krivanek, M., and Rasson, J.-P.
• IEEE Trans. Pattern Analysis and Machine
  Intelligence, 16652-656, 1994.
• Simulated Annealing applet for Traveling Salesman
  problem
• http//www.heatonresearch.com/articles/64/page1.ht
  ml