010'141 Engineering Mathematics II Lecture 12 Stochastic Search

1
010.141 Engineering Mathematics IILecture
12Stochastic Search

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University

2
Outline

• Complete Deterministic Search
• Heuristic Search
• Probabilistic Search
• Stochastic hillclimbing
• Evolution Strategies
• Genetic Algorithms
• Variants
• Applications

3
Lecture Evaluation

• http//cse.snu.ac.kr/survey/TakeSurvey.asp?PageNum
  ber1SurveyID6K3m93MI6o9KK
• Dates 2006. 10. 15(Mon)?to 2006. 10. 26(Fri)
• Password 9922
• Confidentiality of your responses is guaranteed
• However the office will ask me to give a penalty
  (I think 1) if the survey is not completed in
  time

4
Reminder Complete Search

• You have previously studied complete search
  algorithms
• Breadth-First Search
• Depth-First, Backtracking Search
• The former is guaranteed to find a solution, but
  maybe very inefficiently
• The latter may be faster, but can also get into
  infinite loops
• Neither takes advantage of any knowledge about
  the problem space

5
Reminder Heuristic Search

• Heuristic search algorithms extend complete
  search algorithms with a heuristic to guide
  search
• Heuristic in this context, a value indicating
  how good a potential solution is
• Deterministic Hillclimbing Algorithm
• CandInitial
• While not(solution(Cand)) do
• Newnext_neighbour(Cand)
• if heur(New) gt heur(Cand) Cand New
• End while

6
Deterministic Hillclimbing Search

• Deterministic hillclimbing search can be very
  fast and effective in simple search spaces
• But it has a serious problem
• If the search starts close to a local optimum, it
  will get stuck at the local optimum
• If a search landscape has many local optima,
  deterministic hillclimbing will perform poorly

7
Stochastic Hillclimbing Search

• A small change to deterministic hillclimbing
  makes it a much more reliable algorithm
• Introduce a small probability ?, and accept a new
  solution even if it is worse, with probability ?
• CandInitial
• While not(solution(Cand)) do
• Newnext_neighbour(Cand)
• random(P)
• if (heur(New) gt heur(Cand) or (P lt ?)) Cand
  New
• End while

8
Stochastic Hillclimbing Search

• However such an algorithm can still get stuck in
  very long or infinite loops depending on the
  next_neighbour algorithm
• Generating the neighbour randomly can help
• CandInitial
• While not(solution(Cand)) do
• Newrandom_mutate(Cand)
• random(P)
• if (heur(New) gt heur(Cand) or (P lt ?)) Cand
  New
• End while

9
Stochastic Hillclimbing Search

• Considering the different possible values of Cand
  as states, the search algorithm has become a
  Markov chain
• In particular, with most probability choices, it
  satisfies the ergodic property
• As a consequence, the probability of eventually
  finding the optimal solution is 1.0 for most
  variants
• Of course, eventually might be a very long time

10
Multi-start Hillclimbing Search

• Our initial starting point might be very bad
• We can run the algorithm multiple times
• Repeat
• Candrandom
• While not termination_condition do
• Newrandom_mutate(Cand)
• random(P)
• if (heur(New) gt heur(Cand) or (P lt ?)) Cand
  New
• End while
• Until solution(Cand)

11
Parallel Hillclimbing Search

• But it might be more efficient to do it in
  parallel
• Poprandom
• While not (solution ? Pop do
• randomly select Ind ? Pop biased by heur(Ind)
• Newrandom_mutate(Ind)
• if (heur(New) gt heur(Ind)) replace Ind by New in
  Pop
• End while

12
Evolution Strategies

• This is a common form of an evolutionary
  algorithm known as an evolution strategy
• Most commonly, in evolution strategies, an
  individual is a vector of real numbers, and the
  mutation operation is Gaussian mutation
• The individual New is generated by setting the
  mean of the Gaussian (normal) distribution to
  Ind, and the standard deviation ? to some fixed
  value, then sampling to find New
• More sophisticated versions either change ?
  according to a fixed schedule, or evolve ? as
  well
• Enables the algorithm to find the optimum
  accurately

13
Generational Hillclimbing Search

• A variant uses separate generations
• Pop(0)random t0
• While not (solution ? Pop(t)) do
• while size(Pop(t1)) lt size(Pop(t)) do
• randomly select Ind ? Pop(t) biased by
  heur(Ind)
• Newrandom_mutate(Ind)
• add New to Pop(t1)
• end while
• tt1
• End while

14
Recombination in Hillclimbing Search

• So far, the only way we have used to generate new
  individuals is to change existing ones in some
  stochastic way
• Mutation
• But for many problems, combining good candidate
  solutions is another good way to generate new
  good candidates

15
Recombinational Hillclimbing Search

• Pop(0)random t0
• While not (solution ? Pop(t)) do
• while size(Pop(t1)) lt size(Pop(t)) do
• randomly select Ind1, Ind2 ? Pop(t) biased by
  heuristic
• with probability p1
• Newrandom_mutate(Ind1)
• else with probability p2
• Newrandom_combine(Ind1, Ind2)
• else New Ind1
• add New to Pop(t1)
• end while
• tt1
• End while

16
Genetic Algorithm

• This algorithm is generally known as a genetic
  algorithm
• It is often used with a discrete representation
  (integer, string, binary)
• A wide range of stochastic mutation and
  recombination (crossover) operators can be
  defined
• Also a wide range of selection operators

17
Evolution
18
Darwinian Evolution (1850s)

• Multiple populations competing for limited
  resources
• Dynamically changing populations with births and
  deaths
• Inheritance children are like their parents
• Variation children are not exactly the same as
  their parents
• Fitness different individuals have different
  probabilities to survive and reproduce

19
Generation Structure
Variants

• Steady state
• Children compete with parents
• Evolution Strategies
• Evolutionary Programming
• Kangaroos

• Generational
• Children replace parents
• Genetic Algorithms
• Genetic Programming
• Antechinus
• Many insects

20
Operators Mutation only
Variants

• Early Evolution Strategies
• Early Evolutionary Programming
• Bacteria

21
Mutation and Crossover (Sexual Reproduction)
Variants

• Most Higher Animals and Plants
• Genetic Algorithms
• Genetic Programming

22
Representation Linear, Discrete
Variants

• Usual in Genetic Algorithms
• All known organisms

23
Representation Continuous
Variants

• Usual in Evolution Strategies, and today common
  in Evolutionary Programming

• Crossover
• X,Y ? rX (1-r)Y
• r drawn from uniform distribution
• 0,1
• or -0.5, 1.5

24
Representation Tree Structured
Variants

• Genetic Programming

25
Diversity Resource Competition
Variants

• Resource competition encourages animals to
  specialise in niches

• Simulated in artificial systems
• Fitness sharing
• Very effective in encouraging diversity

26
Diversity Speciation
Variants

• In natural systems, separate species arise
  rapidly
• Separate species do not inter-breed
• Allows each gene pool to develop separately
• Herring Gulls and Black-backed Gulls do not breed

• Simulated in artificial systems
• Tag-template matching
• Not very effective in encouraging diversity

27
Diversity Spatial Distribution
Variants

• Spatial separation allows groups of animals to
  evolve separately
• A natural process leading to speciation and
  diversity
• Herring / Black-backed gulls are an example
• Two separate species in Britain, but gradual
  change around the North Pole

28
Co-evolution
Variants

• Models reality
• Useful for competitive games, social modelling
• Can encourage diversity generality

29
Where are we?UsingEvolutionarySystems
30
Why not use Evolutionary Algorithms?
Application

• Evolutionary algorithms are computationally
  expensive
• If there is already a good method, evolutionary
  methods may be unsuitable
• Use simplex for a linear programming problem
• Though I recently saw a paper showing that a GA
  could out-perform simplex for very large LPPs
• But many problems dont have good, simple
  techniques
• We often tailor the problem to the technique
• No longer necessary
• Sometimes the cost of learning to use the
  technique is greater than the computational cost
• Evolutionary techniques are relatively easy to
  master
• Evolutionary algorithms are naturally parallel

31
Optimisation
Application

• Evolutionary methods are highly flexible

• Problems can be

• High Dimensional
• Mixed
• Deceptive

• Objective can be

32
Optimisation
Application

• Constraints can be

• Optima can be

33
Multi-Objective Optimisation
Application

• Multi-objective
• Requires multiple evaluation
• EC computational cost comparable

34
Learning Relationships
Application

• Example learning the spatial relationships
  determining animal distributions

35
Learning Games
Application

• Co-evolutionary systems
• Recreational Games
• Checkers
• Fogels Blondie
• Backgammon

36
Planning
Application

• Aler et al
• EVOCK system
• Evolves plans

37
Modelling
Application

• Evolving Systems of Equations
• Difference Equations
• Ordinary Differential Equations
• Partial Differential Equations

38
Design
Application

• Radar Beam

39
Evolvable Hardware
Application

• Adaptive Cell Phones

40
Summary

• Complete Deterministic Search
• Heuristic Search
• Probabilistic Search
• Stochastic hillclimbing
• Evolution Strategies
• Genetic Algorithms
• Variants
• Applications

41
?????