Coevolution

1
Coevolution

• Chapter 6, Essentials of Metaheuristics, 2013
• Spring, 2014
• Metaheuristics
• Byung-Hyun Ha

R2
R3
2
Outline

• Introduction
• 1-Population competitive coevolution
• 2-Population competitive coevolution
• N-Population cooperative coevolution
• Niching diversity maintenance methods
• Summary

3
Introduction

• Coevolution in biology
• The change of a biological object triggered by
  the change of a related object (from Wikipedia)
• In metaheuristics
• Considering competitive or cooperative case,
  usually
• To provide diversity in system
• To discover not just high-quality but robust
  solutions
• To solve complex and high-dimensional problems by
  breaking them along semi-decomposable lines

fitness evaluation by challenging
a population
another population
competitive
cooperative
fitness evaluation, jointly
4
Introduction

• Examples (Talbi, 2009)
• Predator-prey coevolution for constraint
  satisfaction
• Main population potential solutions
• High-quality those that satisfy a large number
  of constraints
• Secondary population constraints
• High-quality those that are violated by many
  solutions
• ? Focusing on hard constraints by preferring them
• Coevolution for function optimization
• Rosenbrock
• Populations
• Each one representing a decomposed function,
  fi(xi, xi1)
• An individual corresponding to xi
• Competitive coevolution
• Each population finding local optimum
• Cooperative coevolution
• Iterative evolution of a population based on the
  others

5
Introduction

• Types
• 1-Population competitive coevolution
• 2-Population competitive coevolution
• N-Population cooperative coevolution
• Diversity maintenance (Niching)
• ...
• Fitness in coevolution
• Assessing fitness
• Gathering test results of an individual in the
  context of others
• Assessing fitness of the individual based on the
  results
• Internal vs. external fitness
• Relative vs. absolute
• Possible problems
• Selection and breeding
• Determining progress of search

?
6
1-Population Competitive Coevolution

• Optimizing solution designed to compete in some
  game, mostly
• Each individuals fitness is assessed by playing
  against other individuals.
• e.g., checkers players, robot soccer team
  strategies
• Intuition to internal fitness
• Problems in improving learning gradient of search
  space
• c.f., situation of beginners evaluated by a guru
• Alternatives
• Assessing how badly one loses
• Employing panels of various levels of skill
• Making them play each other
• Giving self-adjusting learning gradient

7
1-Population Competitive Coevolution

• Assessing external fitness
• Testing against guru
• Testing against players from previous generations
• Testing against some external system -)
• An abstract procedure

8
1-Population Competitive Coevolution

• Relative internal fitness assessment (n
  individuals)
• c.f., cost of testing
• Pairwise relative fitness assessment
• n/2 tests (problem?)
• Complete relative fitness assessment
• n(n 1)/2 tests (too many?)
• K-fold relative fitness assessment
• kn tests (still many?)
• c.f., More precise K-fold relative fitness
  assessment
• Single-elimination tournament relative fitness
  assessment
• (n 1) tests
• Better individuals, more tests, but fairly noisy
• Fitnessless selection
• e.g., using tournament selection with size 2
• c.f., not totally free from prematurity

9
2-Population Competitive Coevolution

• Role of two populations
• Primary population
• Looking for good (and robust) solutions
• Alternative (or foil) population
• Searching for most challenging test cases
• e.g., Finding sorting network that gives fewest
  comparisons
• Primary sorting network
• Alternative hard-to-sort arrays of numbers

10
2-Population Competitive Coevolution

• Fitness assessment and breeding strategies
• Sequential
• Parallel
• Growing up together reduced number of tests
• Parallel Previous
• Improving gradient a bit

11
2-Population Competitive Coevolution

• Fitness assessment
• Specific way for each strategy
• c.f., number of tests, statistical dependency,
  using fittest individuals, ...
• Arms races and loss of gradient
• P improves too rapidly than others
• All in P have the same fitness, so do all of Q
• Selection not working
• Solution
• Pause evolution of too rapidly improving
  population until gradient recovers?
• Using Parallel Previous?

12
2-Population Competitive Coevolution

• Example of container terminal operations
• Container-grounding position determination by
  weighted sum of scores
• Solution as a list of weights

13
N-Population Cooperative Coevolution

• Attacking a problem decomposable into subproblems
• e.g., approaches robot soccer team with different
  players
• 1. a team as an individual
• 2. a player as an individual
• 3. middle ground?
• Finding good subsolutions and integrating them
• A population for each subproblem
• Testing a solution by grouping individuals from
  populations
• Assessing an individuals fitness from test
  results of group including it
• Coevolution strategies
• Sequential
• Parallel

14
N-Population Cooperative Coevolution

• An abstract sequential procedure
• Parallel procedure
• Fitness assessment

15
N-Population Cooperative Coevolution

• Pathological conditions in testing
• Possibility of laziness
• e.g., fitness assessment with one excellent
  player
• Relative overgeneralization
• c.f., fitness as average of test results, usually
• Miscoordination

16
N-Population Cooperative Coevolution

• Example of container terminal operations
• Stacking container in a yard
• Two-level approach
• 1. determining a block by weighted sum of scores
• 2. determining a stack by weighted sum of scores
• Solution as a list of weights
• Employing cooperative coevolution approach

17
Other Variations

• Cooperative-Competitive Coevolution
• ...
• Co-adaptive ...

18
Niching Diversity Maintenance Methods

• Preventing early convergence (revisited)
• Increasing sample (population) size
• Adding noise to Tweak procedure
• Being less selective among individuals (picking
  less fit ones more often)
• Adding random restarts to system
• Adding explicit separation constraints in your
  population
• Adding different individuals from the current
  ones in the population
• e.g., Scatter Search with Path Relinking
• And more, here..
• Punish individuals in some way for being too
  similar to one another
• Fitness sharing
• Crowding

19
Niching Diversity Maintenance Methods

• Fitness sharing
• Encouraging diversity in individuals by reducing
  fitness for being too similar to on another
• Sharing function s
• i receives punishment s(i, j) for j is near i
• where d(i, j) distance between i and j, ?
  neighborhood radius
• Revised fitness
• fi (ri)?/?j s(i, j), where ri is actual fitness
• Crowding
• Making similar individuals more likely to be
  picked for death in a steady-state system
• Algorithms
• Restricted Tournament Selection, Deterministic
  Crowding, ...

20
Niching Diversity Maintenance Methods

• Similarity
• Three ways, at least
• Phenotypically they behave similarly
• Genotypically they have roughly the same makeup
• Individuals may have similar fitness (?)
• Examples of measure
• (Metric) distance
• e.g., Euclidian distance, Hamming distance
• Proximity
• e.g., Jaccard coefficient, cosine similarity
• Graphs? trees?

21
Summary

• Coevolution
• Internal vs. external fitness
• Fitness assessment by testing
• 1-Population competitive coevolution
• 2-Population competitive coevolution
• Fitness assessment and breeding strategies
• Arms races and loss of gradient
• N-Population cooperative coevolution
• A problem decomposable into subproblems
• Pathological conditions in testing
• Diversity maintenance methods
• Fitness sharing, crowding