Pareto Coevolution

1
Pareto Coevolution

• Presented by KC Tsui
• Based on 1

2
Sorting Networks and MOO

• First work on coevolution with host-parasite
  model and two independent evolving but
  interacting gene pool
• sorting networks (SNs) and test cases (TCs)
• Two fitness functions
• SN score according to number of test cases it
  succeeded in handling
• TC score according to number of failed SNs that
  tests on it
• In MOO, a pareto front defines a set of solutions
  that have the same fitness according to a
  aggregated measure of all objectives

3
Motivations

• Coevolutionary systems plays an arms race game
  and provides a task for each other to tackle
• Each system requires the ability to dynamically
  adjust the learning environment
• There is no guarantee that coevolution will lead
  to effective learning
• Borrow idea from multi-objective optimization to
  formulate the task to be learned

4
Search as Teaching and Learning

• Search involve a fitness landscape, but can be
  dynamically changed according to different
  objectives
• Teachers create a search gradient
• Learners following a search gradient
• A good teacher is one that is able to identify
  some knowledge gap in some learners
• A good learner is one that has learned the tasks
  set by some teachers
• Evolution The process of variation to discover
  better teachers and students

5
Learning

• Pareto dominance, commonly used in MOO, is used
  to obtain a rank among the population concerned
• Learner x (pareto) dominates learner y iff Gx,w gt
  Gy,w and Gx,v gt Gy,v, G is a payoff matrix and
  w,v are teachers
• x and y are mutually non-dominating iff Gx,w gt
  Gy,w and Gx,v lt Gy,v

6
Learning (cont.)

• Learning a recursive process of identifying the
  non-dominated learners, exclude them from the
  population and start over again (find the pareto
  layers)
• Pareto layer Fn is less broad in competence than
  some learners in Fn-1
• Every learner in Fn-1 can do something better
  than some other learner in Fn
• Ranking is done by some kind of tournament

7
Teaching

• Given the payoff matrix G (rowlearners
  columnsteachers) for assessing student
  performance, transform it to become a student
  dominance matrix M (rowteachers, columnpair of
  students) for assessing teacher performance
• Score of a teacher j is
• i.e. the value of a learner pair across the
  learners distinguished by j discounted by total
  number of teachers that distinguish it
• j distinguish x from y if Gx,j gt Gy,j

8
Results

• Second best performance in the majority problem
  for cellular automata
• Similar idea has been applied to game strategy
  discovery 2

9
Discussion

• Pros
• Smooth divide-and-conquer strategy
• Cons
• Payoff matrix G (and hence M) is not always
  readily available or computed easily
• Requires a lot of function evaluations

10
References

1. Sevan G. Ficici and Jordan B. Pollack, Pareto
   Optimality in Coevolutionary Learning, Computer
   Science Technical Report CS-01-216, University of
   Brandeis.
2. J. Noble and R.A. Watson, Pareto coevolution
   Using performance against coevolved opponents in
   a game as dimensions for Pareto selection , in
   Proceedings of the Genetic and Evolutionary
   Computation Conference, GECCO-2001, pp. 493-500.
   Morgan Kauffman, San Francisco.