Some Applications. I: The Traveling Salesman Problem.

1
Lecture 6

• Some Applications. I The Traveling Salesman
  Problem.
• This is a typical NP-Complete problem, with no
  known (expected?) polynomial-time solution. By
  1990, problems in VLSI fabrication were asking
  for good solutions in the case of 1.2 million
  cities.
• Since the only exact solution known is of the
  type generate all permutations on N elements ,
  compute the cost of the tour corresponding to the
  permutation, update the known smallest tour and
  repeat, the computational cost is O(N!). It
  would appear that, at the moment, exact solutions
  are feasible for problems around 100 (or slightly
  more) cities it would also appear that problems
  involving 10,000 cities are still on the far
  side for evolutionary based optimization
  approaches.
• Our discussion is based on Michalewicz and
  Fogels How to Solve it, Ch. 8.

2
Lecture 6

• Note one of the reasons for choosing this
  problem is that it is well-known and is
  non-trivial. Much work has been done on it, with
  many clever variants it should be a good problem
  to stimulate ideas in other areas.
• Note A survey article on combinatorial
  optimization approaches to the the problem is
  JohnsonMcGeoch1997. A follow-up
  (Walshaw2001) provides some recent variants.
  Essentially, the best algorithm for obtaining
  approximate solutions is based on one invented by
  Lin and Kernighan in 1973. The original
  Lin-Kernighan algorithm (the 1973 paper does not
  appear to be available on-line) was the best
  algorithm until 1989, and has been superseded by
  some more efficient variants. No genetic
  algorithm approaches seem to come close to it, at
  this point.

3
Lecture 6

• Test Cases in trying to determine the efficiency
  (and accuracy) of a proposed approximation
  algorithm, it is necessary to set up benchmarks.
  There are two generally accepted ways of setting
  up test cases
• The cities are distributed at random in the
  Euclidean plane, according to some uniform random
  variable. Euclidean distance is assumed. One of
  the reasons for this is that an empirical formula
  for the expected length of the minimal tour
  exists L ksqrt(NR), where N is the number
  of cities, R is the area of the square box
  containing them and k is an empirical formula -
  its suggested (from various considerations) value
  is k 0.749.
• Publicly available test cases (TSPLIB), with
  documented optimal or best-known solutions.

4
Lecture 6

• Variation Operators. For TSP, the evaluation
  function is trivial add up the lengths of the
  edges in a given tour. Fitness is thus easy to
  compute, and fitness proportionate reproduction
  is easy to set up.
• Several problems arise in the decision of which
  operators to use to provide the next generation.
• Do we use binary or integer representation for
  the names of the cities? Binary representation
  requires Nceil(log2(N)) bits and both mutation
  and crossover applied at the bit level are likely
  to lead to non-tours or, even worse, to indices
  that do not correspond to any city. A reasonable
  choice is to pick an integer representation.
  Even in this case, an array-of-integers
  representation for the chromosomes is very likely
  to introduce non-tours under both mutation and
  crossover, requiring either re-tries or patching
  up operators.

5
Lecture 6

• 2. Do we attempt to invent variants of whatever
  operators we want to introduce so that they
  repair errors (after all, error repair is a
  fairly common event in real DNA), or do we look
  for different representations for our
  chromosomes, with operators that do not introduce
  errors?
• 3. How can we guarantee that our changes in
  representation and operators will still provide a
  search over the whole search space (or that our
  repair techniques will not introduce unacceptable
  biases in our search populations)?

6
Lecture 6

• A reasonable choice (it has been the historical
  choice) attempts to introduce appropriate data
  structures and appropriate operators, trying to
  repair as little as possible. The operant
  mantra is
• lots of repair wrong data type.
• We will use the examples in MF, with 9 cities,
  numbered from 1 to 9.
• We start with a series of node-based operators
  we will look at edge-based ones later.

7
Lecture 6

• Adjacency Representation. Encode a tour as a
  list of cities. City j is listed in position i
  iff the tour leads from city i to city j. The
  vector (2 4 8 3 9 7 1 5 6) ? 1 2 4 3
  8 5 9 6 7.
• This may be slightly counterintuitive, but a bit
  of practice will convince you that it works as an
  unambiguous way of representing tours in term of
  ordered lists or arrays.
• It should be clear that not all such vectors can
  represent tours (2 4 8 1 9 3 5 7 6) has the
  disjoint cycles 1 2 4 1 and 3 8 7 5
  9 6 3.
• This representation does not support simple
  cut-and-slice crossover operators (as well as
  not supporting point mutations). We can choose
  to repair or we can choose to modify the
  operators so they leave us with legal members of
  the population ( tours).

8
Lecture 6

• J. Grefenstette published in 1985 (paper
  apparently not available on-line) several
  potential operators, and ran some experiments.
• Alternating Edges Crossover. Randomly choose an
  edge from the first parent select an appropriate
  edge from the second parent then select from
  first parent etc. If the new edge from a parent
  introduces a cycle into the partial tour being
  constructed, select a random edge from the
  remaining ones that does not introduce a cycle.
• p1 (2 3 8 7 9 1 4 5 6)
• p2 (7 5 1 6 9 2 8 4 3)
• might lead to the partial offspring
• s1 (2 5 8 7 9 1 ? ? ?)
• since up to this point the picking of alternate
  edges gives no cycle.

9
Lecture 6

• At this point, the choice of 8 from the second
  parent would introduce the edge 7 8, but the
  edge endpoint 8 has already been picked. The
  still missing vertices are 3, 4 and 6. Choosing
  the edges 7 3 and 7 6, does not introduce any
  immediate cycles, while 7 4 introduces the
  cycle 7 4 7.
• A potential partial offspring is s1 (2 5 8 7
  9 1 3 ? ?) with the parents (p1 as potential
  donor)
• p1 (2 3 8 7 9 1 4 5 6)
• p2 (7 5 1 6 9 2 8 4 3)
• 5 has already been chosen 4 and 6 remain. The
  choice 8 4 introduces the cycle 8 4 7 3
  8, so choose 8 6, which forces 9 4 s1 (2
  5 8 7 9 1 3 6 4). The offspring (2 5 8 7 9 1 6 4
  3) is also possible - as well as others.

10
Lecture 6

• Question what is the worst cost of this
  crossover scheme? The expected cost? It is
  clear that we may have to perform several checks
  on long paths before we can determine whether a
  proposed increment of a partial tour is still a
  useful partial tour.
• Subtour-chunks Crossover. Choose a random length
  subtour from one parent, another random length
  subtour from the other. Extend the tour by
  choosing edges alternating between parents. Pick
  random non-cycle-producing edges if the next
  choice would introduce a cycle. This is clearly
  quite similar to the previous crossover scheme.

11
Lecture 6

• Heuristic Crossover. Choose a random city to
  start the tour. For each of the parents, check
  which edge emanating from that city is shorter.
  Take it. That will give you the next city.
  Repeat with the parents. If both parents provide
  edges introducing a cycle, pick a random city
  (and edge) not introducing a cycle.
• Modification 1) if the shorter edge from a
  parent introduces a cycle, check the longer
  before going random 2) if you need to go
  random select the shortest edge from a pool of
  randomly selected q (parameter) edges (similar to
  tournament selection).
• To improve local optimization, another operator
  was introduced (this has a long history in its
  own right) randomly select two edges (i, j) and
  (k, m) and check if (i, j) (k, m) gt (i,
  m) (k, j). If true, replace (i, j) and (k,
  m) by (i, m) and (k, j).

12
Lecture 6

• None of the three crossover operators developed
  in this context has shown outstanding
  performance. They all suffer from the creation of
  too much disruption it would seem that the
  Adjacency List representation, although good for
  thinking in terms of schemata that fix good
  edges and attempt to construct around them, is
  somehow not very good for the problem, because it
  does not really maintain good schemata.

13
Lecture 6

• Ordinal Representation. Assume the existence of a
  reference list that will be used as a starting
  point for all other lists. This list could be C
  (1 2 3 4 5 6 7 8 9), although it does not have
  to be. A tour such as 1 2 4 3 8 5 9
  6 7 can be represented by the list l (1 1 2
  1 4 1 3 1 1) How?
• The first number on l is 1, so take the first
  city on C as the first city of the tour, and
  remove it from C. The resulting partial tour is
  1. the second number on l is also a 1, so pick
  the first element on the current version of C,
  which is 2. Remove from C partial tour 1 2.
  The next item on l is 2, corresponding to the 4
  on C. Remove the 4 from C. Partial tour 1 2
  4. Continue until all elements of l have been
  accounted for.

14
Lecture 6

• Advantage splicing works! Example the splice
  point is .
• Parents
• p1 (1 1 2 1 4 1 3 1 1) 1 2 4 3 8 5
  9 6 7
• p2 (5 1 5 5 5 3 3 2 1) 5 1 7 8 9 4
  6 3 2
• Offspring
• s1 (1 1 2 1 5 3 3 2 1) 1 2 4 3 9 7
  8 6 5
• s2 (5 1 5 5 4 1 3 1 1) 5 1 7 8 6 2
  9 3 4
• Disadvantage only the first part of the tour
  survives, the second becoming, essentially,
  random. There is too little inheritance, and the
  experimental results bear this out.

15
Lecture 6

• Path Representation. This is what one would
  expect the path 5 1 7 8 9 4 6
  2 3 is represented as the list (5 1 7 8 9
  4 6 2 3). It should be clear that this
  particular tour can be represented via 9
  equivalent lists (rotate left or right).
• Crossover Operators.
• Partially Mapped (PMX) crossover choose a
  subsequence of a tour from one parent and
  preserve the order and position of as many ciries
  as possible from the other parent. As an
  example, consider the parents with cut-points
  indicated by
• p1 (1 2 3 5 4 6 7 8 9)
• p2 (4 5 2 1 8 7 6 9 3)
• The offspring would be generated as follows

16
Lecture 6

• s1 (x x x 1 8 7 6 x x)
• s2 (x x x 4 5 6 7 x x)
• This swap also defines a mapping 1 4, 8 5, 7
  6, 6 7.
• Next, fill additional cities from the parents
  that dont lead to conflict.
• p1 (1 2 3 5 4 6 7 8 9)
• p2 (4 5 2 1 8 7 6 9 3)
• The easy ones
• s1 (x 2 3 1 8 7 6 x 9)
• s2 (x x 2 4 5 6 7 9 3)
• For the harder ones, use the mapping
• s1 (4 2 3 1 8 7 6 5 9)
• s2 (1 8 2 4 5 6 7 9 3).

17
Lecture 6

• 2) Order (OX) crossover choose a a subsequence
  of a tour from one parent and preserve the
  relative order of the cities from the other.
• Example
• p1 (1 2 3 5 4 6 7 8 9)
• p2 (4 5 2 1 8 7 6 9 3)
• s1 (x x x 5 4 6 7 x x)
• s2 (x x x 1 8 7 6 x x)
• The tour in p2, starting from its second cut
  point, is
• 9 3 4 5 2 1 8 7 6. Remove the
  cities already in s1, obtaining the partial tour
  9 3 2 1 8. Insert this partial tour after
  the second cut point of s1, obtaining the tour
• s1 (2 1 8 5 4 6 7 9 3 ). Similarly s2 (3
  4 5 1 8 7 6 9 2).

18
Lecture 6

• 3) Cycle (CX) crossover each city and its
  position comes from one of the parents. Example
• p1 (1 2 3 4 5 6 7 8 9)
• p2 (4 1 2 8 7 6 9 3 5)
• Start by taking the first city from p1
• s1 (1 x x x x x x x x)
• The next city must be from p2, and from the same
  position. This gives city 4, which is in position
  4 on p1 s1 (1 x x 4 x x x x x). In p2, in the
  same position as 4 in p1, we have city 8 s1 (1
  x x 4 x x x 8 x). We continue with s1 (1 x 3 4
  x x x 8 x), s1 (1 2 3 4 x x x 8 x). Note that
  the selection of 2 now forces the selection of 1
  and we have a cycle in our scheme. We now use
  the second parent to fill in s1 (1 2 3 4 7 6 9
  8 5).

19
Lecture 6

• If we now start form p2,
• p1 (1 2 3 4 5 6 7 8 9)
• p2 (4 1 2 8 7 6 9 3 5)
• s1 (4 1 2 8 x x x 3 x) completes the first
  cycle. Filling in from p1 s1 (4 1 2 8 5 6 7
  3 9) .
• Several other path-based operators have been
  tried, and one may wish to look in the literature
  for more.

20
Lecture 6

• Other Reordering Operators.
• Inversion. Select two points along the
  permutation, cut it at these points and re-insert
  the reversed string.
• (1 2 3 4 5 6 7 8 9) (1 2 6 5 4 3 7 8
  9).
• 2. Insertion. Select a city and insert it in a
  random place.
• 3. Displacement. Select a subtour and insert it
  in a random place.
• 4. Reciprocal Exchange. Swap two cities.
• 5. Heuristic Crossover with multiple (gt 2)
  parents. Just as Heuristic Crossover but with as
  many choices for the next city as there are
  parents - pick the shortest edge out

21
Lecture 6

• Edge-based Operators. Various people tried to
  introduce operators that would make better use of
  edge information. Grefensteette introduced a
  class of heuristic operators along the following
  lines
• Randomly select a city to be the current city c
  of the offspring.
• Select 4 edges (2 from each parent) incident to
  c.
• Define a probability distribution over the
  selected edges based on their cost. Edges
  incident on a previously visited city have
  probability 0.
• If at least one edge has positive probability,
  select probabilistically otherwise select at
  random to reach an unvisited city.
• City at other end of edge is new c.
• If tour is complete, stop if not goto 2.

22
Lecture 6

• A number of experiments indicate that only about
  60 of the edges are transferred from the
  parents 40 are random. Too much randomness to
  be effective in building more efficient
  solutions.
• Edge Recombination require that the number of
  edges inherited from the parents be as large as
  possible.
• Start from the idea that, in a tour (3 1 2 8 7 4
  6 9 5) the edges are (3 1), (1 2), (2 8), (8 7),
  (7 4), (4 6), (6 9), (9 5), (5 3). Recall that
  the edges are undirected (5 3) (3 5)
  direction is not important. Position of a city
  on a tour is not important tours are circular.
• Evaluation Function minimize the total cost of
  the edges that constitute a legal tour.

23
Lecture 6

• Example. Start with two parents p1 (1 2
  3 4 5 6 7 8 9), p2 (4 1 2 8 7 6 9 3 5).
• Using both parents, collect the edges available
• City 1 (1 9), (1 2), (1 4)
• City 2 (2 1), (2 3), (2 8)
• City 3 (3 2), (3 4), (3 9), (3 5)
• City 4 (4 3), (4 5), (4 1)
• City 5 (5 4), (5 6), (5 3)
• City 6 (6 5), (6 7), (6 9)
• City 7 (7 6), (7 8)
• City 8 (8 7), (8 9), (8 2)
• City 9 (9 8), ( 1), ( 6), (9 3).

24
Lecture 6

• The algorithm. Start with either one of the
  start cities in the edge lists of the parents
  or with a city with the smallest number of edges
  - this latter criterion maximizes the probability
  that you will finish the tour using the parental
  set of edges. Once you have decided on the first
  city, add an edge to a city with the smallest
  number of edges. Continue.
• If we start with City 1 we can reach 2, 4, 9. 2
  and 4 have 3 edges 9 has 4. Pick, randomly,
  between 2 and 4. Say you picked 4. You now have
  (1 4 x x x x x x x). 4 has edges to 1, 3, 5.
  The edge to 1 has already been used 5 has fewer
  edges than 3 (1 4 5 x x x x x x). Continuing in
  this fashion, we arrive at the offspring (1 4 5 6
  7 8 2 3 9) - without needing to introduce a new
  edge to complete the tour. It appears
  (experimentally) that failure occurs in less than
  1.5 of the cases.

25
Lecture 6

• A variant attempts to maintain subtours common to
  both parents note that, if a city has only two
  or three edges associated, one (or both) of the
  edges must be common to both parents. The
  algorithm prefers to choose edges common to
  both, before looking at any others.
• This seems to have led to better results.
• Finding further operators that improve on the
  know edge ones is an open question (or was as of
  2000).

26
Lecture 6

• Matrix Representations and Operators. There have
  been at least 3 attempts.
• Precedence Matrix. A tour (3 1 2 8 7 4 6 9 5) is
  represented by the matrix the element mij
  contains a 1
• iff the city i occurs before the city j
• on the tour. Properties of the matrix
• 1. The number of 1s is exactly n(n - 1)/2.
• 2. mii 0 for all 1 i n.
• 3. If mij 1 and mjk 1 then mik 1.

27
Lecture 6

• Claim If the number of 1s is less than n(n -
  1)/2, with the other conditions still satisfied,
  the cities are partially ordered, which is
  another way of saying that the matrix can be
  completed in at least one way to obtain a legal
  tour.
• The operators devised in Fox McMahon, Genetic
  Operators for Sequencing Problems, in Foundations
  of Genetic Algorithms, G. J. E. Rawlins, ed.,
  Morgan Kaufman, 1991 were the operators of
  intersection and union.
• Intersection. The intersection operator is based
  on the observation that the (bitwise)
  intersection of two tour matrices results in a
  matrix that satisfies conditions 2 and 3, with a
  number of 1s no greater than n(n - 1)/2, and
  thus extendible to a tour

28
Lecture 6

• The two parents p1 (1 2 3 4 5 6 7 8
  9) p2 (4 1 2 8 7 6 9 3
  5) correspond to the matrices below

29
Lecture 6

• The intersection is given by the matrix
• And we have a partial order. For example, city 1
  must precede cities 2, 3, 5, 6, 7, 8 and 9 city
  6 is only required to precede city 9 etc..
• How do we complete the tour?

30
Lecture 6

• Select one of the parents add some 1s unique to
  this parent complete the matrix into a tour
  sequence through an analysis of the sums of the
  rows and columns. A possible completion is given
  by the matrix on the right below, which gives the
  tour (1 2 4 8 7 6 3 5 9).

31
Lecrture 6

• The second operator is the union operator. It is
  based on the observation that subsets from two
  matrices can be safely combined provided the two
  subsets have empty intersection. It thus
  partitions the set of cities into two disjoint
  groups and copies the bits of one matrix for the
  first group, and the bits of the other for the
  second group.
• For example, p1 can lead to the set 1, 2,
  3, 4, p2 to the set 5, 6, 7, 8, 9, with the
  union matrix at the right. It needs to
  be completed, using the same kind of
  techniques used for the completion of the
  intersection.

32
Lecture 6

• 2. Binary Tours. The matrix element mij contains
  a 1 iff the tour goes directly from city i to
  city j. There is only one nonzero entry in each
  row and column. Matrix (a) below represents the
  tour ( 1 2 4 3 8 6 5 7 9) (or any rotation left
  or right). Matrix (b) represents another tour
  (or does it?).

33
Lecture 6

• The requirement that each row and column contain
  a single 1 allows for non-tours to be represented
  - matrix (b) on the previous slide represents the
  two subtours (1 2 4 5 7) and (3 8 6 9). An
  example of subtours that can then be connected to
  make up a full tour occurs below. The genetic
  algorithm will be restricted to subtours of
  length at least 3 - fixed into full tours after
  it terminates.

34
Lecture 6

• Operators. Two were defined. The first took a
  matrix, randomly selected several rows and
  columns, removed the set bits at the
  intersections of those rows and columns and
  replaced them randomly.
• Ex. matrix (a) corresponds to a tour. Assume
  rows 4, 6, 7, 9 and columns 1, 3, 5, 8, 9 are
  selected. The marginal sums are calculated and
  stored the bits at the intersections are removed
  and replaced randomly, agreeing with the marginal
  sums. We have an example below

35
Lecture 6

• Replacing the old rows and column fragments with
  the new ones, we have a matrix that represents
  not a single tour but two subtours (1 2 4 5 7)
  and (3 8 6 9) - the full matrix (8.6(b))
  introduced earlier.
• The second operator starts with two parents and a
  matrix of 0s. It fills the new matrix with the
  result of the intersection of the parents 1
  bits in both result in a 1 bit in the offspring.
  After this initial phase, the operator copies
  alternately one set bit from each parent until no
  bits exist in either parent that can be copied
  without violating the matrix restrictions. If,
  at this point, the matrix still has some rows
  without a 1, they receive a 1 at random - again
  satisfying the matrix constraints.

36
Lecture 6

• We start with matrix (a) below, representing the
  subtours (1 5 3 7 8) and (2 4 9 6). The second
  parent matrix (b) represents the full tour (1 5 6
  2 7 8 3 4 9).

37
Lecture 6

• The first result of the first phase of the
  operator application appears in (a) below the
  second phase leads to (b). Notice that a number
  of rows (columns) in (b) have no 1 it would not
  be possible to fill from a parent without
  violating the conditions.

38
Lecture 6

• Random filling (with the constraint of having
  just one 1 in each row and column) leads to the
  matrix below. It does represent a tour, (1 5 6 2
  3 4 9 7 8).
• Another solution would lead to the two subtours
  (1 5 3 4 9) and (2 7 8 6).

39
Lecture 6

• Binary Matrices with Crossover (XO) Operators. We
  define crossover operators by first choosing one
  or more between column positions, swapping the
  values in the blocks so identified - see the
  picture below. The resulting matrices are
  (generally) illegal, although they have the
  correct total number of 1s.

40
Lecture 6

• We see the result in the picture below some rows
  (columns) have no 1s, some have two. We need to
  repair these intermediate matrices. For example,
  the 1 in position m1,4 in (a) could be moved to
  position m1,8.
• After this type of correction we may end up
  with the first offspring providing the legal tour
  (1 2 8 4 3 6 5 7 9), while the second offspring
  provides the two subtours (1 6 5 7 2 8 9) and (3
  4).

41
Lecture 6

• The second step of the repair algorithm needs to
  be applied only to the second matrix cut and
  connect subtours to produce a legal tour. You
  can use information about which edges exist in
  the parents to choose where to splice the
  subtours for example, (2 4) is present in one of
  the parents, so we splice (3 4) and (4 3) between
  2 and 8.
• A heuristic inversion operator was also
  introduced. The claim - based on a number of
  experiments - was that this was reasonable, with
  one 318-city experiment resulting in a tour
  within 0.6 of optimal.
• When the splicing of subtours is required, local
  considerations can be useful - and some systems
  incorporate them.

42
Lecture 6

• A modified GA scheme would involve some local
  optimizations applied to all elements of the
  population.

43
Lecture 6

• We will introduce some notions from
  Papadimitriou Steiglitz Combinatorial
  Optimization and then well continue. They
  refer to the TSP.
• Definition. Let f and g denote tours. A k-change
  (also know as k-opt) neighborhhood of f is
  defined as
• Nk(f) g g is a tour, obtained from f as
  follows remove k edges from f and replace them
  with k edges. Example for 2-change

5
2
5
2
1
1
3
4
3
4
6
6
7
7
44
Lecture 6

• It has been found that 2-change and 3-change
  (especially 3-change) lead to very effective
  heuristics for TSP. 4-change does not seem to be
  sufficiently effective to bother the extra
  computational cost is not paid back in extra
  improvements leading to faster convergence to a
  better approximation results mostly due to Lin.
• The local optimization algorithm can be stated as
  follows
• Let t be a tour.
• procedure local search
• begin
• t some initial tour
• while k-improve(t) ? no do
• t k-improve(t)
• return t
• end

45
Lecture 6

• An algorithm was proposed
• Use a 2-opt procedure to replace each tour in
  the current population with a locally optimal
  tour.
• Allow higher quality solutions to generate more
  offspring.
• Use recombination and mutation.
• Search for the minimum by using local search
  within each individual.
• Repeat steps 2-4 until a termination condition
  is met.
• For reproduction, a variant of the order
  crossover (OX) was used.

46
Lecture 6

• Two parents are given - with the cut points
  marked by p1 (1 2 3 4 5 6 7 8
  9) p2 (4 5 2 1 8 7 6 9 3), and
  they produce offspring as follows. First, the
  segments between cut points are copied into the
  offspring s1 (x x x 4 5 6 7 x
  x) s2 (x x x 1 8 7 6 x x).
• Next, instead of starting from the second cut
  point of one parent - as was the case for OX -
  the cities from the other parent are copied in
  the same order from the beginning of the string,
  omitting those symbols that are already present.
  This leads to the descendants s1 (2 1 8
  4 5 6 7 9 3) s2 (2 3 4 1 8 7 6 5
  9).

47
Lecture 6

• Edge Assembly Crossover (EAX). Assume two
  parents have been selected Y. Nagata S.
  Kobayashi, Edge Assembly Crossover a High-Power
  Genetic Algorithm for the TSP, in Proceedings of
  the Seventh International Conference on GAs,
  Morgan Kaufman, 1997

48
Lecture 6

• Construct a graph G that contains the edges of
  both parents (A and B).
• Then construct a set of AB-cycles an AB-cycle is
  an even length subcycle of G with edges that come
  alternately from A and B. An AB-cycle can repeat
  cities but not edges.
• Construction assume we start at city 6 and that
  the first edge of the AB-cycle is (6 1). This
  edge comes from parent A.

49
Lecture 6

• We pick the next edge from parent B, originating
  from city 1. Select edge (1 3). You can further
  select edges (in order) (3 4) from A, (4 5) from
  B, (5 6) from A (which gives an odd cycle) and (6
  3) from B. The latter choice gives us an even
  cycle, after dropping the first two edges.
  Remove the edges of the AB-cycle from G, and
  repeat. This may well lead to ineffective
  AB-cycles containing just two edges.

50
Lecture 6

• The next step involves selecting a subset of
  AB-cycles (an E-set) to extend into a suitable
  tour. Two selection mechanisms were introduced
  1) a deterministic heuristic method 2) a random
  method (each AB-cycle has probability 0.5). Once
  the E-set is constructed, we construct an
  intermediate offspring C
• set C A
• for each edge e Î E-set do
• if e Î A then C C - e
• if e Î B then C C ? e
• C is a set of disjoint subtours that cover all
  cities.
• We now introduce a greedy procedure that
  incorporates local search to obtain a single
  legal tour.

51
Lecture 6

• The construction
• Start with a subtour T with the smallest number
  of edges.
• Select two edges, one from T and another from
  all edges in remaining subtours. Assume that
  (vq, vq1) Î T and (vr, vr1) Ï T, where the
  indices are modulo the number of nodes in the
  corresponding subtours.
• Take the two edges out of the subtours, reducing
  the cost by cut(q, r) L (vq, vq1) L (vr,
  vr1), where L denotes the length of the
  edge.
• Define link(q, r) minL (vq, vr) L (vq1,
  vr1), L (vq, vr1) L (vq1, vr), and
  replace the original edges, if the new ones
  provide a shorter connection - we seek
  minr,qlink(q, r) - cut(q, r).

52
Lecture 6

• 5) After making a connection, the number of
  subtours is decreased by one and we repeat the
  process until we have a full tour, which is the
  offspring of parents A and B.
• Observations and conclusions
• EAX produced results that are a small fraction
  of 1 above optimum for many test cases from 100
  to over 3000 cities.
• A very high selective pressure is used. An
  offspring replaces a parent only if it is better
  than both parents. If the offspring isnt better,
  the EAX operator is applied for up to N 100
  times in an attempt to improve it.
• EAX incorporates an implicit mutation by
  introducing new edges into an offspring during
  the process of connecting subtours.

53
Lecture 6

• The inver-over operator. How about a pure
  evolutionary algorithm? No local search local
  optima avoidance strong selection pressure. Tao
  Michalewicz, Evolutionary algorithms for the
  TSP, Proceedings of the 5th Parallel Problem
  Solving from Nature Conference, Lecture Notes in
  CS, 1498, Springer-Verlag, 1998.
• Each individual competes only with its
  offspring
• There is only one (adaptive) variation operator
• The number of times the operator is applied to
  an individual during a generation is variable.

54
Lecture 6

• The Algorithm (p is small in (0, 1).

55
Lecture 6

• Example. Assume the current S (2 3 9 4 1 5 8 6
  7) and c 3.
• If rand() lt p, then select a (random) city c
  from S, say 8, and invert (2 3 9 4 1 5 8 6 7)
  (2 3 8 5 1 4 9 6 7).
• If not, select another (random) individual from
  the population, say (1 6 4 3 5 7 9 2 8). Search
  this individual for the city c which is next
  to c in this new individual you find city 5.
  Thus the segment for inversion in S starts at 3
  and ends at 5. The new offspring is S
  (2 3 5 1 4 9 8 6 7).
• Note that a substring 3-5 came from the second
  parent. Also note that the repeat loop can be
  repeated many times.

56
Lecture 6

• Some claims (from Michalewicz Fogel, or,
  beating your own drum - maybe with
  justification).
• Probably quickest evolutionary algorithm
  developed up to 1998.
• Three parameters p (the probability of
  generating random inversion), population size,
  and number of iterations in termination
  condition.
• Precision and stability appear (empirically)
  good for small ( 105 cities - almost 100
  accuracy for tour), computational time acceptable
  ( 4s).
• New operator introduces a mix of inversion
  (mutation) and crossover. Better than algorithms
  that rely on random inversion only.
• p ( 0.02 for all tests) determines the
  proportion of blind and guided inversions.