Scatter Search and Path Relinking: Methodology and Applications

1
Scatter Search and Path Relinking Methodology
and Applications

• Manuel Laguna

2
Metaheuristic

• A metaheuristic refers to a master strategy that
  guides and modifies other heuristics to produce
  solutions beyond those that are normally
  generated in a quest for local optimality.
• A metaheuristic is a procedure that has the
  ability to escape local optimality

3
Typical Search Trajectory
4
Metaheuristic Classification

• x/y/z Classification
• x A (adaptive memory) or M (memoryless)
• y N (systematic neighborhood search) or S
  (random sampling)
• Z 1 (one current solution) or P (population of
  solutions)
• Some Classifications
• Tabu search (A/N/1)
• Genetic Algorithms (M/S/P)
• Scatter Search (M/N/P)

5
SS and PR Publications
Source Martí, R. (2004) Scatter Search
Wellsprings and Challenges, to appear in EJOR.
6
SS and PR Bibliography
7
SS Web Impact
Source Martí, R. (2004) Scatter Search
Wellsprings and Challenges, to appear in EJOR.
8
Recent Scatter Search Applications

• Neural Network Training
• Multi-Objective Routing Problem
• OptQuest A Commercial Implementation
• A Context-Independent Method for Permutation
  Problems
• Classical and Periodic Vehicle Routing
• Matrix Bandwidth Minimization
• Arc Crossing Minimization
• Project Scheduling under Uncertainty
• P-Median Problems
• Software Testing
• DNA Sequencing
• Network Design Problems in Telecommunications
• Variable Selection Problems
• Bus Routing

9
Scatter Search

• Seminal ideas originated in the late 60s and
  first description appeared in
• Glover, F. (1977) Heuristics for Integer
  Programming Using Surrogate Constraints,
  Decision Sciences, vol. 8, pp. 156-166.
• Modern version of the method is described in
• Laguna, M. and R. Martí (2003) Scatter Search
  Methodology and Implementations in C, Kluwer
  Academic Publishers Boston, ISBN 1-4020-7376-3,
  312 pp.

10
Scatter Search Overview
Repeat until P PSize
Diversification Generation Method
Improvement Method
Improvement Method
Reference Set Update Method
Stop if MaxIter reached
Solution Combination Method
Improvement Method
Subset Generation Method
No more new solutions
Diversification Generation Method
11
Reference Set Update Method(Initial RefSet)
b1 high-quality solutions
Objective function value to measure quality
Max-min criterion according to distances that
measure diversity
b2 diverse solutions
RefSet of size b
12
Subset Generation

• Subset Type 1 all 2-element subsets.
• Subset Type 2 3-element subsets derived from the
  2-element subsets by augmenting each 2-element
  subset to include the best solution not in this
  subset.
• Subset Type 3 4-element subsets derived from the
  3-element subsets by augmenting each 3-element
  subset to include the best solutions not in this
  subset.
• Subset Type 4 the subsets consisting of the best
  i elements, for i 5 to b.

13
Combination Method for Continuous Variables
14
Alternative Combination Method for Continuous
Variables
15
Variable Number of Solutions
Best
Quality
1
2
. . .
Generate 5 solutions
Generate 3 solutions
Generate 1 solution
Worst
b
RefSet of size b
16
Dynamic RefSet Update Method
Best
Quality
1
2
. . .
Worst
b
RefSet of size b
17
Static RefSet Update
Pool of new trial solutions
Best
Quality
1
2
. . .
Updated RefSet Best b from RefSet ? Pool
Worst
b
RefSet of size b
18
2-Tier RefSet
Solution Combination Method
Improvement Method
RefSet
b1
Try here first
b2
If it fails, then try here
19
3-Tier RefSet
Solution Combination Method
Improvement Method
RefSet
b1
Try here first
b2
If it fails, then try here
Try departing solution here
b3
20
Rebuilding
RefSet
Rebuilt RefSet
b1
b2
Diversification Generation Method
Reference Set Update Method
21
GA vs. SS
22
Parallel Scatter Search (EJOR Special Issue)

• An empirical investigation on parallelization
  strategies for Scatter Search
• B. Adenso-Díaz, S. García-Carvajal, S. Lozano
• Solving feature subset selection problem by a
  parallel scatter search
• F. G. López, M. G. Torres, B. Melián, J. A.
  Moreno and J. M. Moreno-Vega

23
Parallel SS of Díaz, et al (2004)
Phase I
Phase II
24
Phase I Single Walk
25
Phase I Multiple Walk/Independent
26
Phase I Multiple Walk/Cooperative
27
Phase II Single Walk
28
Phase II Multiple Walk/Independent Threads
29
Phase II Multiple Walks/Cooperative Threads (A)
30
Phase II Multiple Walks/Cooperative Threads (B)
31
Results of Testing 18 Variants

• Solution Quality
• Static updating is better than dynamic
• No interaction effects between phases
• Phase I
• MW outperforms SW
• Cooperation not significantly better
• Phase II
• Cooperative variants are superior

• Execution Time
• Static updating is faster than dynamic
• Phase I
• Cooperative MW
• Single walk
• Independent MW
• Phase II
• Independent MW
• Single walk
• Cooperative MW

32
Parallel SS of López, et al (2004)
33
Multiobjective Optimization

• SSPMO A Scatter Search Procedure for Non-Linear
  Multiobjective Optimization
• J. Molina, M. Laguna, R. Marti and R. Caballero

34
Phase I Tabu Searches
2
x1
x2
5
x4
x6
1
7
6
3
4
x5
x3
35
Phase II Scatter Search

• RefSet consists of
• Best single-objective solutions
• Diverse solutions
• Linear combinations
• An updated list of efficient solutions is
  maintained throughout the search

36
Improvement Method
Efficient frontier
xi
Ideal (xi , xj)
f2
Compromise point for (xi, xj)
xj
New trial solution
Search area
f1
37
SSPMO vs. SPEA2
38
Disjoint Frontier
39
Path Relinking

• Seminal ideas originated in connection with tabu
  search
• Glover, F. and M. Laguna (1993) Tabu Search, in
  Modern Heuristic Techniques for Combinatorial
  Problems, C. Reeves (ed.) Blackwell Scientific
  Publications, pp. 70-150.
• Modern versions have been applied as a
  combination method within scatter search and in
  the improvement phase of GRASP

40
Path Relinking Research
Source Web of Science 24 Articles found on
3/11/2004
41
Relinking Solutions
Guiding solution
Initiating solution
Original path Relinked path
42
Multiple Guiding Solutions
Guiding solution
Initiating solution
Original path Relinked path
43
Linking Solutions
Initiating solution
Guiding solution
Original path Relinked path
44
GRASP (Greedy Randomized Adaptive Search
Procedure)

• Multi-start and local search procedure introduced
  by Feo and Resende (1989)
• do
• x ? RandomizedGreedyConstruction(?)
• x ? LocalSearch(x)
• x ? UpdateBest(x)
• while (termination criterion not met)

45
GRASP with Path Relinking

• Originally suggested in the context of Graph
  Drawing by Laguna and Marti (1999)
• A guiding solution is selected from a small set
  (size 3) of elite solutions
• Initiating solutions are the result of GRASP
  iterations
• The number of relinking steps is the number of
  vertices in the graph
• Local search is applied every ? E steps
• Extensions and comprehensive review are due to
  Resende and Riberio (2003) GRASP with Path
  Relinking Recent Advances and Applications
  http//www.research.att.com/mgcr/doc/sgrasppr.pdf

46
Applications

• three index assignment problem 1, 3
• job-shop scheduling problem 1, 2
• prize-collecting Steiner tree problem 9
• MAX-CUT problem 17
• quadratic assignment problem 31
• routing private circuits in telecommunication
  networks 40
• p-median problem 43
• 2-path network design problem 47
• Steiner problem in graphs 49
• capacitated minimum spanning tree problem 53

47
Relinking Strategies

• Periodical relinking ? not systematically applied
  to all solutions
• Forward relinking ? worst solution is the
  initiating solution
• Backward relinking ? best solution is the
  initiating solution
• Backward and forward relinking ? both directions
  are explored
• Mixed relinking ? relinking starts at both ends
• Randomized relinking ? stochastic selection of
  moves
• Truncated relinking ? the guiding solution is not
  reached

48
Issues for Future PR Research

• Selection of initiating and guiding solutions
• Application of local search to intermediate
  solutions
• Testing of standalone PR procedures

49
Questions
http//leeds-faculty.colorado.edu/laguna