Heuristic Optimization Methods Prologue

1
Heuristic Optimization MethodsPrologue

• Chin-Shiuh Shieh

2
Course Information

• ???? ?????
• Heuristic Optimization Methods
• ???? ???
• ???? (?)1-2,5
• ???? ?504
• Could be switched to ?501B
• ????
• http//bit.kuas.edu.tw/csshieh

3
Course Information (cont)

• Objective
• The study of heuristic optimization methods
• Course Outline
• Introduction to Optimization
• Calculus, Optimization, and Search
• Linear Programming, Combinatorial Optimization
• Heuristic Approaches to Optimization
• Genetic Algorithm
• Ant Colony System
• Simulated Annealing
• Particle Swarm Optimization
• Tabu Search
• Memetic Algorithm

4
Course Information (cont)

• Readings
• Kwang Y. Lee and Mohamed A. El-Sharkawi, Eds.,
  Modern Heuristic Optimization Techniques, John
  Wiley Sons, 2008.
• Johann Dreo, Patrick Siarry, Alain Petrowski,
  and Eric Taillard, Metaheuristics for Hard
  Optimization, Springer, 2006.

5
What Is Optimization?

• Optimization defined in Wikipedia
• In mathematics, statistics, empirical sciences,
  computer science, or management science,
  mathematical optimization (alternatively,
  optimization or mathematical programming) is the
  selection of a best element (with regard to some
  criteria) from some set of available alternatives.

6
What Is Optimization? (cont)

• A mathematical formulation
• Given a function f  A?R from some set A to
  the real numbers
• Sought an element x0 in A such that f(x0)  f(x)
  for all x in A ("minimization") or such
  that f(x0)  f(x) for all x in A ("maximization").

7
Challenges

• Object function
• High dimensionalities
• Local optimum
• Huge search space

8
Example Optimization Problems

• Minimize the costs of shipping from production
  facilities to warehouses
• Maximize the probability of detecting an incoming
  warhead (vs. decoy) in a missile defense system
• Place sensors in manner to maximize useful
  information
• Determine the times to administer a sequence of
  drugs for maximum therapeutic effect
• Find the best red-yellow-green signal timings in
  an urban traffic network
• Determine the best schedule for use of laboratory
  facilities to serve an organizations overall
  interests

9
Major Subfields

• Linear Programming, a type of convex programming,
  studies the case in which the objective function
  f is linear and the set of constraints is
  specified using only linear equalities and
  inequalities.
• Nonlinear Programming studies the general case in
  which the objective function or the constraints
  or both contain nonlinear parts.

10
Major Subfields (cont)

• Integer Programming studies linear programs in
  which some or all variables are constrained to
  take on integer values. This is not convex, and
  in general much more difficult than regular
  linear programming.
• Stochastic Programming studies the case in which
  some of the constraints or parameters depend on
  random variables.

11
Major Subfields (cont)

• Calculus of Variations seeks to optimize an
  objective defined over many points in time, by
  considering how the objective function changes if
  there is a small change in the choice path.
• Combinatorial Optimization is concerned with
  problems where the set of feasible solutions is
  discrete or can be reduced to a discrete one.

12
Major Subfields (cont)

• Heuristics and Meta-heuristics make few or no
  assumptions about the problem being optimized.
  Usually, heuristics do not guarantee that any
  optimal solution need be found. On the other
  hand, heuristics are used to find approximate
  solutions for many complicated optimization
  problems.

13
Optimization Methods

• (Exact) Algorithms
• An algorithm is sometimes described as a set of
  instructions that will result in the solution to
  a problem when followed correctly.
• Unless otherwise stated, an algorithm is assumed
  to give the optimal solution to an optimization
  problem.
• That is, not just a good solution, but the best
  solution.

14
Optimization Methods (cont)

• Approximation Algorithms
• Approximation algorithms (as opposed to exact
  algorithms) do not guarantee to find the optimal
  solution.
• However, there is a bound on the quality, e.g.,
  for a maximization problem, the algorithm can
  guarantee to find a solution whose value is at
  least half that of the optimal value.
• We will not see many approximation algorithms
  here, but mention them as a contrast to heuristic
  algorithms.

15
Optimization Methods (cont)

• Heuristic Algorithms
• Heuristic algorithms do not guarantee to find the
  optimal solution.
• Heuristic algorithms do not even necessarily have
  a bound on how bad they can perform.
• However, in practice, heuristic algorithms
  (heuristics for short) have proven successful.
• Near optimal solutions in reasonable time.
• Most of this course will focus on this type of
  heuristic solution method.

16
What Is Heuristic?

• Heuristic defined in Wikipedia
• Heuristic (Greek ????s??", "find" or "discover")
  refers to experience-based techniques for
  problem solving, learning, and discovery. Where
  the exhaustive search is impractical, heuristic
  methods are used to speed up the process of
  finding a satisfactory solution mental short
  cuts to ease the cognitive load of making a
  decision.

16
17
Why Not Always Exact Methods?

• The running time of the algorithm
• For reasons explained soon, the running time of
  an algorithm may render it useless on the problem
  you want to solve.
• The link between the real-world problem and the
  formal problem is weak
• Sometimes you cannot properly formulate a COP/IP
  that captures all aspects of the real-world
  problem. If the problem you solve is not the
  right problem, it might be just as useful to have
  one (or more) heuristic solutions, rather than
  the optimal solution of the formal problem.

17
18
Heuristic Optimization Methods

• Random Walk
• Hill-climbing
• Genetic Algorithms
• Particle Swarm Optimization
• Ant Colony Optimization
• Simulated Annealing
• Tabu Search
• Memetic Algorithm

19
A Challenge

• Maximize F1(x,y)

20
(No Transcript)