Heuristic and Metaheuristic Algorithms

1
Heuristic and Metaheuristic Algorithms

• Sam Allen
• sda_at_cs.nott.ac.uk
• ASAP Research Group

2
Heuristic algorithms What are they?

• Wikipedia says
• In computer science, a heuristic is a technique
  designed to solve a problem that ignores whether
  the solution can be proven to be correct, but
  which usually produces a good solution or solves
  a simpler problem that contains or intersects
  with the solution of the more complex problem.
• Heuristics are intended to gain computational
  performance or conceptual simplicity, potentially
  at the cost of accuracy or precision.

3
Huh?

• Basically heuristic algorithms solve really
  difficult problems reasonably well (which is
  pretty subjective) in a reasonable amount of time
  (also pretty subjective).
• Examples of problems that are tackled with
  heuristic algorithms (more on these in Matts
  lecture on Friday)
• Cutting/Packing
• Graph problems, i.e. Travelling Salesman Problem
• Scheduling/Timetabling

4
2D packing an example
5
2D packing an example
5
2
3
1
4
Sequence 1,2,3,4,5
X
6
2D packing an example
5
3
2
1
4
Sequence 1, 4, 3, 2, 5
?
7
That was easy!

• For 5 pieces there are 5 x 4 x 3 x 2 x 1
  different orderings for the placement
• 120 combinations
• Piece of cake for a computer to try each
  combination in (nearly) no time at all

8
How do we pack all of these?
?
(50 pieces)
9
Thats not so easy

• 50 pieces means 50 x 49 x..2 x 1 different
  orderings
• 304140932017133780436126081660647688443776415689
  60512000000000000
• Thats quite a lot really. If a computer could
  evaluate 1000 orderings per second itd still
  take approximately
• 96442456880115988215412887386050129516671872047693
  1506 years! (and thats without being allowed to
  rotate the pieces)

10
Heuristic algorithms

• Cast your mind all the way back to the second
  slide.
• Maybe a heuristic algorithm would be good for
  tackling this type of problem? (commonly known as
  a combinatorial optimis,zation problem)
• I hope so! (my PhD depends on it)

11
My current research

• The 3-D strip packing problem
• Given a container (a big box, a lorry, an
  aeroplane cargo hold etc)
• and a number of boxes to load inside the
  container
• what is the best way to place them all so that
  they take up the shortest (height) space as
  possible?

12
Applications of strip packing

• This problem can be seen to have applications
  directly in industry
• Loading pallets
• Cutting blocks of polystyrene or wood from a
  larger block minimising wastage
• And more theoretical applications
• multi-dimensional resource scheduling

13
How it works (Preprocessing stage)

• First the algorithm takes a list of boxes, and
  the width and length of the container
• The boxes are then rotated so that each of their
  width length height
• They are then sorted decreasingly by width

14
How it works (Packing stage)
1
2
3
4
Gap
15
How it works (Packing stage)
2
3
4
1
Gap
16
How it works (Packing stage)
2
3
1
X
4
17
How it works (Packing stage)
2
3
4
1
X
18
How it works (Packing stage)
2
3
4
Gap
1
X
19
How it works (Packing stage)
3
4
Gap
2
1
X
20
How it works (Packing stage)
4
2
3
Gap
1
X
21
How it works (Packing stage)
4
Gap
2
3
X
1
X
22
How it works (Packing stage)
4
2
3
X
3
X
1
X
23
How it works (postprocessing)

• Due to the nature of the algorithm, towers may
  form.
• Towers are boxes with large height dimensions
  and lower width and length dimensions that are
  placed late on in the packing process, jutting
  out over the top of the profile.
• The tallest tower is removed from the packing,
  rotated so that it is effectively knocked down
  and placed back into the packing at the lowest
  point available. This is repeated until the
  solution is not improved any further.

24
1
12
9
8
3
2
10
5
6
7
11
4
3
25
1
12
9
8
3
2
10
5
6
7
11
4
3
26
1
12
9
8
3
2
10
5
6
7
11
4
3
27
1
12
9
8
3
2
10
5
6
7
11
4
3
28
old height
1
Improvement
new height
12
9
8
3
2
10
5
6
7
11
4
3
29
Have you been paying attention?
1
4
3
2

• Q) How many different orders can these boxes be
  placed in?
• A) 4 x 3 x 2 x 1 24 different orders
• Q) Which of the following problems could be
  solved with heuristic techniques?

30

• Cracking a combination lock

31

• Planning a route from Nottingham to London

32
Metaheuristics what are they?

• Wikipedia says
• A metaheuristic is a heuristic method for
  solving a very general class of computational
  problems by combining user given black-box
  procedures usually heuristics themselves in a
  hopefully efficient way.

33
Examples of metaheuristics

• Hill climbing / Greedy search
• Tabu search
• Simulated Annealing
• Genetic algorithms
• Ant colony optimization
• many more

34
Explanation part 1

• A metaheuristic is a higher level algorithm
• This means it doesnt actually know what the
  problem is its solving
• So it can be applied to many different types of
  problems

35
Explanation part 2

• So all a programmer needs to do to adapt a
  metaheuristic to their new problem is provide
• A function to generate new valid solutions based
  on the current one g()
• An evaluation function f()

36
Explanation part 3

• Advantage of metaheuristics
• Generally produce higher quality results (given
  enough time) than simple heuristics
• Disadvantage
• They take a lot longer as they have to generate
  and evaluate many solutions rather than just one

37
Greedy search for 2d packing

• Lets define our function g then.
• g will take a solution, in this case the order of
  boxes to pack
• It will return a new solution by randomly
  swapping the order of 2 of the boxes
• E.g. 1,2,3,4 -gt 3,2,1,4 or 4,2,3,1 etc
• So g(1,2,3,4) 3,2,1,4 perhaps

38
Greedy search for 2d packing

• So now we have g (the harder function to define
  in this case) lets define f
• f() simply returns the highest location in the
  packing when placed in a bottom right way
• E.g.
• f(1,2,3,4) 6

39
Greedy search for 2d packing

• Were going to run 3 iterations of a greedy
  search to find the best packing we can
• Start with our initial solution 1,2,3,4,5
• f(1,2,3,4,5) 14, our best solution so far
• Iteration 1 g(1,2,3,4,5) 1,2,5,4,3
• f(1,2,5,4,3) 12, better than our best so far
  so well take this new order as our current one

40
Greedy search for 2d packing

• Iteration 2 g(1,2,5,4,3) 2,1,5,4,3
• f(2,1,5,4,3) 13, worse than our current best
  so ignore this order
• Iteration 3 g(1,2,5,4,3) 1,4,5,2,3
• f(1,4,5,2,3) 10, our best so far so we keep
  this
• Return 1,4,5,2,3 as our best solution

41
Demonstration/Questions?

• http//www.cs.nott.ac.uk/sda/viewer.zip