Chapter 4 Informed Search and Exploration

1
Chapter 4 Informed Search and Exploration
2
Outline

• Informed (Heuristic) search strategies
• (Greedy) Best-first search
• A search
• (Admissible) Heuristic Functions
• Relaxed problem
• Subproblem
• Local search algorithms
• Hill-climbing search
• Simulated anneal search
• Local beam search
• Genetic algorithms
• Online search
• Online local search
• learning in online search

3
Informed search strategies

• Informed search
• uses problem-specific knowledge beyond the
  problem definition
• finds solution more efficiently than the
  uninformed search
• Best-first search
• uses an evaluation function f(n) for each node
• e.g., Measures distance to the goal lowest
  evaluation
• Implementation
• fringe is a queue sorted in increasing order of
  f-values.
• Can we really expand the best node first?
• No! only the one that appears to be best based on
  f(n).
• heuristic function h(n)
• estimated cost of the cheapest path from node n
  to a goal node
• Specific algorithms
• greedy best-first search
• A search

4
Greedy best-first search

• expand the node that is closest to the goal
• Straight line distance
  heuristic

5
Greedy best-first search example
6
Properties of Greedy best-first search

• Complete?
• Optimal?
• Time?
• Space?

No
7
A search

• evaluation function f(n) g(n) h(n)
• g(n) cost to reach the node
• h(n) estimated cost to the goal from n
• f(n) estimated total cost of path through n to
  the goal
• an admissible (optimistic) heuristic
• never overestimates the cost to reach the goal
• estimates the cost of solving the problem is less
  than it actually is
• e.g., never overestimates the
  actual road distances
• A using Tree-Search is optimal if h(n) is
  admissible
• could get suboptimal solutions using Graph-Search
• might discard the optimal path to a repeated
  state if it is not the first one generated
• a simple solution is to discard the more
  expensive of any two paths found to the same node
  (extra memory)

8
Straight line distance heuristic
9
A search example
10
Optimality of A

• Consistency (monotonicity)
• n is any successor of n, general triangle
  inequality (n, n, and the goal)
• consistent heuristic is also admissible
• A using Graph-Search is optimal if h(n) is
  consistent
• the values of f(n) along any path are
  nondecreasing

11
Properties of A

• Suppose C is the cost of the optimal solution
  path
• A expands all nodes with f(n)
• A might expand some of nodes with f(n) C on
  the goal contour
• A will expand no nodes with f(n) C, which are
  pruned!
• Pruning eliminating possibilities from
  consideration without examination
• A is optimally efficient for any given heuristic
  function
• no other optimal algorithm is guaranteed to
  expand fewer nodes than A
• an algorithm might miss the optimal solution if
  it does not expand all nodes with f(n)
• A is complete
• Time complexity
• exponential number of nodes within the goal
  contour
• Space complexity
• keeps all generated nodes in memory
• runs out of space long before runs out of time

12
Memory-bounded heuristic search

• Iterative-deepening A (IDA)
• uses f-value (g h) as the cutoff
• Recursive best-first search (RBFS)
• replaces the f-value of each node along the path
  with the best f-value of its children
• remembers the f-value of the best leaf in the
  forgotten subtree so that it can reexpand it
  later if necessary
• is efficient than IDA but generates excessive
  nodes
• changes mind go back to pick up the second-best
  path due to the extension (f-value increased) of
  current best path
• optimal if h(n) is admissible
• space complexity is O(bd)
• time complexity depends on the accuracy of h(n)
  and how often the current best path is changed
• Exponential time complexity of Both IDA and RBFS
  
• cannot check repeated states other than those on
  the current path when search on Graphs Should
  have used more memory (to store the nodes
  visited)!

13
Straight line distance heuristic
14
RBFS example
15
Memory-bounded heuristic search (contd)

• SMA Simplified MA (Memory-bounded A)
• expands the best leaf node until memory is full
• then drops the worst leaf node the one has the
  highest f-value
• regenerates the subtree only when all other paths
  have been shown to look worse than the path it
  has forgotten
• complete and optimal if there is a solution
  reachable
• might be the best general-purpose algorithm for
  finding optimal solutions
• If there is no way to balance the trade off
  between time an memory, drop the optimality
  requirement!

16
(Admissible) Heuristic Functions
the number of misplaced tiles
total Manhattan (city block) distance
7 tiles are out of position
h1?
40331021 14
h2?
17
Effect of heuristic accuracy

• Effective branching factor b
• total of nodes generated by A is N, the
  solution depth is d
• b is b that a uniform tree of depth d containing
  N1 nodes would have
• well-designed heuristic would have a value close
  to 1
• h2 is better than h1 based on the b

18
Effect of heuristic accuracy

• Domination
• h2 dominates h1 if for any
  node n
• A using h2 will never expand more nodes than A
  using h1
• every node n with will be
  expanded
• the larger the better, as long as it does not
  overestimate!

19
Inventing admissible heuristic functions

• h1 and h2 are solutions to relaxed (simplified)
  version of the puzzle.
• If the rules of the 8-puzze are relaxed so that a
  tie can move anywhere, then h1 gives the shortest
  solution
• If the rules are relaxed so that a tile can move
  to any adjacent square, then h2 gives the
  shortest solution
• Relaxed problem A problem with fewer
  restrictions on the actions
• Admissible heuristics for the original problem
  can be derived from the optimal (exact) solution
  to a relaxed problem
• Key point the optimal solution cost of a relaxed
  problem is no greater than the optimal solution
  cost of the original problem
• Which should we choose if none of the h1 hm
  dominates any of the others?
• We can have the best of all worlds, i.e., use
  whichever function is most accurate on the
  current node
• Subproblem
• Admissible heuristics for the original problem
  can also be derived from the solution cost of the
  subproblem.
• Learning from experience

20
Local search algorithms and optimization

• Systematic search algorithms
• to find (or given) the goal and to find the path
  to that goal
• Local search algorithms
• the path to the goal is irrelevant, e.g.,
  n-queens problem
• state space set of complete configurations
• keep a single current state and try to improve
  it, e.g., move to its neighbors
• Key advantages
• use very little (constant) memory
• find reasonable solutions in large or infinite
  (continuous) state spaces
• (pure) Optimization problem
• to find the best state (optimal configuration )
  based on an objective function, e.g. reproductive
  fitness Darwinian, no goal test and path cost

21
Local search example
22
Local search state space landscape

• elevation the value of the objective function
  or heuristic cost function

heuristic cost function
global minimum

• A complete local search algorithm finds a
  solution if one exists

• A optimal algorithm finds a global minimum or
  maximum

23
Hill-climbing search

• moves in the direction of increasing value until
  a peak
• current node data structure only records the
  state and its objective function
• neither remember the history nor look beyond the
  immediate neighbors
• like climbing Mount Everest in thick fog with
  amnesia

24
Hill-climbing search - example

• complete-state formulation for 8-queens
• successor function returns all possible states
  generated by moving a single queen to another
  square in the same column (8 x 7 56 successors
  for each state)
• the heuristic cost function h is the number of
  pairs of queens that are attacking each other

best moves reduce h 17 to h 12
local minimum with h 1
25
Hill-climbing search greedy local search

• Hill climbing, the greedy local search, often
  gets stuck
• Local maxima a peak that is higher than each of
  its neighboring states, but lower than the global
  maximum
• Ridges a sequence of local maxima that is
  difficult to navigate

• Plateau a flat area of the state space landscape
• a flat local maximum no uphill exit exists
• a shoulder possible to make progress
• can only solve 14 of 8-queen instance but fast
  (4 steps to S and 3 to F)

26
Hill-climbing search improvement

• Allows sideways move with hope that the plateau
  is a shoulder
• could stuck in an infinite loop when it reaches a
  flat local maximum
• limits the number of consecutive sideways moves
• can solve 94 of 8-queen instances but slow (21
  steps to S and 64 to F)

• Variations
• stochastic hill climbing
• chooses at random probability of selection
  depends on the steepness
• first choice hill climbing
• randomly generates successors to find a better
  one
• All the hill climbing algorithms discussed so far
  are incomplete
• fail to find a goal when one exists because they
  get stuck on local maxima
• Random-restart hill climbing
• conducts a series of hill-climbing searches
  randomly generated initial states
• Have to give up the global optimality
• landscape consists of a large amount of
  porcupines on a flat floor
• NP-hard problems

27
Simulated annealing search

• combine hill climbing (efficiency) with random
  walk (completeness)
• annealing harden metals by heating metals to a
  high temperature and gradually cooling them
• getting a ping-pong ball into the deepest crevice
  in a humpy surface
• shake the surface to get the ball out of the
  local minima
• not too hard to dislodge it from the global
  minimum
• simulated annealing
• start by shaking hard (at a high temperature) and
  then gradually reduce the intensity of the
  shaking (lower the temperature)
• escape the local minima by allowing some bad
  moves
• but gradually reduce their size and frequency

28
Simulated annealing search - Implementation

• Always accept the good moves
• The probability to accept a bad move
• decreases exponentially with the badness of the
  move
• decreases exponentially with the temperature T
  (decreasing)
• finds a global optimum with probability
  approaching 1 if the schedule lowers T slowly
  enough

29
Local beam search

• Local beam search keeps track of k states rather
  than just one
• generates all the successors of all k states
• selects the k best successors from the complete
  list and repeats
• quickly abandons unfruitful searches and moves to
  the space where the most progress is being made
• Come over here, the grass is greener!
• lack of diversity among the k states
• stochastic beam search chooses k successors at
  random, with the probability of choosing a given
  successor having an increasing value
• natural selection the successors (offspring) if
  a state (organism) populate the next generation
  according to is value (fitness).

30
Genetic algorithms

• Genetic Algorithms (GA)successor states are
  generated by combining two parents states.
• population s set of k randomly generated states
• each state, called individual, is represented as
  a string over a finite alphabet, e.g. a string of
  0s and 1s 8-queens 24 bits or 8 digits for
  their positions

31
Genetic algorithms (contd)

• fitness (evaluation) function return higher
  values for better states,
• e.g., the number of nonattacking pairs of queens
  (min 0, max 8 7/2 28)
• randomly choosing two pairs for reproducing based
  on the probability proportional to fitness
  score not choosing the similar ones too early
• 24/(24232011) 31
• 23/(24232011) 29 etc

32
Genetic algorithms (contd)

• a crossover point is randomly chosen from the
  positions in the string
• larger steps in the state space early and smaller
  steps later
• each location is subject to random mutation with
  a small independent probability

• schema a substring in which some of the
  positions can be left unspecified
• instances strings that match the schema
• GA works best when schemas correspond to
  meaningful components of a solution.

33
Genetic algorithms (contd)