Exploration

1
Exploration

• Chapter 4COMP151 Feb 5 2007

2
Chap 4

• Best-first search
• Greedy best-first search
• Recursive best-first search
• A search
• Heuristics
• Local search algorithms
• Hill-climbing search
• Simulated annealing search
• Local beam search
• Genetic algorithms
• Online Search

informed searchsystematic algorithms
sections 4.3-4.5todays focus
3
Review Heuristic Search

• Previous lecture looked at informed search
  algorithms
• heuristics rules that encode some useful
  knowledge about the search space
• For these algorithms, the result was a path from
  the start state to the goal state
• informed search through systematic algorithms

4
Local Search

• Now well look at search wherethe path is
  irrelevant the goal state itself is the
  solution
  
• State space set of "complete" configurations
• Goal some configuration satisfying constraints
• local search
• keep a single "current" state, try to improve it
  in next state
  

5
Local Search Example

• Problem put n queens on an n n board with no
  two queens on the same row, column, or diagonal
  
• path is irrelevant, just need to reach goal

6
Local Search Example

• Robot is standing in parking lot between geology
  and computer science, with no maps or other
  knowledge
• goal arrive at Burns Tower
• variation robot has sensor that allows it to
  determine its current distance from Burns Tower

7
Local Search Algorithms

• maintain current state, move only to neighbor
  states
• benefits
• low memory usage usually constant
• can find solutions in large spaces where
  systematic search is unsuitable

8
Optimization Problems

• Rather than an explicit goal, optimization
  problems look for states that maximize some
  objective function
• For this lecture, will assume goal is to maximize
  objective function. Minimization problems are
  easily converted to maximization functions (? -1)

9
State Space Landscape
10
Hill-Climbing Search 8-queens Problem

• h number of pairs of queens that are attacking
  each other, either directly or indirectly
• h 17 for the above state

11
Hill-Climbing Search 8-queens Problem

• A local minimum with h 1

12
Local Search

• State Space Landscape
• location state
• height objective function
• global maximum best state in entire landscape
• local maximum a state in which all neighbor
  states have a lower height
• complete always finds a goal if one exists
• optimal always finds a global maximum

13
Hill-Climbing Search

• Always move to highest neighbor state(best
  objective function value)
• climbing Everest in thick fog with amnesia
• greedy local search

14
Ridges in State Landscape
15
Hill-Climbing Search

• Problem depending on initial state, can get
  stuck in local maxima
  

16
Plateaux and Shoulders

• Basic hill climbing will not take side-steps to
  states with same height.
• treats flat spaces as local maxima
• Allowing sideways moves can get off flat spaces,
  but will cause infinite loops on plateux
• solution limit number of consecutive sideways
  steps

17
8 Queens Example

• Basic Hill Climbing
• succeeds on 14 initial states (4 moves)
• gets stuck on 86 initial states (3 moves)
• Hill Climbing with sideways steps
• max 100 sidesteps
• 94 success
• 21 steps on success, 64 steps on failure

18
Stochastic Hill Climbing

• Chose a random uphill move
• rather than best uphill move
• probability can favor steeper moves
• slower ascent, but may find better solutions
• Variation First choice hill climbing
• generate random moves, pick first that is an
  ascent
• good when state has many (thousands) of successors

19
Random Restart Hill Climbing

• If at first you dont succeed
• Conduct a series of hill climbing searches from
  random state states
• complete with probability ? 1 (will eventually
  generate goal as start state)
• expected number of restarts 1/pwhere p is
  probability of success(14 for 8-queens ? 7
  restarts)

20
Simulated Annealing Search

• Escape local maxima by allowing some "bad" moves
  but gradually decrease their frequency
  

21
Simulated Annealing Search

• One can prove If T decreases slowly enough, then
  simulated annealing search will find a global
  optimum with probability approaching 1
  
• Widely used in VLSI layout, airline scheduling,
  etc
  

22
Local Beam Search

• Keep track of k states rather than just one
• Start with k randomly generated states
• At each iteration, all the successors of all k
  states are generated
  
• If any one is a goal state, stop else select the
  k best successors from the complete list and
  repeat.
  

23
Genetic Algorithms

• A successor state is generated by combining two
  parent states
  
• Start with k randomly generated states
  (population)
  
• A state is represented as a string over a finite
  alphabet (often a string of 0s and 1s)
  
• Evaluation function (fitness function) has
  higher values for better states.
  
• Produce the next generation of states by
  selection, crossover, and mutation
  

24
Genetic Algorithms

• Fitness function number of non-attacking pairs
  of queens (min 0, max 8 7/2 28)
  
• 24/(24232011) 31
• 23/(24232011) 29 etc

25
Genetic Algorithms
26
(No Transcript)
27
Local Search Continuous Space

• Problem continuous space successor functions
  return infinitely many states
• Solution 1 discretize the neighborhood of each
  state
• Solution 2 Use gradient functions
• gradients are vectors that give magnitude and
  direction of steepest slope (derivatives of the
  objective function)
• (Well avoid this issue for now)

28
Online Search

• Online algorithms must process data as it is
  received (rather than having all data before
  beginning)
• Online search is useful in
• (semi)dynamic environments where there is a
  penalty for sitting and thinking too long
• stocastic domains
• Online search is necessary for explorations
  problems where states and actions are unknown

29
Online Search

• Online search can only be performed by an agent
  executing actions, rather than a computational
  process
• agent cannot know successor states except by
  executing actions
• Agent knows
• actions(s) list of actions allowed in state s
• c(s,a,s) step cost function
• goal-test(s)

30
Robot Maze
31
Online Search Assumptions

• Agent can recognize states it has visited before
• avoids infinite loops
• Actions are deterministic
• Agent may have a heuristic functione.g. distance
  to goal
• Objective reach goal while minimizing cost
• Safely Explorable space goal can be reached
  from any reachable state (no dead ends)

32
Performance of Online Search

• Competitive Ratio total path cost actually
  traverse vs. cost agent would traverse if it knew
  state space in advance.
• best achievable CR is often unbounded (very
  high, possibly infinite)
• adversary argument
• Better to evaluate performance relative to size
  of state space, rather than depth of shallowest
  goal

33
Adversary Arguments

• Imagine an adversary that constructs state space
  as agent is moving through it and can adjust
  unexplored space to create worst case behavior.
• The space that would be created by this adversary
  is a space that would yield worst-case behavior.

34
Adversaries
agents view
adversarys view
35
Adversaries
36
(No Transcript)
37
Online Hill Climbing?

• Hill climbing is on-line, since it only keeps on
  current state
• Problem still exists local maxima
• Random restart is not possible

38
Random Walks

• Instead of random restart, we can consider random
  walks for online search
• select a possible action at random, give
  precedence to untried actions
• probability of success ? 1 for finite space

39
Quicksand
40
Online Search with Memory

• Store current best estimate of cost H(s) for each
  visited state.
• start with heuristic, update as agent gains
  experience
• H(s) ? c(s,a,s) H(s)

41
(No Transcript)
42
LRTA in one dimension