CS B551: Elements of Artificial Intelligence

1
CS B551 Elements of Artificial Intelligence

• Instructor Kris Hauser
• http//cs.indiana.edu/hauserk

2
Recap

• Agent Frameworks
• Problem Solving with Search

3
Agenda

• Search problems
• Searching, data structures, and algorithms
• Breadth-first search
• Depth-first search
• Uniform-cost search

4
Search problems
5
Defining a Search Problem
S

• State space S
• Successor function x ? S ? SUCC(x) ? 2S
• Initial state s0
• Goal test
• x?S ? GOAL?(x) T or F
• Arc cost

6
State Graph

• Each state is represented by a distinct node
• An arc (or edge) connects a node s to a node s
  if s ? SUCC(s)
• The state graph may contain more than one
  connected component

7
8-Queens Problem

• State repr. 1
• Any placement of 0-8 queens
• State repr. 2
• Any non-conflicting placement of 0-8 queens

8
Representation 1

• State any placement of 0-8 queens
• Initial state 0 queens
• Successor function
• Place queen in empty square
• Goal test
• Non-conflicting placement of 8 queens
• of states 64x63xx57 3x1014

9
Representation 2

• State any placement of non-conflicting 0-8
  queens in columns starting from left
• Initial state 0 queens
• Successor function
• A queen placed in leftmost empty column such that
  it causes no conflicts
• Goal test
• Any state with 8 queens
• of states 2057

10
Path Planning
What is the state space?
11
Formulation 1
12
Optimal Solution
This path is the shortest in the discretized
state space, but not in the original continuous
space
13
Formulation 2
14
Formulation 2
Visibility graph
15
Solution Path
The shortest path in this state space is also the
shortest in the original continuous space
16
Example 8-Puzzle
2
8
1
2
3
3
4
7
4
5
6
1
5
6
7
8
Initial state
Goal state
State Any arrangement of 8 numbered tiles and an
empty tile on a 3x3 board
17
15-Puzzle

• Introduced (?) in 1878 by Sam Loyd, who dubbed
  himself Americas greatest puzzle-expert

18
15-Puzzle

• Sam Loyd offered 1,000 of his own money to the
  first person who would solve the following
  problem

19

• But no one ever won the prize !!

20
How big is the state space of the (n2-1)-puzzle?

• 8-puzzle ? ?? states

21
How big is the state space of the (n2-1)-puzzle?

• 8-puzzle ? 9! 362,880 states
• 15-puzzle ? 16! 2.09 x 1013 states
• 24-puzzle ? 25! 1025 states
• But only half of these states are reachable from
  any given state(but you may not know that in
  advance)

22
Permutation Inversions

• Wlg, let the goal be
• A tile j appears after a tile i if either j
  appears on the same row as i to the right of i,
  or on another row below the row of i.
• For every i 1, 2, ..., 15, let ni be the number
  of tiles j lt i that appear after tile i
  (permutation inversions)
• N n2 n3 ? n15 row number of empty tile

4
3
2
1
5
6
7
8
12
11
10
9
15
14
13
n2 0 n3 0 n4 0 n5 0 n6 0 n7 1 n8
1 n9 1 n10 4 n11 0 n12 0 n13 0 n14
0 n15 0
4
3
2
1
5
10
7
8
? N 7 4
12
11
6
9
15
14
13
23

• Proposition (N mod 2) is invariant under any
  legal move of the empty tile
• Proof
• Any horizontal move of the empty tile leaves N
  unchanged
• A vertical move of the empty tile changes N by an
  even increment (? 1 ? 1 ? 1 ? 1)

s
N(s) N(s) 3 1
24

• Proposition (N mod 2) is invariant under any
  legal move of the empty tile
• ? For a goal state g to be reachable from a state
  s, a necessary condition is that N(g) and N(s)
  have the same parity
• It can be shown that this is also a sufficient
  condition
• ? The state graph consists of two connected
  components of equal size

25
Searching the State Space

• It is often not feasible (or too expensive) to
  build a complete representation of the state
  graph

26
8-, 15-, 24-Puzzles
8-puzzle ? 362,880 states
15-puzzle ? 2.09 x 1013 states 24-puzzle ?
1025 states
0.036 sec
55 hours
gt 109 years
100 millions states/sec
27
Searching
28
Searching the State Space

• Often it is not feasible (or too expensive) to
  build a complete representation of the state
  graph
• A problem solver must construct a solution by
  exploring a small portion of the graph

29
Searching the State Space
Search tree
30
Searching the State Space
Search tree
31
Searching the State Space
Search tree
32
Searching the State Space
Search tree
33
Searching the State Space
Search tree
34
Searching the State Space
Search tree
35
Search Nodes and States
If states are allowed to be revisited,the search
tree may be infinite even when the state space is
finite
36
Data Structure of a Node
Depth of a node N length of path from
root to N (depth of the root 0)
37
Node expansion

• The expansion of a node N of the search tree
  consists of
• Evaluating the successor function on STATE(N)
• Generating a child of N for each state returned
  by the function
• node generation ? node expansion

N
38
Fringe of Search Tree

• The fringe is the set of all search nodes that
  havent been expanded yet

2
8
3
4
7
1
5
6
2
7
8
3
4
1
5
6
2
8
8
2
2
8
4
8
2
7
3
4
3
4
7
3
4
7
3
7
1
5
6
1
5
6
6
1
5
6
1
5
39
Is it identical to the set of leaves?
40
Search Strategy

• The fringe is the set of all search nodes that
  havent been expanded yet
• The fringe is implemented as a priority queue
  FRINGE
• INSERT(node,FRINGE)
• REMOVE(FRINGE)
• The ordering of the nodes in FRINGE defines the
  search strategy

41
Search Algorithm 1

• SEARCH1
• If GOAL?(initial-state) then return initial-state
• INSERT(initial-node,FRINGE)
• Repeat
• If empty(FRINGE) then return failure
• N ? REMOVE(FRINGE)
• s ? STATE(N)
• For every state s in SUCCESSORS(s)
• Create a new node N as a child of N
• If GOAL?(s) then return path or goal state
• INSERT(N,FRINGE)

Expansion of N
42
Performance Measures

• CompletenessA search algorithm is complete if it
  finds a solution whenever one existsWhat about
  the case when no solution exists?
• OptimalityA search algorithm is optimal if it
  returns a minimum-cost path whenever a solution
  exists
• ComplexityIt measures the time and amount of
  memory required by the algorithm

43
Blind Search Strategies
44
Blind Strategies

• Breadth-first
• Bidirectional
• Depth-first
• Depth-limited
• Iterative deepening
• Uniform-Cost(variant of breadth-first)

45
Breadth-First Strategy

• New nodes are inserted at the end of FRINGE

FRINGE (1)
46
Breadth-First Strategy

• New nodes are inserted at the end of FRINGE

FRINGE (2, 3)
47
Breadth-First Strategy

• New nodes are inserted at the end of FRINGE

FRINGE (3, 4, 5)
48
Breadth-First Strategy

• New nodes are inserted at the end of FRINGE

FRINGE (4, 5, 6, 7)
49
Important Parameters

• Maximum number of successors of any state?
  branching factor b of the search tree
• Minimal length (? cost) of a path between the
  initial and a goal state? depth d of the
  shallowest goal node in the search tree

50
Evaluation

• b branching factor
• d depth of shallowest goal node
• Breadth-first search is
• Complete? Not complete?
• Optimal? Not optimal?

51
Evaluation

• b branching factor
• d depth of shallowest goal node
• Breadth-first search is
• Complete
• Optimal if step cost is 1
• Number of nodes generated ???

52
Evaluation

• b branching factor
• d depth of shallowest goal node
• Breadth-first search is
• Complete
• Optimal if step cost is 1
• Number of nodes generated 1 b b2 bd
  ???

53
Evaluation

• b branching factor
• d depth of shallowest goal node
• Breadth-first search is
• Complete
• Optimal if step cost is 1
• Number of nodes generated 1 b b2 bd
  (bd1-1)/(b-1) O(bd)
• ? Time and space complexity is O(bd)

54
Time and Memory Requirements
Assumptions b 10 1,000,000 nodes/sec
100bytes/node
55
Time and Memory Requirements
Assumptions b 10 1,000,000 nodes/sec
100bytes/node
56
Remark

• If a problem has no solution, breadth-first may
  run for ever (if the state space is infinite or
  states can be revisited arbitrary many times)

57
Bidirectional Strategy
2 fringe queues FRINGE1 and FRINGE2
Time and space complexity is O(bd/2) ?? O(bd) if
both trees have the same branching factor b
Question What happens if the branching factor
is different in each direction?
58
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
59
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
60
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
61
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
62
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
63
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
64
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
65
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
66
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
67
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
68
Depth-First Strategy

• New nodes are inserted at the front of FRINGE

1
69
Evaluation

• b branching factor
• d depth of shallowest goal node
• m maximal depth of a leaf node
• Depth-first search is
• Complete?
• Optimal?

70
Evaluation

• b branching factor
• d depth of shallowest goal node
• m maximal depth of a leaf node
• Depth-first search is
• Complete only for finite search tree
• Not optimal
• Number of nodes generated (worst case) 1 b
  b2 bm O(bm)
• Time complexity is O(bm)
• Space complexity is O(bm) or O(m)
• Reminder Breadth-first requires O(bd) time and
  space

71
Depth-Limited Search

• Depth-first with depth cutoff k (depth at which
  nodes are not expanded)
• Three possible outcomes
• Solution
• Failure (no solution)
• Cutoff (no solution within cutoff)

72
Iterative Deepening Search

• Provides the best of both breadth-first and
  depth-first search
• Main idea

Totally horrifying !
IDS For k 0, 1, 2, do Perform
depth-first search with depth cutoff k (i.e.,
only generate nodes with depth ? k)
73
Iterative Deepening
74
Iterative Deepening
75
Iterative Deepening
76
Performance

• Iterative deepening search is
• Complete
• Optimal if step cost 1
• Time complexity is (d1)(1) db (d-1)b2
  (1) bd O(bd)
• Space complexity is O(bd) or O(d)

77
Calculation

• db (d-1)b2 (1) bd
• bd 2bd-1 3bd-2 db
• (1 2b-1 3b-2 db-d)?bd
• ? (Si1,,? ib(1-i))?bd bd (b/(b-1))2

78
Number of Generated Nodes (Breadth-First
Iterative Deepening)

• d 5 and b 2

120/63 2
79
Number of Generated Nodes (Breadth-First
Iterative Deepening)

• d 5 and b 10

123,456/111,111 1.111
80
Recap

• BFS Complete, optimal
• O(bd) time and space
• DFS Not complete nor optimal
• O(bd) space, unbounded time
• ID Complete, optimal
• O(bd) space, O(bd) time

81
Homework

• Readings RN Ch. 3.4-3.5
• HW1 due on 9/22