Problem Solving Using Search

1
Problem Solving Using Search

• Reduce a problem to one of searching a graph.
• View problem solving as a process of moving
  through a sequence of problem states to reach a
  goal state
• Move from one state to another by taking an
  action
• A sequence of actions and states leading to a
  goal state is a solution to the problem.

2
(No Transcript)
3
Trees

• A tree is made up of nodes and links connected so
  that there are no loops (cycles).
• Nodes are sometimes called vertices.
• Links are sometimes called edges.
• A tree has a root node.
• Where the tree starts.
• Every node except the root has a single parent
  (aka direct ancestor).
• An ancestor node is a node that can be reached by
  repeatedly going to a parent.
• Each node (except a terminal, aka leaf) has one
  or more children (aka direct descendants).
• A descendant node is a node that can be reached
  by repeatedly going to a child.

4
(No Transcript)
5
Graphs

• Set of nodes connected by links.
• But, unlike trees, loops are allowed.
• Also, unlike trees, multiple parents are allowed.
• Two kinds of graphs
• Directed graphs.
• Links have a direction.
• Undirected graphs.
• Links have no direction.
• A tree is a special case of a graph.

6
(No Transcript)
7
Representing Problems with Graphs

• Nodes represent cities that are connected by
  direct flight.
• Find route from city A to city B that involves
  the fewest hops.
• Nodes represent a state of the world.
• Which blocks are on top of what in a blocks
  scene.
• The links represent actions that result in a
  change from one state to the other.
• A path through the graph represents a plan of
  action.
• A sequence of steps that tell how to get from an
  initial state to a goal state.

8
(No Transcript)
9
Problem Solving with Graphs

• Assume that each state is complete.
• Represents all (and preferably only) relevant
  aspects of the problem to be solved.
• In the flight planning problem, the identity of
  the airport is sufficient.
• But the address of the airport is not necessary.
• Assume that actions are deterministic.
• We know exactly the state after an action has
  been taken.
• Assume that actions are discrete.
• We dont have to represent what happens while the
  action is happening.
• We assume that a flight gets us to the scheduled
  destination without caring what happens during
  the flight.

10
(No Transcript)
11
(No Transcript)
12
Classes of Search

• Uninformed, Any-path
• Depth-first
• Breadth-first
• In general, look at all nodes in a search tree in
  a specific order independent of the goal.
• Stop when the first path to a goal state is
  found.
• Informed, Any-path
• Exploit a task specific measure of goodness to
  try to reach a goal state more quickly.

13
Classes of Search

• Uninformed, optimal
• Guaranteed to find the best path
• As measured by the sum of weights on the graph
  edges
• Does not use any information beyond what is in
  the graph definition
• Informed, optimal
• Guaranteed to find the best path
• Exploit heuristic (rule of thumb) information
  to find the path faster than uninformed methods

14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Scoring Function

• Assigns a numerical value to a board position
• The set of pieces and their locations represents
  a singel state in the game
• Represents the likelihood of winning from a given
  board position
• Typical scoring function is linear
• A weighted sum of features of the board position
• Each feature is a number that measures a specific
  characteristic of the position.
• Material is some measure of which pieces one
  has in a given position.
• A number that represents the distribution of the
  pieces in a position.

19
(No Transcript)
20
(No Transcript)
21
Scoring Function

• To determine next move
• Compute score for all possible next positions.
• Select the one with the highest score.
• If we had a perfect evaluation function, playing
  chess would be easy!
• Such a function exists in principle
• But, nobody knows how to write it or compute it
  directly.

22
Combinatorial Optimization

• Combinatorial optimization and enumeration
  problems are modeled by state spaces that usually
  lack any regular structure.
• Combinatorial optimization algorithms solve
  instances of problems that are believed to be
  hard in general, by exploring the usually-large
  discreet solution space of these instances.
• Goal is to find the best possible solution in
  the state space.
• Exhaustive search is often the only way to handle
  such combinatorial chaos
• Many real problems exhibit no regular
  structures to be exploited, and that leaves
  exhaustive enumeration as the only approach in
  sight.
• Combinatorial optimization algorithms try to
  reduce the effective size of the space, and
  explore the space efficiently.

23
Branch and Bound Algorithms

• Branch and bound is a general algorithmic method
  for finding optimal solutions of problems in
  combinatorial optimization.
• Solution space is discrete.
• The general idea
• Find the minimal value of a function f(x) over a
  set of admissible values of the argument x called
  feasible region.
• Both f and x may be of arbitrary nature.

24
Branch and Bound Algorithms

• A branch-and-bound procedure requires two tools.
• First, a smart way of covering the feasible
  region by several smaller feasible subregions.
• This is called branching, since the procedure is
  repeated recursively to each of the subregions
  and all produced subregions naturally form a tree
  structure.
• Tree structure is called search tree or
  branch-and-bound-tree.
• Its nodes are the constructed subregions.

25
Branch and Bound Algorithms

• Second tool is bounding
• a fast way of finding upper and lower bounds for
  the optimal solution within a feasible subregion.
  

26
Basic Approach

• Based on a simple observation (for a minimization
  task)
• If the lower bound for a subregion A from the
  search tree is greater than the upper bound for
  any other (previously examined) subregion B, then
  A may be safely discarded from the search.
• This step is called pruning.
• It is usually implemented by maintaining a global
  variable m that records the minimum upper bound
  seen among all subregions examined so far
• any node whose lower bound is greater than m can
  be discarded.

27
Basic Approach

• When the upper bound equals the lower bound for a
  given node, the node is said to be solved.
• Ideally, the algorithm stops when all nodes have
  either been solved or pruned.
• In practice the procedure is often terminated
  after a given time
• at that point, the minimum lower bound and the
  maximum upper bound, among all non-pruned
  sections, define a range of values that contains
  the global minimum.

28
Basic Approach

• The efficiency of the method depends critically
  on the effectiveness of the branching and
  bounding algorithms used.
• Bad choices could lead to repeated branching,
  without any pruning, until the sub-regions become
  very small.
• In that case the method would be reduced to an
  exhaustive enumeration of the domain, which is
  often impractically large.
• There is no universal bounding algorithm that
  works for all problems.
• Little hope that one will ever be found.
• General paradigm needs to be implemented
  separately for each application.
• Branching and bounding algorithms are specially
  designed for it.

29
Min-Max Algorithm

• Limited look-ahead plus scoring
• I look ahead two moves (2-ply)
• First me relative level 1
• Then you relative level 2
• For each group of children at level 2
• Check to see which has the minimum score
• Assign that number to the parent
• Represents the worst that can happen to me after
  your move from that parent position
• I pick the move that lands me in the position
  where you can do the least damage to me.
• This is the position which has the maximum value
  resulting from applying Step 1.
• Can implement this to any number (depth) of
  min-max level pairs.

30
(No Transcript)
31
May 1997Used Min-Max.256 specialized chess
processors coupled into a 32 node supercomputer.
Examined around 30 billion moves per minute.
Typical search depth was 13ply - but in some
dynamic situations it could go as deep as 30.
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
Alpha-Beta Pruning

• Pure optimization of min-max.
• No tradeoffs or approximations.
• Dont examine more states than is necessary.
• Cutoff moves allow us to cut off entire
  branches of the search tree (see following
  example)
• Only 3 states need to be examined in the
  following example
• Turns out, in general, to be very effective

36
(No Transcript)
37
(No Transcript)
38
Move Generation

• Assumption of ordered tree is optimistic.
• Ordered means to have the best move on the left
  in any set of child nodes.
• Node with lowest value for a min node.
• Node with highest value for a max node.
• If we could order nodes perfectly, we would not
  need alpha-beta search!
• The good news is that in practice performance is
  close to optimistic limit.

39
(No Transcript)
40
Move Generator

• Goal is to produce ordered moves
• Needed to take advantage of alpha-beta search.
• Encodes a fair bit of knowledge about a game.
• Example order heuristic
• Value of captured piece value of attacker.
• E.g., pawn takes Queen is the highest ranked
  move in this ordering

41
Static Evaluation

• Other place where substantial game knowledge is
  encoded
• In early programs, evaluation functions were
  complicated and buggy
• In time it was discovered that you could get
  better results by
• A simple reliable evaluator
• E.g., a weighted count of pieces on the board.
• Plus deeper search

42
Static Evalution

• Deep Blue used static evaluation functions of
  medium complexity
• Implemented in hardware
• Cheap PC programs rely on quite complex
  evaluation functions.
• Cant search as deeply as Big Blue
• In general there is a tradeoff between
• Complexity of evaluation function
• Depth of search.

43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
Time of Defeat August 1994Read all about it
athttp//www.cs.ualberta.ca/chinook/
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
TD-Gammon

• Neural network that is able to teach itself to
  play backgammon solely by playing against itself
  and learning from the results
• Based on the TD(Lambda) reinforcment learning
  algorithm
• Starts from random initial weights (and hence
  random initial strategy)
• With zero knowledge built in at the start of
  learning (i.e. given only a "raw" description of
  the board state), the network learns to play at a
  strong intermediate level
• When a set of hand crafted features is added to
  the network's input representation, the result is
  a truly staggering level of performance
• The latest version of TD Gammon is now estimated
  to play at a strong master level that is
  extremely close to the world's best human
  players.

51
(No Transcript)
52
The Match

• Exhibition match played in 1998.
• 100 games were played over 3 days to reduce any
  element of luck.
• Final result was narrow 8 point win for Davis.
• Davis and Tesauro conclude performance was
  superhuman.
• TDGammon made only one serious mistake in 100
  games!

53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
Murakami defeated 6 games to 0 in 1997.
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
The name Proverb comes from "probabilistic
cruciverbalist," meaning a crossword solver based
on probability theory.
62
Proverb

• Will Shortz, New York Times crossword puzzle
  editor, first issued his challenge to computer
  designers in 1997 after Deep Blue beat Gary
  Kasparov.
• Shortz contended that crossword puzzles, unlike
  chess, draw on particular human skills and
  thought processes that can be inaccessible to
  computing machines.

63
Proverb
"Is the computer going to be able to solve the
clues involving puns and wordplay? I don't think
so," Shortz wrote in an introduction to a volume
of The New York Times daily crossword puzzles. He
gave examples of clues he felt computers would
miss, such as "Event That Produces Big Bucks"
referring to "Rodeo," "Pumpkin-colored"
translating as "Orange," or "It Might Have
Quarters Downtown" meaning "Meter."
64
Proverb
"So if you were one who lamented the loss of the
human to the computer in chess, don't despair.
In a much more wide ranging and, frankly, complex
game like crosswords, we humans still rate just
fine."
65
And then again
There was a lot of discussion at the 1999
tournament (American Crossword Puzzle Tournament)
of computer solutions to the contest puzzles. Two
weeks before the event I had sent advance copies
of them to Michael Littman at Duke University.
Michael heads a team of computer scientists that
has developed a program called Proverb--the
world's first computer program designed to solve
standard crosswords. He immediately put Proverb
through its paces. The results were so
interesting (in fact, so amazing) that I printed
them out on large sheets of paper and posted
them, along with Michael's analysis, after each
round at the event. --Will Shortz
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
Wanna Bet?
(December 1996)In Zia's book, Bridge, My Way,
which appeared a few years ago, he offered to
take a one-million-pound bet that no computer
would be able to beat him at the bridge table.
The stunt seemed to work in that it produced a
lot of publicity for his book. That is, until
last month when word reached him that bridge
program GIB, brainchild of American professor
Ginsberg, proved capable of incredible feats of
declarer play.
71
The Bet is Off!
(December 1996)Zia is the big star of the Fall
Nationals, having just triumphed in the premier
event, the Reisinger. Smiling from ear to ear, he
accepts the congratulations of his predominantly
female admirers. Then he is accosted by a man
he's never seen before. "Mr. Mahmood, my
congratulations and incidentally, may I ask you
something?" "But, of course," replies the always
amiable Pakistani, "what's it about?" "It
concerns a one-million-pound bet." The Pakistani
grows pale. "What is your name, sir?," he
immediately asks. "Matthew Ginsberg," says the
man. Suddenly there's little left of the great
Zia with his aura of invincibility. He cringes,
and mumbles, "The bet is off!," and walks out of
the room.
72
And then
Two years later GIB became World champion
computer bridge, and defeated the vast majority
of the world-top bridge players (including Zia
Mahmood) participating in the 1998 Par Contest.
However, such a par contest measures technical
bridge analysis skills only, and in 1999 Zia did
beat various computer programs including GIB in
an individual round robin match. But the story
is only beginning