Artificial Intelligence in Game Design

1
Artificial Intelligence in Game Design

• Heuristics and Other Ideas in Board Games

2
Good and Bad Heuristics

• Heuristic for evaluating board must be accurate
• Directly related to likelihood of win
• Inversely related to distance from win
• Example TicTacToe heuristicH(board) 2 of
  possible rows/columns/diagonals where X could win
  in one move 1 of possible
  rows/columns/diagonals where X could win in two
  moves - 2 of possible
  rows/columns/diagonals where O could win in one
  move - 1 of possible
  rows/columns/diagonals where O could win in two
  moves
• Gives these two boards same measure

Guaranteed loss for AI
X
O X
O X O
O X
X
O X O
-2
-2
3
Good and Bad Heuristics

• Better heuristic must take this into account!
• H(board)
• if my move next and I have 2 in row ? MAXINT
• if opponent move next and they have 2 in row ?
  -MAXINT
• if my move and opponent has gt 1 instance of 2 in
  row ? -MAXINT
• if opponent move and I have gt 1 instance of 2 in
  row ? MAXINT
• else (above function)
• Will work better, but more complex to compute!

Speed to computeheuristic value
Accuracy ofheuristic value
tradeoff
4
Linear Heuristic Functions

• Heuristic is some function of individual pieces
  on board
• Usually weighted in some way
• Overall board value S H(piecei)
  i
• Very fast to compute
• Example Chess
• Pieces have standard values
• Heuristic sum of AI pieces
  sum of player pieces

9 5 3 3 1
5
Linear Heuristic Functions

• Often based on position of pieces on board
• Examples
• Games where purpose is to move all pieces to
  some goal
• Backgammon
• Sorry
• Can often send other player back to start
• Heuristic value total distance of AI pieces
  from goal total
  distance of player pieces from goal

6
Linear Heuristic Functions

• Reversi (Othello)
• Board value of based entirely on piece positions
• Corners very valuable (cant be flipped)
• Sides somewhat valuable (very difficult to flip)
• Middle little value (will easily be flipped)
• H(board) C1 number of pieces in corner
  C2 number of pieces on side
  C3 number of pieces in middle
  C1 gtgt C2 gtgt C3
• Reversi easiest type of game for AI
• Low branching factor (5 to 15 possible moves)
• Good heuristics
• Single move can greatly change board
• Hard for human player to see
• Easy for MinMax lookahead to see

7
Nonlinear Heuristics

• Based on relationships between pieces on board
• Simple example
• Prefer chess pieces to protect one another
• Piece value piece value 1.5 if protected by
  another
• Piece value piece value 0.5 if not protected
  and threatened by opponent piece

8
Nonlinear Heuristics

• Drawback Usually much more expensive to compute
• n pieces ? O (n2) relationships between them
• May be able to explore more moves with simpler
  (linear) heuristic
• Example
• Simple linear heuristic takes k ms to evaluate
  per board
• Complex nonlinear heuristic takes 400k ms to
  evaluate per board
• Average of 20 possible next moves per board
• 400 possible next two moves
• On average, could explore two additional moves if
  use linear heuristic

9
Linear vs. Nonlinear Heuristics

• Nonlinear heuristics often detect future changes
  in board
• Pieces threatened by others might be captured
• Pieces protected by others less likely to be
  captured
• Often see effect of nonlinear heuristic with
  additional levels of linear heuristic
• Example knight fork in chess
• Can look for this using nonlinear heuristic

10
Linear vs. Nonlinear Heuristics

• Searching 2 more moves in game tree will show
  result of knight fork
• Shows state where rook now gone
• Higher value for linear heuristic based on piece
  values
• MinMax then gives knight fork state a high value

Board with knight fork
High value h
Black moves king
Min level
Board with knight threatening rook
Other moves give checkmate (value MAXINT)
High value h
Knight takes rook
Max level
Board with blackrook gone
High value h
11
Horizon Effect

• Major weakness of purely linear heuristics based
  on piece values
• Throwing good money after bad
• May need to recognize when move does not improve
  board position
• Chess example

Queen pinned by bishops
Could move rook in way
But rook captured and will still lose queen
12
Horizon Effect
Board with queen pinned
Move rook in front
Queen takes bishop
Bishop takes queen
Bishop takes rook
Board with queen lost
Board with rook lost
White down 6 points
White down 5 points
Queen takes bishop
This looks worse at cutoff level, but is actually
best in long run!
Bishop takes queen
Board with queen and rook lost
White now down 11 points!
13
Trappy Minmax

• Idea Use minmax to set traps for player
• Branch appears to have short-term benefit for
  player
• In long run, branch has benefit for AI

Good chance player might choose this branch
AI move
player move
This state looks ok to player
This state looks good to player
So worth considering making this move
Ok result for AI
Great result for AI
14
Trappy Minmax
Takes advantage of player horizon effect
Queen cancapture rook
Some other move
Queen takes rook
Board with rook ignored
Board with rook captured
White unchanged
White up 5 points
Knight captures queen
Board with queen lost
Player did not lookahead far enough to see this
White now down 4 points!
15
Factors in Trappy Minmax

• Trappiness Estimate of how likely player will
  choose branch corresponding
  to trap
• Usually computed based on median or maximum of
  player heuristics down best branch

Possible player move
5
9
Path player thinks game will follow if they make
this move
Median 7
6
8
-7
2
16
Factors in Trappy Minmax

• Profitability Score if player follows trap
  branch score if player
  follows their best branch
• Trap Quality Trappiness Profitability
• Use trap quality to adjust heuristic measure of a
  stateTrappy_heuristic(state)
  normal_heuristic(state)
  f(trap_quality(state))

17
Data-Driven Approaches

• Basing actions on known strategies rather than
  tree search
• Opening books of initial moves (Chess)
• Often 20-30 moves at start of Grandmaster match
• Each book consists of
• List of moves
• Evaluation of final outcome
• Should we follow this strategy
• Allows faster processing
• No need to search game tree until end of
  sequence
• Can just use evaluation as heuristic

Currentboard
Boards in sequence
End of sequence
Now start branching
18
Opening Books

• Choose as own opening strategy
• Must have good final evaluation!
• Make moves according to script
• If opponent follows script, keep following
• If opponent leaves script, start MinMax
• Will probably be in same way that benefits us
• Must also recognize when opponent uses an opening
  book
• Keep database of moves in opening books
• Match current board to those in database to find
  whether it is part of a sequence
• If so, make decision about whether following
  script is good idea

19
Other Set Plays

• End Games
• Many games have different strategies when few
  pieces left
• Forcing checkmate in chess
• Kings vs. kings in checkers
• Getting last pieces home in backgammon
• Recognize based on pieces left
• Follow set strategies
• Set Evaluation Values
• No heuristic evaluation of board instead,
  match board to database to get evaluation
• Works best if can match subboards
• Example Edge configurations in Othello
• This edge has a known value

20
Alternative Approaches

• Go
• 19 ? 19 board ? branching factor of 361
• Impossible for MinMax
• Only known approaches based on template matching
• Look in local area for configurationsthat match
  known strategies
• Still very open problem
• Best AI for Go only plays atamateur level