An Agent that plays Pacman

1
An Agent that plays Pacman

• Abbas Mehrabian, Arian Khosravi
• Computer Engineering Department
• Sharif University of Technology
• mehrabian, ar_khosravi_at_ce.sharif.edu
• 3 December 2007

2
About the game of Pacman

• The program gets its input from the screen!
• This decreases speed by 50.

3
Glossary

• Pacman
• Ghost
• Edible
• Pill
• Power Pill
• Snack
• Walls and Tunnels

4
Game objectives

• Escape from the ghosts.
• Eat all of the pills to finish the level.
• When hunting, eat the maximum number of edibles.
• Eat the power pills at good times, i.e. when
  there are lots of ghosts around.

5
Modeling the game gametable

• Gametable a 31x31 table that models the game
  state.
• Distance can be defined in the gametable.

6
Modeling the game gameobjects

• Class GameObject position in
  gametable direction distance to other
  GameObjects
• Pacman, Ghost, Edible, Snack are subclasses of
  GameObject.
• Definition A connected set is a set of
  pixelswith the same color.
• Every connected set may be the graphicalrepresent
  ation of some gameobject.

7
Program outline
Image Processing Phase

• while true capture the screen extract all
  connected sets for each connected set cs
  find a gameObject obj such that cs is the
  representation of obj update
  position of obj in gametable
• among the four possible positions to go
  find the position bestNext such that
  evaluate (bestNext) is maximum
• press the key to go to bestNext
• Note Since this is an infinite loop, it consumes
  CPU too much!

Decision Making Phase
8
Locality of decision making

• Agent makes decision only at this moment to go to
  one of the neighbors.
• This looks greedy, why dont we predict future
  and devise long-term plans, or use min-max
  algorithm?
• The ghosts are very hard to predict.
• The number of possible future game states is
  large, and it will take a lot of time to
  calculate deeper. Recall that this is a real-time
  game!
• Greedy is simple to implement, and it worked!

9
How to evaluate a point?

• evaluate (point n) Compute parameters
  p1,p2,,pm for point n return c1p1 c2p2
  cmpm
• Some of the parameters are
• Distance of n to the nearest pill
• Distance of n to the nearest ghost
• Distance of n to the nearest power pill
• The coefficients c1,c2, , cm are important and
  should be optimized!
• When a coefficient is large, it means that the
  corresponding parameter is important.

10
How to meet game objectives?

• Escape from the ghosts.
• Eat all of the pills to finish the level.
• When hunting, eat the maximum number of edibles.
• Eat the power pills at good times, i.e. when
  there are lots of ghosts around.
• In order to achieve each of these objectives,
  some parameters should be defined.
• Today I will talk about just the first objective,
  since it is the most important and challenging!

11
Escape parameters - 1

• The most obvious parameter is distance to the
  nearest ghost. Try to maximize it.
• But, this is not perfect! Look at this figure
• Obviously, it is better if Pacman goes down, but
  this parameter says that it should go up!
• Conclusion We should find more visionary
  parameters!

12
Escape parameters - 2

• Definition Location f is called available from
  point p, if Pacman can start from p and reach f
  before any ghost.
• Definition Safeness of a point is the number of
  available locationsfrom that point.
• If a point has more safeness, we are more likely
  to go there.

13
Escape parameters - 3

• The ghosts rarely reverse their direction, so we
  can use this property to predict them a little.
• Every ghost, spreads its energy throughout the
  gametable. Most of the energy goes forward.
• Definition Energy of a point is the summation
  of the ghosts energies at that point.
• The less energy a point has, the more likely we
  are to go there.

14
The relative effect of parameters

• The largest a coefficient ci is, the more effect
  the parameter pi would have on our move.
• But we dont want a certain parameter to have a
  constant importance in all situations.
• For example, escaping parameters should affect
  more when the situation is red!
• Conclusion The coefficients should not be
  constant during all phases of the game
• First, we evaluate our current position, and get
  a feeling about the situation.
• Then we set the coefficients.

15
Implementation

• Several situations are defined. In each
  situation, certain coefficients should be larger
• (before making decision)if a ghost is near or
  safeness is low then //Situation is
  danger c1 else endif(now, make
  decision)

16
The first version program

• Our first version, finished on 14 Sep. 2007, has
  4 different situations and 14 parameters!
• It gets 17300 points on average on my PC.
• It has the best official record of an agent that
  plays this version of Pacman

17
Current issues and future work

• We have made the game discrete, but this is
  sometimes very bad
• Devise long-term plans.
• There are at least 56 coefficients in the
  program, which we have tuned our self.
• Use learning algorithms to tune the parameters.
• In every learning algorithm, there should be some
  way to evaluate the current playing strategy.
  Here, the strategy is represented by the
  coefficients.

18
Log-based learning

• There is another program (called the teacher)
  that improves the agents ability over time.
• We assume that there is a human master who plays
  the game very well.
• First, the master plays. At every moment teacher
  saves the parameters and masters move in a log
  file.
• Second, teacher reads the log file, and at every
  moment the agent is provided with the parameters.
  Agents move is logged, and is compared with
  masters move.
• The more the agent plays like the master, better
  the strategy is.
• Teacher then tries to change the agents
  coefficients to arrive at the optimal strategy.

19
References

• Simon Lucas, Ms Pac-Man Competition CEC 2007
  Results, University of Essex, Nov. 2007, Nov.
  2007, http//cswww.essex.ac.uk/staff/sml/pacman/CE
  C2007Results.html.
• Any Questions?