10.3 Understanding Pattern Recognition Methods

1
10.3 Understanding Pattern Recognition Methods

• Chris Kramer

2
Pattern Recognition is ...

• Abstracting relevant information from game world
• Constructing concepts or models and deducing
  patterns for higher-level reasoning and
  decision-making systems.
• necessary especially when game world is not
  deterministic
• built-in randomness
• player actions.

3
Approaches to Understanding

• Functional
• Derived from the roles Pattern Recognition has in
  games.
• Methodological
• Applying computer science concepts to PR in games

4
Functional Approach

• Think about role of PR in an actual game
• Sports game read the match
• RTS identify threats strategize
• 1 on 1 react to opponent's favored moves
• 3 Aspects determine role of a PR system
• Level of decision making
• Stance toward player
• Use of modeled knowledge

5
Level of Decision Making

• 3 classic military levels Strategic, Tactical,
  Operational
• Strategic Lots of data and time Many
  speculative decisions cost of single poor
  decision is high
• Tactical Mediator between Strategic and
  Operational Accomplish strategic plan
  Coordinate groups of entities More real-time
• Operational Very concrete Many short-term,
  reactive decisions individual unit actions

6
Stance toward player

• Enemy provide a challenge be purposeful, if not
  intelligent
• Ally assist player communicate in an accessible
  format be consistent
• Neutral fairness especially in case of observer
  AI which governs camera movement in sports game

7
Use of Modeled Knowledge

• Generators and Symbols game world seen as a
  machine generating a series of different states
  or actions
• Symbols are the fundamental reasoning unit
  resulting from PR
• Sequence of symbols used in 2 ways Prediction
  and Production
• Prediction model player as generator and predict
  next symbol
• Production AI models self as generator and
  determines best symbol to execute

8
Methodological Approach

• Apply general computer science innovations to
  pattern recognition in games
• 3 aspects Optimization, Adaptation, Uncertainty
• Optimization mathematical maximization of some
  objective function
• Adaptation create model based on known results
  of previous models
• Uncertainty applying methods which account for
  uncertainty

9
Optimization

• 3 mathematical elements objective function,
  variables, constraints
• Techniques are iterative, time-consuming and
  usually offline
• Problem like any math function, there are local
  optimums in addition to the global optimum.
• Many approaches to find global optimum most
  focus on multiple traces of search space
• Genetic algorithms best when variables are
  independent
• Swarm algorithms iterators fly in the search
  space

10
Optimization Issues

• Usually multi-dimensional (many variables)
  impossible to visually represent
• Computational difficulty can be eased with
  heuristic weakening of optimality criteria
• Real-time usage must be linear heuristic search
  with few variables.

11
Optimization ExampleAge of Empires

• AoE RTS game with many varied units and teams
• Genetic algorithm used to balance units
• Set up mock battles, use results of battles to
  guide selection
• Objective function want both sides to have equal
  numbers of wins and losses

12
Adaptation

• Recognizing changing circumstances
• Best for indeterminate or unknown factors
• Methods include Reinforcement Learning,
  Influence Maps, Neural Networks

13
Uncertainty

• Soft Computing refers to methods that account for
  uncertainty probabalistic reasoning, fuzzy logic
• Fuzzy Sets items have partial or probabalistic
  membership in a set.
• Fuzziness can be incorporated with other methods
  to better deal with uncertainty Fuzzy genetic
  algorithm, Fuzzy Neural Nets

14
10.5 Getting Around the Limits of Machine Learning

• Method of Analysis 4 Questions
• Analysis of Machine Learning in 3 actual games
• Specific Limits to learning

15
Analysis Method

• 4 Questions
• 1. Cheap to recognize what to learn from?
• 2. Cheap to store the knowledge?
• 3. Cheap to use the knowledge?
• 4. Does game benefit from learning?
• Major divisions of the problem
• implicit vs. explicit
• online vs. offline

16
Black White

• Player dictates good or bad creature behavior
• Explicit Online learning
• Answer to all 4 Questions YES!
• 2 might have been problem in past
• Testing started early and was integral to project
• Innate programmed behaviors constrained learning

17
Command and Conquer Renegade

• Unimplimented feature if player goes between
  areas by an unkown path, AI will learn that path
• Online Implicit Learning
• 4 Questions yes
• Path is already processed, Storage of new
  pathnodes is negligible

18
Re-Volt

• Genetic algorithm used during development to tune
  parameters of car AIs
• Offline Implicit Learning
• Lap Times provided simple criteria for selection

19
Limits to Learning

• Context of first 3 questions
• Recognizing something to learn from
• More realism in games results in problems similar
  to machine learning in the real world
• Noisy Inputs Lots of data, Little useful data
• Time Dependencies
• Some algorithms to costly for online learning
• Use of knowledge can be expensive also

20
Good Knowledge Representation Helps Everywhere

• Preprocessing input signals and boiling them down
  to easier-to-use units
• High level of representation is easier to work
  with
• Beware loss of information
• MIDI vs. Raw audio

21
Seeing More Clearly

• Learning from the present modern games are good
  at this
• Learning from past Causal Chains
• Recognizing key precursors to learning event
• Keep detailed history to learn from
• Never been implemented

22
Seeing the Wrong Thing in theRight Examples

• Instance-based learning models often learn the
  wrong things
• Military System detect if a tank is present in
  an aerial photograph
• Performed well on training and test sets
• But very poorly on real data
• All training photos of tanks were on sunny days,
  and photos of nothing were on cloudy days
• Must have difficult data

23
Storing New Knowledge

• Find ways to incorporate new knowledge online
• Complex algorithms recalculation unfeasible
• Cannot lose performance against original data set
• Over-fitting
• Existing algorithm for neural nets Temporal
  Differences

24
Conclusion

• Question 4 is the most important!