Learning Classifier Systems

1
Learning Classifier Systems

• Navigating the fitness landscape?
• Why use evolutionary computation?
• Whats the concept of LCS?
• Early pioneers
• Competitive vs Grouped Classifiers
• Beware the Swampy bits!
• Niching
• Selection for mating and effecting
• Balance exploration with exploitation.
• Balance the pressures
• Zeroth level classifier system
• The X-factor
• Alphabet soup
• New Slants - piecewise linear approximators
• Why don't LCS rule the world?
• Simplification schemes
• Cognitive Classifiers
• Neuroscience Inspirations
• Application Domains

2
Cognitive LCS
http//pages.cpsc.ucalgary.ca/jacob/Courses/Winte
r2003/CPSC601-73/Slides/18-ClassifierSystems.pdf
3
Cognitive LCS
Adaptation in Natural and Artificial Systems An
Introductory Analysis with Applications to
Biology, Control, and Artificial Intelligence
John H. Holland, Genetic algorithms are
playing an increasingly important role in studies
of complex adaptive systems, ranging from
adaptive agents in economic theory to the use of
machine learning techniques in the design of
complex devices such as aircraft turbines and
integrated circuits. Adaptation in Natural and
Artificial Systems is the book that initiated
this field of study, presenting the theoretical
foundations and exploring applications.The MIT
Press (April 29, 1992) ISBN 0262581116
4
Cognitive LCS
Induction Processes of Inference, Learning and
Discovery (Computational Models of Cognition and
Perception) John H. Holland, Keith J. Holyoak,
Richard E. Nisbett and Paul ThagardTwo
psychologists, a computer scientist, and a
philosopher have collaborated to present a
framework for understanding processes of
inductive reasoning and learning in organisms and
machines. Theirs is the first major effort to
bring the ideas of several disciplines to bear on
a subject that has been a topic of investigation
since the time of Socrates. The result is an
integrated account that treats problem solving
and induction in terms of rulebased mental
models. MIT Press (1 Jan 1986) ISBN
0262081601
5
Cognitive LCS

• Martin V. Butz 
• Department of Cognitive Psychology, University of
  Würzburg, Germany
• Abstract
• Rule-based evolutionary online learning systems,
  often referred to as Michigan style learning
  classifier systems (LCSs), were originally
  inspired by the general principles of Darwinian
  evolution and cognitive learning.
• In fact, when John Holland proposed the basic LCS
  framework, he actually referred to LCSs as
  cognitive systems (CSs).
• Inspired by stimulus-response principles in
  cognitive psychology, Holland designed CSs to
  evolve a set of production rules that convert
  given input into useful output.
• Temporary memory in the form of a message list
  was added to simulate inner mental states
  situating the system in the current environmental
  context.

6
Cognitive LCS

• Call for Papers for IWLCS 2007
• The Tenth International Workshop on Learning
  Classifier Systems (IWLCS 2007)
• Held in London, UK, July 7-8, 2007 during the
  Genetic and Evolutionary Computation Conference
  (GECCO-2007).
• Since Learning Classifier Systems (LCSs) were
  introduced by John H. Holland as a way of
  applying evolutionary computation to machine
  learning problems, the LCS paradigm has broadened
  greatly into a framework encompassing many
  representations, rule discovery mechanisms, and
  credit assignment schemes. Current LCS
  applications range from data mining, to automated
  innovation, and to the on-line control of
  cognitive systems. LCS is a very active area of
  research that encompasses various system
  approaches. Wilsons accuracy-based XCS system
  has received the highest attention and gained the
  highest reputation.

7
Cognitive Systems
Artificial Intelligence
Artificial Neural Networks
Control Theory and Operations Research
Psychology
Philosophy
Cognitive Systems
Cybernetics
Neuroscience
Sensorimotor Systems
Adapted from Sutton and Barto
8
Cybernetics

• First order
• (Weiner 1948)
• Inspired by control theory and dynamical systems
• Simple feedback control systems are the prime
  theoretical tool
• (e.g. thermostat controls room temperature)
• Second order
• (revival Port and van Gelder)
• Agent and environment constituting the
  meta-cybernetic system are inseparable and
  concerns itself with the results of their
  interaction
• (e.g. room affects thermostat)

9
Cognitive Systems definition

• Current robots are poor cognitive systems. Need
  to improve devices that we use every day and
  investigate medical benefits.
• "Cognitive systems are natural or artificial
  information processing systems, including those
  responsible for perception, learning, reasoning
  and decision-making and for communication and
  action"
• DTI Foresight initiative.

10
Environmental Interaction

• Perceive receive from the environment
• Represent Environment, agent,
• Reason about environment and self,
• Learn about environment and self,
• Action Act within the environment

11
Learning

• Learning The acquisition of new knowledge and
  skills, and their incorporation in future system
  activities, provided this acquisition and
  incorporation is conducted by the system itself
  and leads to an improvement in its performance.
• Learning is constructing or modifying
  representations of what is being experienced.
• Michalski et al. 86

12
Supervision

• 3 types of supervision Barto 90
• Supervised learning The environment contains a
  teacher that (directly or indirectly) provides
  the correct response for certain environmental
  states as a training signal for the learning
  signal.
• Unsupervised learning The learning system has an
  internally defined teacher with a prescribed goal
  that does not need utility feedback of any kind.
• Reinforcement learning The environment does not
  directly indicate what the correct response
  should have been. Instead, it only provides
  reward or punishment to indicate the utility of
  actions that were actually taken by the system.

13
System Response

• 3 types of overall system response
• Stimulus response The learning system responds
  immediately to the input with an output.
• (classification tasks, e.g. disease prognosis)
• Ultimate response The learning system may
  require more than one input before an output is
  reached.
• (temporal tasks, e.g. maze navigation)
• Continuous response The learning system responds
  with an output to each input in order to reach an
  ultimate goal.
• (temporal tasks, e.g. missile avoidance)

14
Consciousness

• Consciousness has been described as a general
  term consisting of 4 categories
• Phenomenal consciousness (P-consciousness)
• Access consciousness (A-consciousness)
• Self-consciousness (S-consciousness)
• Monitoring consciousness (M-consciousness)
• Block, N. (1995).
• On a confusion about a function of consciousness.
  
• Behavioral and Brain Sciences, 18, 227-247.

15
Anticipatory LCS
Anticipatory Learning Classifier Systems Martin
V. Butz , Describes the state of the art of
anticipatory learning classifier systems-adaptive
rule learning systems that autonomously build
anticipatory environmental models. An
anticipatory model specifies all possible
action-effects in an environment with respect to
given situations. It can be used to simulate
anticipatory adaptive behavior. Springer
January 25, 2002 ISBN 0792376307
16
Anticipatory LCS

• Anticipations influence cognitive systems and
  illustrates the use of anticipations for
• Faster reactivity
• Adaptive behavior beyond reinforcement
  learning
• Attentional mechanisms
• Simulation of other agents
• The implementation of a motivational module.
• A particular evolutionary model learning
  mechanism, a combination of
• a directed specializing mechanism and
• a genetic generalizing mechanism.
• Experiments show that anticipatory adaptive
  behavior can be simulated by exploiting the
  evolving anticipatory model for even faster model
  learning, planning applications, and adaptive
  behavior beyond reinforcement learning.

17
Anticipatory LCS
Anticipations added by inclusion of effect part
(E) The effects that the classifier believes
to be caused by the specified action The mark
(M) records the values of each attribute of all
situations in which the classifier did not
anticipate correctly sometimes The quality (q)
measures the accuracy of the anticipations A
reward prediction (r) and immediate reward (ir)
prediction used
18
Anticipatory LCS
Cl1 10 - 1 - 0101 Cl2 01110 - 1 -
Where condition is dont care Where
effect is pass through Cl3 - 0 -
staying still! The system compares the
anticipation of each classifier with the real
next situation the classifier is modified (and
if necessary a new classifier generated). If the
generated classifier already exists, the quality
of the old classifier is increased by the WH rule
19
Abstraction
ABSTRACTED RULES
Abstraction algorithm generates meta-rules
covering the discovered accurate rules
LCS LEARNT RULES
LCS generates accurate and general rules covering
states, together with a utility of the rules
ALL POSSIBLE STATES
20
Abstracted Rules
21
Abstracted LCS