David Lamb Introductory Presentation

1
David LambIntroductory Presentation

• D.Lamb_at_2005.ljmu.ac.uk
• Room 608, ext 2280
• http//www.staff.ljmu.ac.uk/cmpdlamb

2
Introduction

• Overview of research to date
• Cognitive Immunity Subsystems
• Detection Machine Learning techniques
• Danger Theory
• Novelty Detection
• Chance Discovery
• Anticipatory Learning Classifier Systems
• The Research Problem
• Planned Future Research

3
Cognitive Immunity

• A Cognitive Immune system should
• Identify problems
• both vulnerabilities and newly-introduced
  problems
• Diagnose the cause and severity of problems
• Then, ideally, restore the system to correct
  functionality

4
Cognitive Immunity Subsystems

• A proposed layered model of CI subsystems
• Detection detects events in environment
• Diagnosis maps/combines events to form
  situations
• Planning plans actions to resolve a situation
• Enactment assesses impact of actions, and
  performs them
• Learning/Evolution evolves the system based on
  feedback from environment
• Self-Organisation re-organises the system to
  avoid vulnerabilities

5
Current Research

• Research thus far has been in the area of
  Cognitive Immunity
• Detection and Diagnosis subsystems
• Tried to concentrate on mechanisms that can
  provide services suitable for Detection
• Also looked at existing cognitive software /
  system models to gain insight into the design of
  this type of software
• The following slides aim to present an overview
  of this research

6
Novelty Detection Introduction

• Novelty Detection systems are concerned with
• Detecting the data in a given set of inputs that
  may be considered abnormal or novel.
• Effectively generalising the known type i.e.
  not just a pattern match!
• Useful to Immune Systems as a detector a known
  vs. unknown discriminator, allowing a response to
  unknown data or signals.

7
Novelty Detection Statistics

• Traditional Mathematical / Statistical approaches
  can determine novelty by
• Plotting all known data, according to its
  defining attributes, in an n-dimensional space.
• Identifying clusters of plots as known types or
  classes
• Identifying plots outside of these clusters as
  novel, abnormal, or unrecognised
• This approach is complicated by
• High-dimensional data
• Outliers (standalone plots outside a cluster)
  both legitimate and those as a result of noisy
  data
• Quality of known data samples

8
Novelty Detection Neural Networks 1

• Trained Neural Networks can be used as Novelty
  Detectors
• Training a Neural Network an overview
• Standard Multi-Layer Perceptron Neural Networks
  can be trained to produce certain outputs for
  certain inputs.
• This is achieved by repeatedly presenting the
  network with sample input data, and appropriate
  target output data.
• After many training cycles, the network will
  reproduce the target output data for the
  specified inputs
• If the network inputs described the data well and
  the training data varied sufficiently, the
  network should perform well on data similar to
  the training data

9
Novelty Detection Neural Networks 2

• This allows MLP networks to behave as data
  classifiers
• Training data is comprised of samples of known
  types and suitable output class indication
• A high signal on a particular network output
  indicates a particular class/type has been
  presented as input
• Confidence scores can be added to quantify the
  confidence in any given classification
• However, using classification networks for
  novelty detection poses the following problems
• Accurate classification clearly depends on good
  quality training data
• Expensive to retrain to recognise additional
  classes - typically a full retraining from
  scratch is required
• May result in confusion at outputs when presented
  with truly novel data

10
Novelty Detection SO(F)Ms

• Self Organising (Feature) Maps
• A special type of neural network that undergoes
  unsupervised training
• i.e. the network is trained solely on input data,
  and doesnt require additional target output data
• Can easily derive classes (clusters) of data
  based on the variety in the input set
• SOM visualisations are particularly appropriate
  for presenting high dimensional data in a 2D map
  output

11
Novelty Detection Other Approaches

• Detectors / Selection approaches
• Known data is coded, typically as binary strings
• A set of random detectors are created as strings
• The random detectors are tested on the known/self
  data
• Those that match (against self) are eliminated
• Evolutionary approaches (genetic algorithms)
• Rules to match known/unknown are coded in strings
• Generations are evolved based on fitness of
  previous parents and modification via evolution
  operators
• Mutation one or more bits are changed
• Combination/Crossover x bits from one parent, y
  bits from other parent

12
Novelty Detection and Classification

• In addition to differentiating between known and
  unknown, several of the proposed novelty
  detectors can also serve as advanced classifiers
• Classification can also prove useful to the
  Detection layer of an Immune System
• An ideal data classifier should
• Generalise classes (or types) of inputs
• Operate at a higher, more abstract level than a
  simple pattern match
• (i.e. not just X Y, but X is similar to Y)

13
The Danger Theory Introduction

• Originated in biological immunology
• Changes emphasis of response to a specified
  Danger Signal, rather than reacting purely to
  non-self
• Can provide a localised response (within the
  Danger Zone)
• Simple, (mostly) independent interactions
  repeated on a large scale produce the desired
  immune response

14
The Danger Theory Biological Model

• Response to Danger Signal (in the illustrated
  case, cell damage) triggers antibody reaction
  within the Danger Zone
• Matching antibodies are then duplicated to
  facilitate more antigen matching

15
The Danger Theory in Software

• Using the Danger Theory model in software
  presents some problems
• Representation of Danger Signal(s)
• Representation of spatial Danger Zone
• How to implement antibody / antigen recognition
• How to implement antibody suppression of antigens

16
Chance Discovery Introduction

• New-ish field, some disagreement on definition
  and application
• Some argue it is simply a variation on existing
  data mining themes
• Broad Definition, Discovers valuable chance
  events those that are rare, but important

17
Chance Discovery Overview

• A Chance Discovery system must be able to perform
  two main tasks

• Identification/Prediction of Chance Events
• Identification/Prediction of Consequences
• Identifying consequences e.g. associate cause
  with effect, based on system history. Find the
  value (or cost) of the cause.
• Prediction of consequences where history is not
  available or inappropriate, prediction with
  bounded accuracy

18
Chance Discovery Current Systems

• Despite the fact that the field is quite new,
  some prototype/research CD systems exist
• Key Graphs
• A method initially created to index documents
• Clusters co-occurrences of terms in a document
• These clusters should indicate topics
• Index terms are then chosen based on their
  relationship to other clusters
• Chance events (i.e. index terms) are chosen based
  on their links to significant high-frequency
  events (term clusters).

19
Chance Discovery Current Systems

• Knowledge Base (Change-based CD)
• World knowledge is modelled as rules in a KB
• Chance discoveries are made as this knowledge is
  changed, based on the fact that changes may
• Enable/disable some goal(s)
• Alter the cost/reward of achieving some goal(s)
• Dialogue approach
• Dialogue facilitates communication between
  separate knowledge bases
• Can be viewed as a distributed extension of the
  KB approach, deals with separate (and possibly
  differing) KBs

20
(Anticipatory) Learning Classifier Systems

• ALCS Currently research in progress!
• ALCS are cognitive systems that form
  anticipations about future events based on
  current behaviour and observations
• They are of interest, as they may represent a
  significant building block towards a CI system
  model

21
ALCS Components

• Two essential ALCS components
• ALP Anticipatory Learning Process compares
  anticipations with actual results, resulting in
  specialised rules that describe the observed
  behaviour.
• Genetic Generalisation Mechanism Generalises
  accurate rules from the over-specified ALP
  output, making the model more compact

22
The Research Problem

• How to create a reusable model for Cognitive
  Immunity, and to find components suitable for use
  in that model
• How to apply that model to a real-world example
• How to implement a complete system, using the
  proposed subsystems for a real-world problem

23
Planned Research

• How do I get from where I am now to where I want
  to be?
• Which significant areas of research must be
  covered?
• A continuation of ALCS research, plus
• Research into other types of Cognitive Systems
  and Artificial Immune Systems to understand the
  various ways of modelling these systems
• Research into more components or services
  suitable for the identified CI subsystems

24
The End

• Thanks for listening!!
• Any questions?