Artificial Immune Systems

1
Artificial Immune Systems

• Steve Cayzer
• Semantic and Adaptive Systems
• Hewlett-Packard Laboratories, Bristol
• February 2005

2
About me
3
(No Transcript)
4
The Immune System is
Immune system a system that protects the body
from foreign substances and pathogenic organisms
by producing the immune response

• Immunity state or quality of being resistant
  (immune), either by virtue of previous exposure
  (adaptive immunity) or as an inherited trait
  (innate immunity)

5
Or, maybe, the Immune System is
KNOW THYSELF? The Self Assertion View
Hugues Bersini While host defense is a critical
function, it is hardly the only one of interest.
Indeed the immune system might be regarded as
primarily fulfilling an altogether different
role

• Francisco Varela Homeostasis, self-develop an
  efficient communication pathway in order to
  create (assert) and maintain a coherent self
• John Stewart rejection and memory are side
  effects of the homeostatic maintain.

Immune system only knows itself, no recognition
is at play
ftp//iridia.ulb.ac.be/pub/bersini/ImmunoSelf.pdf
6
But for the purposes of this seminar

• The human body is constantly under attack from
  pathogens which produce antigens (foreign
  proteins)
• The immune system creates antibodies which match
  the antigens and cause the pathogens to be
  destroyed
• without destroying the host (self proteins)
• Each antibody matches a range of proteins as a
  population, antibodies (learn to) cover non-self
  space.
• Adaptive, self organising system good paradigm
  for new computing?

7
(No Transcript)
8
Antibodies map non-self space

• Non-Self

Self
X
Antibody (with recognition radius)
9
Antibodies map non-self space

• Non-Self

Self
X
10
Antibodies map non-self space

• Non-Self

Self
Autoreactive Antibody Destroyed
X
11
Antibodies map non-self space

• Non-Self

Self
X
X
Antigen (matched by antibody)
12
Antibodies map non-self space

• Non-Self

Self
X
X
X
X
Clonal Maturation With hypermutation
13
Definition of AIS
de Castro and Timmis Artificial Immune Systems
(AIS) are adaptive systems, inspired by
theoretical immunology and observed immune
functions, principles and models, which are
applied to problem solving

• basic model of an immune component (eg antibody)
• design informed from immunology
• aimed at problem solving

http//www.cs.kent.ac.uk/people/staff/jt6/aisbook/

14
Models, Design Features, Applications

• Models
• Negative Selection
• Positive Selection
• Danger model
• Self Assertion

• Features
• Learning
• Associative Memory
• Avoids self
• Autonomous

• Applications
• Security
• Classification/Clustering
• Optimisation
• Modelling

15
GA basic algorithm

• Initialise population
• WHILE (not finished)
• Calculate fitness
• Select
• Reproduce
• Crossover and mutation
• Replace
• END WHILE

16
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Calculate fitness
• Select
• Reproduce
• Crossover and mutation
• Replace
• END WHILE

17
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Calculate fitness
• Select
• Reproduce
• Crossover and mutation
• Replace
• END WHILE

18
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness
• Select
• Reproduce
• Crossover and mutation
• Replace
• END WHILE

19
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce
• Crossover and mutation
• Replace
• END WHILE

20
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce
• Crossover and mutation
• Replace
• END WHILE

21
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce (clonal expansion)
• Crossover and mutation
• Replace
• END WHILE

22
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce (clonal expansion)
• Mutation (no crossover)
• Replace
• END WHILE

23
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce (clonal expansion)
• Mutation (no crossover)
• Replace (variable population)
• END WHILE

24
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce (clonal expansion)
• Mutation (no crossover)
• Replace (variable population)
• Memory cells
• END WHILE

25
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce (clonal expansion)
• Mutation (no crossover)
• Replace (variable population)
• Memory cells
• END WHILE

26
Framework for AIS design
Algorithms
Affinity (cf fitness in GA)
Representation
Application Domain
27
Philosophical Divide
AIS can be thought of as a special case of

• Neural network
• Pattern classification
• Unsupervised learning
• Topographic mapping
• Variable network topology
• Considerations
• Interpreting response
• Training regime

• Genetic algorithm
• Creation (gene libraries)
• Emphasis on mutation
• Matching fitness (?)
• Variable population size
• Considerations
• Role of antigen
• Preservation of diversity

AIS as optimiser
AIS as classifier
28
AIS for classification and clustering (1)
Supervised Tasks AIRS
K- Nearest Neighbour Hamming (usu) Varied (UCI
datasets) Data Mining
29
AIS for classification and clustering (1)
Supervised Tasks AIRS AIRS does competitively
benchmarked against 35 classifiers on some
standard datasets
(number represents ranking where 1 best, 35
worst)
AIRS focus now shifting to parallel properties
Watkins, A., Timmis, J., Boggess, L. 2004
Artificial Immune Recognition System (AIRS) An
Immune-Inspired Supervised Learning Algorithm
Genetic Programming and Evolvable Machines 5
(3) 291-317
30
AIS for classification and clustering (2)
Unsupervised Tasks aiNET
Immune network model Euclidean Expression
levels Gene Expression Clustering
31
AIS for classification and clustering (2)

• Unsupervised Tasks aiNet
• an iterative clustering algorithm that performs
  data compression using a pattern recognition
  process inspired by the human immune system.
• applied to a benchmark data set of gene
  expression levels
• Capable accurately detecting the presence of
  clusters without a priori knowledge about the
  number of clusters

Bezerra, G. B., de Castro, L. N. (2003), A
Hybrid Approach for Gene Expression Data
Clustering, International Conference on
Bioinformatics and Computational Biology,
2003http//www.vision.ime.usp.br/cesar/programa/
pdf/112.pdf
32
AIS for Security (1)
Negative Selection LISYS
Negative Selection r contiguous bits Network
connections Intrusion Detection
33
AIS for Security (1)

• LISYS does quite well on test data
• Problems with false positives?
• Does it scale?

S. A. Hofmeyr and S. Forrest (1999) Immunity by
Design An Artificial Immune System Proceedings
of the Genetic and Evolutionary Computation
Conference (GECCO) pp. 1289-1296 Kim, J. and
Bentley, P. J. (2001) "Evaluating Negative
Selection in an Artificial Immune System for
Network Intrusion Detection" , Genetic and
Evolutionary Computation Conference 2001
(GECCO-2001) pp.1330 - 1337 Balthrop, J. Forrest,
S. Glickman, M.R. (2002) Revisiting LISYS
parameters and normal behavior Proceedings of
the 2002 Congress on Evolutionary Computation,
CEC '02 1045 - 50
34
AIS for Security (2)
Alternatively
35
Problems with the self-nonself worldview

• How do we produce antibodies that react against
  antigens and yet avoid self?
• One way is Generate and Test negatively screen
  antibodies which react to self at production time
• But this is expensive!
• Its difficult to screen against ALL self.
• Self also changes over time
• And it is not necessary to screen against all
  non-self only dangerous non-self
• Aickelin Cayzer 2002 The Danger Theory and Its
  Application to Artificial Immune Systems Proc.
  International Conference on AIS (ICARIS 2002)

36
The Danger Theory

• In the danger model, the idea is to recognise
  danger rather than non self.
• The screening is accomplished post production
  through an external danger signal.
• Thus the production of autoreactive antibodies
  (which react to self) is allowed.
• If an (eg autoreactive) antibody matches a
  stimulus in the absence of danger, it is removed.
  
• Thus harmless antigens are tolerated, and
  changing self accommodated.
• Matzinger (2002) The Danger Model A renewed
  sense of self Science 296 301-304

37
Towards a dangerous IDS
The danger theory suggests that the immune
system reacts to threats based on the correlation
of various (danger) signals, providing a method
of grounding the immune response, i.e. linking
it directly to the attacker.
Aickelin U, Bentley P, Cayzer S, Kim J and McLeod
J (2003) 'Danger Theory The Link between AIS
and IDS?', Proceedings ICARIS-2003, 2nd
International Conference on Artificial Immune
Systems, LNCS 2787, pp 147-155
Danger signals (eg memory usage, SIGABRT
signals) could be useful evidence to help a
security AIS refine its detectors. Danger
Grounding ?
EPSRC Adventure Fund 2004-2007 HP, UCL, UWE,
Nottingham
38
Other ways of using danger
Danger Crime, Antigen Suspect or... Danger
Context ?
It could also be useful for data mining, where
the danger signal is a proxy measure of
interest Danger Zone can be spatial or temporal
Andrew Secker, Alex Freitas, and Jon Timmis
(2005) Towards a danger theory inspired
artificial immune system for web mining in A
Scime, editor, Web Mining applications and
techniques, pages 145-168 (Idea Group)
39
Future Prospects

• Optimization has taken a back seat.
• Data Mining classification, clustering
• AIS for security research ongoing
• Danger Theory One to watch
• Other ideas
• community, modelling, immunocomputing

40
(No Transcript)
41
A slightly less simplified immune system
42
AIS basic algorithm

• Initialise population of antibodies
• WHILE (not finished)
• Present antigen
• Calculate fitness (match to antigen)
• Select
• Reproduce (clonal expansion)
• Mutation (no crossover)
• Replace (variable population)
• Memory cells
• END WHILE

43
AIS Refined algorithm

• Basic Matching Algorithm
• Population of B Cells (antibodies)
• Clonal expansion and hypermutation
• Extensions
• Lifecycle events, screening (positive/negative
  selection)
• Other IS elements (T Cells, cytokines)
• Network interactions (idiotypic effects)
• Other localization, self adaptation, population
  control
• Choices
• Genotype/Phenotype (Representation Shape
  Space)
• Matching (Hamming, Euclidean, r-contiguous,
  other)

44
The Idiotypic Effect Antibody-antibody
interactions
Jernes Big Idea (1974) Idiotype specificity
of antibody (epitopes to which it will
bind) Idiotope An idiotypic epitope Evidence
Antibodies produced against antibodies of same
species (cf individual)
Antigen
45
The Idiotypic Effect Why do we care?

• Biological importance - ???
• Immunological models Varela, Castellani
• Pattern recognition Timmis Hunt
• Non-stationary environments (idiotypic memory)
  Gaspar Collard
• Multimodal Optimisation de Castro
• Recommendation communities Cayzer Aickelin

46
Modelling the Idiotypic Effect

• For N antibodies, n antigens.
• xi is the concentration of antibody i
• yi is the concentration of antigen I
• c, k1 and k2 are scaling constants
• mij is a matching function

47
AIS for Optimisation

• Antibodies as entire solutions building
  blocks
• Antigens as objective functions constraints f
  it/feasible solutions solutions to
  subproblems weight combinations spanning Pareto
  optimal front
• AIS usually hybridised with GA Antibody
  selection Gene library creation
• Emergent fitness sharing (generalist/specialist)

48
AIS for Optimisation

• Evaluation
• Applied to TSP (of course), job shop scheduling,
  time series prediction, truss design, capacitor
  placement, time dependent optimization
• Some good results on test problems
• BUT
• Often little added value to GA
• AIS metaphor somewhat strained
• Difficult to find fair comparisons
• Best viewed as a collection of hybridising
  techniques

49
Example Hajela Yoo 2001
Designs
Feasible
Infeasible
Antigen
Antibody
Crossover Mutation
Small s
best
all
Generalist AIS
GA (unconstrained objective function)