Biologically Inspired Computing: Introduction

1
Biologically Inspired Computing Introduction

• This is a lecture one of
• Biologically Inspired Computing
• Contents
• Course structure, Motivation for BIC , BIC vs
  Classical computing, overview of BIC techniques

2
General and up-to-date information about this
module

• Go to my home page
• www.macs.hw.ac.uk/dwcorne/
• Find my Teaching Materials page, and go on from
  there.

3
Course Contents
Lectures Title What its about
12 Intro to BIC The differences between BIC and ordinary computing, the kinds of problems we need BIC for (including basics of classification, optimisation, and problem complexity), motivation for BIC, and a broad overview of many BIC techniques and the kinds of problems they can solve.
38 Evolutionary Algorithms Algorithms based on natural evolution, for solving real-world problems various different algorithms based on this idea, several example applications
912 Swarm Intelligence Algorithms inspired by natural swarming behaviour, with various applications (ant systems, particle swarm optimisation)
1316 Alife / Cellular Automata Simple but powerful ways to simulate biological systems, with many useful applications. Includes L systems, Game of Life, self-organisation, and more complex CAs
1718 Neural Computing Classification and prediction methods based on linked simple processing units, modelled on real neural tissues.
1920 Other BIC methods A selection of other prominent BIC methods, e.g. Artificial Immune Systems, Cellular Computing, Foraging algorithms.
4
Examinable materials

• All the slides (all online)
• Various additional papers that I will provide
  online
• BSc 80 exam / 20 coursework
• MSc 100 exam, but there is still c/w and
• you must pass the coursework
• I will handout the coursework in either 5th or
  6th lecture, and the handin date will be Friday
  9th March

Coursework
5
Difference between BSc and MSc

• The MSc students will have some additional
  reading materials. The web site will clearly
  indicate this.
• The exams will be different
• The MSc coursework will be based on the BSc
  coursework, but a bit harder with more to do.

6
This Lecture

• Lecture 1
• Classical computation vs biological
  computation
• Motivation for biologically inspired
  computation
• Overview of several biologically inspired
  algorithms

7
What to take from this lecture
What classical computing is, and what kinds of
tasks it is naturally suited for. What classical
computing is not good at. An appreciation of how
computation and problem solving are manifest in
biological systems. Appreciation of the fact that
many examples of computations done by biological
systems are not yet matched by what we can do
with computers. An understanding of the
motivation (consequent on the above) for studying
how computation is done in nature. A first basic
knowledge of the main currently and successfully
used BIC methods
8
Classical Computation vs Bio-Inspired Computation

• the fridge story
• How do you tell the difference between dog and
  cat?
• How do you tell the difference between male and
  female face?
• How do you design a perfect flying machine?
• How would we design the software for a robot that
  could make a cup of tea in your kitchen?
• What happens if you
• Cut off a salamanders tail?
• Cut off a section of a CPU?

9
Classical Computation vs Bio-Inspired Computation
Classical computing is good at
Number-crunching Thought-support (glorified
pen-and-paper) Rule-based reasoning Constant
repetition of well-defined actions.
Classical computing is bad at
Pattern recognition Robustness to damage Dealing
with vague and incomplete information Adapting
and improving based on experience
10
Why dont we have software that can do the
following things well?

• Automatically locate a small outburst of violent
  behaviour in a football crowd
• Classify a plant species from a photograph of a
  leaf.
• Design robust railway timetables
• Make a cup of tea?

11
Pattern Recognition andOptimisation

• These two things tend to come up a lot when we
  think of what we would like to be able to do with
  software, but usually cant do.
• But these are things that seems to be done very
  well indeed in Biology.
• So it seems like a good idea to study how these
  things are done in biology i.e. (usually) how
  computation is done by biological machines

12
Basic notes on pattern recognitionand
optimisation

• Pattern recognition is often called classification

Formally, a classification problem is like
this We have a set of things S (e.g.
images, videos, smells, vectors, ) We have n
possible classes, c1, c2, , cn, and we know that
everything in S should be labelled with
precisely one of these classes. In
computational terms, the problem is Can we
design a computational process that takes a thing
s (from S) as its input, and always outputs the
correct class label for s?
13
Classification examples
What S might be What the classes might be
Images of peoples faces male, female
Smells (e.g. molecular spectra illicit_drugs, ok or fresh meat, good meat, rotten meat
Utterances of hello (e.g. in wav files) child, man, woman or authorised-person, unauthorised
Renditions of my signature genuine, fake
Images of artworks beautiful, good, reasonable, awful
Patient data results of various blood tests malignant, benign
Applications for loans good risk, medium risk, bad risk
Aircraft engine diagnostics safe, needs-maintenance, ground
Ground-penetrating radar image land-mine-here, safe
1
2
3
4
5
6
7
8
9
14
Some notes about those examples

• The idea of these examples is to
• Remind you that pattern recognition is something
  you do easily, and all the time, and you
  (probably) do it much better than we can do with
  classical computation. (e.g. 1, 2, 3, 5)
• Remind (or inform) you that such complex pattern
  recognition problems are not yet done well by
  software (e.g. 1, 2, 3, 5)
• Indicate that there are some very important
  problems that we would like to solve with
  software (9, 8, 6, 2, 7 are obvious, but of
  course we would like to do all nine and much more
  ), which are classification problems, and note
  that these are just as hard as examples 1, 2, 3,
  5.
• So, hopefully we can learn how brains do 1, 2, 5
  etc , so that we can build machines that find
  land mines, tell fake from genuine signatures,
  diagnose disease, and so on

15
How brains seem to do pattern recognition more
in lectures 1718
The business end of this is made of lots of these
joined in networks like this
Our own computations are performed in/by this
network
16
The key idea in brain-inspired computing
The brain is a complex tangle of neurons,
connected by synapses
17
The key idea in brain-inspired computing
When neurons are active, they send signals to
others.
18
The key idea in brain-inspired computing
A neuron with lots of strong active inputs will
become active.
19
The key idea in brain-inspired computing
And, when connected neurons are active at the
same time, the link between them gets stronger
20
The key idea in brain-inspired computing
So, suppose these neurons happen to be active
when you see a fluffy animal with big eyes,
small ears and a pointed face
21
The key idea in brain-inspired computing
So, suppose these neurons happen to be active
when you see a fluffy animal with big eyes,
small ears and a pointed face and suppose your
mother then says Cat, which excites this
additional neuron.
22
The key idea in brain-inspired computing
Links will then strengthen between the active
neurons
So when you see a similar animal again, this
neuron will probably Automatically be activated,
helping you classify it. A slightly different
group of neurons will respond to dogs, and
sometimes both the cat and dog group will be
active, but one will be more active than the
other
23
Notice this in particular
What happens if we damage a single neuron
(remember, in reality there will be thousands
involved in simple classification-style
computations)? Compare this with damaging a
line of code. In classical computing we provide
rules but biology seems to learn gradually from
examples.
24
Basic Notes on Optimisation

• We have 3 items as follows (item 1 20kg
  item2 75kg item 3 60kg)
• Suppose we want to find the subset of items with
  total weight closest to 100kg.

Well done, you just searched the space of
possible subsets. You also found the optimal one.
If the above set of subsets is called S, and the
subsets themselves are s1, s2, s3, etc , you
just optimised the function closest_to_100kg(s)
i.e. you found the s which minimises
the function (weight100) .
25
Search and Optimisation

• In general, optimisation means that you are
  trying to find the best solution you can (usually
  in a short time) to a given problem.

S
We always have a set S of all possible solutions

s1
s2
s3
S may be small (as just seen) S may be very,
very, very, very large (e.g all possible
timetables for a 500-exam/3-week diet) in fact
something like 1030 is typical for real
problems. S may be infinitely large e.g. all
real numbers.
26
How Biology Optimises

We wish to design something we want the best
possible (or, at least a very good) design. The
set S is the set of all possible designs. It is
always much too large to search through this set
one by one, however we want to find good examples
in S.
In nature, this problem seems to be solved
wonderfully well, again and again and again, by
evolution

Nature has designed millions of extremely complex
machines, each almost ideal for their tasks
(assuming an environment that doesnt go
haywire), using evolution as the only mechanism.
Clearly, this is worth trying for solving
problems in science and industry.
27
Quick overview of BIC techniques we will learn
about

• Evolutionary algorithms
• Use natures evolution mechanism to evolve
  solutions to all kinds of problems. E.g. to find
  a very aerodynamic wing design, we essentially
  simulate evolution of a population of wing
  designs. Good designs stay in the population and
  breed to, poor designs die out. EAs are highly
  successful and come in many variants. There is
  also a lot to learn to understand how to apply
  them well to new problems. We will do quite a lot
  on EAs. EAs are all about optimisation, however
  classification is also an optimisation problem,
  so EAs work there too

28
A genetically optimized three-dimensional truss
with improved frequency response.
An EA-optimized concert-hall design, which
improves on human designs in terms of sound
quality averaged over all listening points.
29

• Swarm Intelligence
• How do swarms of birds, fish, etc manage to
  move so well as a unit? How do ants manage to
  find the best sources of food in their
  environment. Answers to these questions have led
  to some very powerful new optimisation methods,
  that are different to EAs. These include ant
  colony optimisation, and particle swarm
  optimisation.
• Also, only by studying how real swarms work
  are
• we able to simulate realistic swarming
  behaviour (e.g. as done in Jurassic Park, Finding
  Nemo, etc )

30

• Artifical Life and Cellular Automata
• This is a research area that tries to learn what
  the fundamental computational structures and
  processes are that are necessary for the things
  that seem to go hand-in-hand with life. For
  example Growth, and Reproduction. One of the
  fruits of Alife are simple rule-based systems
  called L-systems that can be used to simulate
  very lifelike images of plants, that are used in
  computer graphics. Meanwhile, Cellular Automata
  (CA) are very simple computational systems that
  produce very complex behaviour, including
  lifelike reproduction. CAs, as we will see, are
  also very useful for explaining/simulating
  biological pattern generation and other
  behaviours

31

• Neural Computing
• Pattern recognition using neural networks is the
  most widely used form of BIC in industry and
  science. We will learn about the most common and
  successful types of neural network.

This is Stanley, winner of the DARPA
grand Challenge a great example of
bio-inspired computing winning over all other
entries, which were largely classical
32

• Other BIC techniques
• There are many other BIC areas under research,
  but not yet found as successful in practice as
  those we have concentrated on in the course. But
  we will look at the most prominent other
  techniques. At the moment these are
• Artifical Immune System methods which
  lead to algorithms for optimisation and
  classification based on the workings of the human
  immune system.
• Foraging Algorithms which lead to
  optimisation methods based on how herds of
  animals decide where to graze. These are
  different from the current main algorithms that
  have arisen from swarm intelligence.

33

Next Time

• Before we get into looking at Evolutionary
  Algorithms (as well as
• other methods that do optimisation)
• We need to understand a lot of things about
  optimisation, such as
• When we need clever methods to do it, and when
  we dont
• What the alternatives there are to EAs no
  point designing
• an EA for an optimisation problem if it can be
  solved far
• more simply.
• So the next lecture is all about optimisation
  problems in general,
• and some key pure computer-science things you
  need to know.