Artificial Intelligence Methods

1
Artificial Intelligence Methods

• Neural Networks
• Lecture 1
• Rakesh K. Bissoondeeal
• (rakesh.bissoondeeal_at_ntu.ac.uk)

2
Biological Neural Networks
3
Biological Neuron

• Synapses
• - Gap between adjacent neurons across which
  chemicals are transmitted input
• Dendrites
• - Receive synaptic contacts from other neurons
  
• Cell body/Soma
• - Metabolic centre of the neuron processing
• Axon
• - produces the output

4
Artificial Neuron

• Artificial neurons are the building blocks of
  Artificial Neural Networks

5
Artificial Neurons

• Artificial neurons simulate the four basic
  functions of natural neurons
• - Signals are passed between neurons over
  connection links
• - Each connection link has an associated weight
  which multiplies the signal transmitted
• - Each neuron applies an activation function to
  is net input (sum of weighted input signals) to
  produce an output signal

6
Why study Artificial Neural Networks

• Desire to understand the brain and to imitate
  some of its strength
• Traditional computers implement a sequence of
  logical and arithmetic operations but dont have
  the ability to adapt their structure or learn
• Learn from examples, Generalisation
• Used to solved task where it is beneficial to use
  a machine but impossible to program all possible
  outcomes

7
Applications

• List of applications mentioned in the literature
• Aerospace -high performance aircraft autopilot
• Banking check and other document reading
• Defence weapon steering
• Financial financial analysis
• Speech speech recognition

8
Brief History of ANNs

• 1943 W.S. McCulloch and W. Pitts
• - Original idea published
• 1949 D. Hebb
• - Publishes ideas on learning in biological
  neurons
• 1958 F. Rosenblatt
• - First practical working networks called
  perceptrons

9
Brief History of ANNs

• 1969 . M Minsky and S. Papert
• - Rubbish ANNs
• - Most research on ANNs stop
• 1970s Widrow, Parker and others
• - Low level of activity
• - Backpropagation invented
• 1980s Rumelhart and others
• - Rediscovery of Backpropagation
• - Revival of interest in ANNs

10
McCulloch-Pitts Neuron

• First mathematical model of the biological neuron
• - Mc Culloch and Pitts (1943)
• Most models used today are descended from
  McCulloch and Pitts neuron

11
McCulloch-Pitts Neuron

• The output of a neuron is binary. That is, the
  neuron either fires (output of one) or does not
  fire (output of zero).

12
McCulloch-Pitts Neuron

• Neurons in a McCulloch-Pitts network are
  connected by directed, weighted paths
• A connection path is excitatory if the weight on
  the path is positive otherwise it is inhibitory

13
McCulloch-Pitts Neuron

• Each neuron has a fixed threshold (?). If the net
  input to the neuron is greater than the
  threshold, the neuron fires
• If net input gt ?, output1
• If net input lt ?, output 0

14
Example 1

• Logic Functions AND
• True1, False0
• If both inputs true, output true
• Else, output false
• Threshold(Y)2

x1 x2 AND
0 0 0
0 1 0
1 0 0
1 1 1
15
Example 2

• Logic Functions OR
• True1, False0
• If either of inputs true, output true
• Else, output false
• Threshold(Y)2

x1 x2 OR
0 0 0
0 1 1
1 0 1
1 1 1
16
McCulloch-Pitts Neuron

• Structure does not change
• - Fixed system that takes inputs to produce
  output
• Has no concept of learning
• However, McCulloch-Pitts Neuron forms the
  foundation of modern ANNs
• - Changes made to allow learning

17
Recommended Reading

• Fundamentals of neural networks Architectures,
  Algorithms and Applications, L. Fausett, 1994.
• Artificial Intelligence A Modern Approach, S.
  Russel and P. Norvig, 1995.
• An Introduction to Neural Networks. 2nd Edition,
  Morton, IM.