Perceptron Learning Demonstration

1
Perceptron Learning Demonstration

• Langston,
• Cognitive Psychology

2
Perceptron Learning

• How does a perceptron acquire its knowledge?
• The question really is How does a perceptron
  learn the appropriate weights?

3
Perceptron Learning

• Remember our features
• For output
• Good_Fruit 1
• Not_Good_Fruit 0

Taste Sweet 1, Not_Sweet 0
Seeds Edible 1, Not_Edible 0
Skin Edible 1, Not_Edible 0
4
Perceptron Learning

• Lets start with no knowledge

5
Perceptron Learning

• The weights are empty

6
Perceptron Learning

• To train the perceptron, we will show it each
  example and have it categorize each one.
• Since its starting with no knowledge, it is
  going to make mistakes. When it makes a mistake,
  we are going to adjust the weights to make that
  mistake less likely in the future.

7
Perceptron Learning

• When we adjust the weights, were going to take
  relatively small steps to be sure we dont
  over-correct and create new problems.

8
Perceptron Learning

• Im going to learn the category good fruit
  defined as anything that is sweet.
• Good fruit 1
• Not good fruit 0

9
Trained Perceptron

• Show it a banana

Input
1
1
Taste
0.0
Output
0.0
1
.00
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
10
Trained Perceptron

• Show it a banana

Input
1
1
Taste
0.0
Output
0.0
1
.00
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
11
Trained Perceptron

• Show it a banana

Input
1
1
Taste
0.0
Output
0.0
1
.00
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
12
Trained Perceptron

• Show it a banana

Input
1
1
Taste
0.0
Output
Teacher
0.0
1
.00
1
0
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
13
Perceptron Learning

• In this case we have
• (1 X 0) 0
• (1 X 0) 0
• (0 X 0) 0
• It adds up to 0.0.
• Since that is less than the threshold (0.40), we
  responded no.
• Is that correct? No.

14
Perceptron Learning

• Since we got it wrong, we know we need to change
  the weights. Well do that using the delta rule
  (delta for change).
• ?w learning rate X (overall teacher - overall
  output) X node output

15
Perceptron Learning

• The three parts of that are
• Learning rate We set that ourselves. I want it
  to be large enough that learning happens in a
  reasonable amount of time, but small enough that
  I dont go too fast. Im picking 0.25.
• (overall teacher - overall output) The teacher
  knows the correct answer (e.g., that a banana
  should be a good fruit). In this case, the
  teacher says 1, the output is 0, so (1 - 0) 1.
• node output Thats what came out of the node
  whose weight were adjusting. For the first node,
  1.

16
Perceptron Learning

• To pull it together
• Learning rate 0.25.
• (overall teacher - overall output) 1.
• node output 1.
• ?w 0.25 X 1 X 1 0.25.
• Since its a ?w, its telling us how much to
  change the first weight. In this case, were
  adding 0.25 to it.

17
Perceptron Learning

• Lets think about the delta rule
• (overall teacher - overall output)
• If we get the categorization right, (overall
  teacher - overall output) will be zero (the right
  answer minus itself).
• In other words, if we get it right, we wont
  change any of the weights. As far as we know we
  have a good solution, why would we change it?

18
Perceptron Learning

• Lets think about the delta rule
• (overall teacher - overall output)
• If we get the categorization wrong, (overall
  teacher - overall output) will either be -1 or
  1.
• If we said yes when the answer was no, were
  too high on the weights and we will get a
  (teacher - output) of -1 which will result in
  reducing the weights.
• If we said no when the answer was yes, were
  too low on the weights and this will cause them
  to be increased.

19
Perceptron Learning

• Lets think about the delta rule
• Node output
• If the node whose weight were adjusting sent in
  a 0, then it didnt participate in making the
  decision. In that case, it shouldnt be adjusted.
  Multiplying by zero will make that happen.
• If the node whose weight were adjusting sent in
  a 1, then it did participate and we should change
  the weight (up or down as needed).

20
Perceptron Learning

• How do we change the weights for banana?

Feature Learning rate (overall teacher - overall output) Node output ?w
taste 0.25 1 1 0.25
seeds 0.25 1 1 0.25
skin 0.25 1 0 0
21
Perceptron Learning

• Adjusting weight 1

.25 X (1 0) X 1 0.25
Input
1
1
Taste
0.0
Output
Teacher
0.0
1
1
0
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
22
Perceptron Learning

• Corrected weight 1

Input
1
1
Taste
0.25
Output
Teacher
0.0
1
1
0
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
23
Perceptron Learning

• Adjusting weight 2

.25 X (1 0) X 1 0.25
Input
1
1
Taste
0.25
Output
Teacher
0.0
1
1
0
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
24
Perceptron Learning

• Corrected weight 2

Input
1
1
Taste
0.25
Output
Teacher
0.25
1
1
0
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
25
Perceptron Learning

• Adjusting weight 3

.25 X (1 0) X 0 0.00
Input
1
1
Taste
0.25
Output
Teacher
0.25
1
1
0
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
26
Perceptron Learning

• Corrected weight 3

Input
1
1
Taste
0.25
Output
Teacher
0.25
1
1
0
1
Seeds
0.0
If ? gt 0.4 then fire
0
0
Skin
27
Perceptron Learning

• To continue training, we show it the next
  example, adjust the weights
• We will keep cycling through the examples until
  we go all the way through one time without making
  any changes to the weights. At that point, the
  concept is learned.

28
Perceptron Learning

• Show it a pear

Input
1
1
Taste
0.25
Output
Teacher
0.25
0
0.25
0
0
1
Seeds
0.0
If ? gt 0.4 then fire
1
1
Skin
29
Perceptron Learning

• How do we change the weights for pear?

Feature Learning rate (overall teacher - overall output) Node output ?w
taste 0.25 1 1 0.25
seeds 0.25 1 0 0
skin 0.25 1 1 0.25
30
Perceptron Learning

• Adjusting weight 1

.25 X (1 0) X 1 0.25
Input
1
1
Taste
0.25
Output
Teacher
0.25
0
0
0
1
Seeds
0.0
If ? gt 0.4 then fire
1
1
Skin
31
Perceptron Learning

• Corrected weight 1

Input
1
1
Taste
0.50
Output
Teacher
0.25
0
0
0
1
Seeds
0.0
If ? gt 0.4 then fire
1
1
Skin
32
Perceptron Learning

• Adjusting weight 2

.25 X (1 0) X 0 0.00
Input
1
1
Taste
0.50
Output
Teacher
0.25
0
0
0
1
Seeds
0.0
If ? gt 0.4 then fire
1
1
Skin
33
Perceptron Learning

• Corrected weight 2

Input
1
1
Taste
0.50
Output
Teacher
0.25
0
0
0
1
Seeds
0.0
If ? gt 0.4 then fire
1
1
Skin
34
Perceptron Learning

• Adjusting weight 3

.25 X (1 0) X 1 0.25
Input
1
1
Taste
0.50
Output
Teacher
0.25
0
0
0
1
Seeds
0.0
If ? gt 0.4 then fire
1
1
Skin
35
Perceptron Learning

• Corrected weight 3

Input
1
1
Taste
0.50
Output
Teacher
0.25
1
1
0
1
Seeds
025
If ? gt 0.4 then fire
0
0
Skin
36
Perceptron Learning

• Here it is with the final weights

Input
Taste
0.50
Output
0.25
Seeds
0.25
If ? gt 0.4 then fire
Skin
37
Perceptron Learning

• Show it a lemon

Input
0
0
Taste
0.50
Output
Teacher
0.25
0
0
0
0
0
Seeds
0.25
If ? gt 0.4 then fire
0
0
Skin
38
Perceptron Learning

• How do we change the weights for lemon?

Feature Learning rate (overall teacher - overall output) Node output ?w
taste 0.25 0 0 0
seeds 0.25 0 0 0
skin 0.25 0 0 0
39
Perceptron Learning

• Here it is with the adjusted weights

Input
Taste
0.50
Output
0.25
Seeds
0.25
If ? gt 0.4 then fire
Skin
40
Perceptron Learning

• Show it a strawberry

Input
1
1
Taste
0.50
Output
Teacher
0.25
1
1
1
1
1
Seeds
0.25
If ? gt 0.4 then fire
1
1
Skin
41
Perceptron Learning

• How do we change the weights for strawberry?

Feature Learning rate (overall teacher - overall output) Node output ?w
taste 0.25 0 1 0
seeds 0.25 0 1 0
skin 0.25 0 1 0
42
Perceptron Learning

• Here it is with the adjusted weights

Input
Taste
0.50
Output
0.25
Seeds
0.25
If ? gt 0.4 then fire
Skin
43
Perceptron Learning

• Show it a green apple

Input
0
0
Taste
0.50
Output
Teacher
0.25
0
0.25
0
0
0
Seeds
0.25
If ? gt 0.4 then fire
1
1
Skin
44
Perceptron Learning

• If you keep going, you will see that this
  perceptron can correctly classify the examples
  that we have.

45
End Perceptron Learning Demonstration