Introduction to Artificial Neuron Networks [ANN]

1
Introduction to Artificial Neuron Networks ANN
Simulated Annealing SA

• AI for Computer Game
• Dr.Yodthong Rodkaew

2
Introduction to Artificial Neuron Networks ANN

• .

3
Neurons

• ??????????????????????????????????????????????????
  ???????????(Bioelectric Network) ??????
  ?????????????? ??????????? ???? ??????
  (Neurons) ?????????????????? (Synapses)

??????????????????????????????????????????????????
??????? "????????" (Dendrite) ???????? input
?????????????????????????????????? "??????"
(Axon) ?????????????? output ????????
????????????????????????????????????????
??????????????????????????????????????????????????
???????????? ?????????????????????????????????????
??????????????????????????????????????????????????
? ?????????? ???????????????????
????????????????????????????? ? ??????????????????
Ref http//gear.kku.ac.th/nawapak/NeuralNet/neu
ral1.ppt http//www.cdd.go.th/itcenter/manual/bmn
_chapter2.pdf http//www.resgat.net/modules.php?
nameNewsfilearticlesid55
4
????? !
????????????????????????????????????????????
????????????????????????????? ????????????????????
????????????????(????????????) ???????? ????
??????????? ?????????????????????? ??????????
??????????????????????? vs ???????????
???????????????????? ?????????????????????? 1000
Hz
?????????????????????? ??????????????????? 3000
MHz
????????
???????? ???????
??????????????????????????? gt 1000 MFLOP
?????? D
C
????????????? ?????????, ?????
?????? C
??????????? D
Ref http//gear.kku.ac.th/nawapak/NeuralNet/neu
ral1.ppt
5
Artificial Neural Networks

• ??????????????????? (Artificial Neural Networks
  - ANN) ???? ????????? ? ????????????????? (Neural
  Networks ???? Neural Net ???? NN)

http//en.wikipedia.org/wiki/FileNeural_network_e
xample.png
6
??????????????

• ?????????????????????????????????? ???? ??????
  ???????? ???????? ???????
• ????????????????????????????????????????????????
  (?? inputs ??? outputs????????????? inputs ???
  outputs ????????????????????????)
• ????????????????????????????????????
  (?????????????????????????????????)
• ??????????????????????????????
• ???????? ???????????????? ???????????

http//www.resgat.net/modules.php?nameNewsfilea
rticlesid55
7
?????????

• ???????????? neural network

http//www.resgat.net/modules.php?nameNewsfilea
rticlesid55
8
??????????????????? Artificial Neural Networks
?????????????????????????????
???????????????????????? ????????
?????????????????????????
Ref http//gear.kku.ac.th/nawapak/NeuralNet/neu
ral1.ppt
9
???????????????????????????????????????
??????????????? slide ???? ??
Output ????????????
Wij ??????? (weight) ??????????? (connection)
Xi input ????????????
Ref http//gear.kku.ac.th/nawapak/NeuralNet/neu
ral1.ppt
10
Network Architecture
Feedforward network ????????????????????????????
????????????????????????? Input
nodes ???????????????????? output nodes
???????????????????????????? ?????????? Nodes ??
layer ??????????????????????????????
Direction of data flow
Ref http//gear.kku.ac.th/nawapak/NeuralNet/neu
ral1.ppt
11
Network Architecture (cont.)
Feedback network ???????????????????????????
??????????????????????????????????????????
?????????????????????? (????????????? Recurrent
network)
Ref http//gear.kku.ac.th/nawapak/NeuralNet/neu
ral1.ppt
12
A Very Simple Neuron
??????????? Neural networks ?????????? input
????????? network ????? input (xi) ????????
weight (wi) ?????????? ??????????? input ??? ?
????? neuron ????????????????????????????????
threshold ??????????? ????????????????????
threshold ???? neuron ??????? output (yi) ?????
output ?????????????????? input ??? neuron ???? ?
?????????????? network ?????????????? threshold
??????????? output
Ref 202.28.94.55/web/320417/2548/work1/g26/Files/
Report_Neural20Network.doc -
13
Output

• ????????????????????????????????
  ????????????????? -1
• ??????????????????????????????????????????
  ???????????? 1

• ?? ?.?. 1993 Warren Macculloch ??? Walter Pitts
  ??????????????????????????????????????????????????
  ??????????????????????????????????????????????????
  ??
• ??????????????????????????????????????????????????
  ??????????????????????????????????????????????????
  ???????(threshold value) ???? ?

ref Ch8 Sasalak Tongkaw ajsasalak_at_yahoo.com
4124501 Artificial Intelligence AI 2005 Songkhla
Rahabhat University
14
Activation Function
????? X ????????????????????????????????? xi
??????????????????? i wi ????????????????????
?????? i n ????????????????????????? Y
??????????????????????????
Y 1 if X ? -1 if X
? ?
ref Ch8 Sasalak Tongkaw ajsasalak_at_yahoo.com
4124501 Artificial Intelligence AI 2005 Songkhla
Rahabhat University
15
Activation Function

• Activation function ??????????????????????????
  Output ???? ??? output ?????????????? ??? ????
  ?????? ???????????? Threshold function
• ??????? output ?????????????????????????
  ?????????? continuous function ???? Sigmoid
  function

16
Other activation function
ref Ch8 Sasalak Tongkaw ajsasalak_at_yahoo.com
4124501 Artificial Intelligence AI 2005 Songkhla
Rahabhat University
17
A Very Simple Neuron
if (sum(input weight) gt threshold) then output
sum (x1 w1 ) ( x2 w2 ) If (sumgt
threshold) then yi 1 else yi 0
Ref 202.28.94.55/web/320417/2548/work1/g26/Files/
Report_Neural20Network.doc -
18
A Very Simple Neuron
0.5
0.0
0.5
1.0
0.5
sum (0.00.5 ) ( 1.0 0.5 ) 0.5 If (sumgt
threshold ) then yi 1 else yi 0 ??? yi
?????????? ?
19
Multilayer Neuron Network
Input layer
output layer
hidden layer
w1,1
x1
Y1
wH1,1
w1,2
H1
wH1,2
wH2,1
w2,1
x2
H2
w2,2
Y2
wH2,2
H1 threshold (x1w11) (x2w21) H2
threshold (x1w12) (x2w22) Y1 threshold
(H1wh11) (H2wh21) Y2 threshold (H1wh12)
(H2wh22)
20
Shape recognition with NN
Input layer
output layer
hidden layer
w1,1
x1
Y1
wH1,1
w1,2
H1
rectangle
wH1,2
wH2,1
w2,1
x2
H2
w2,2
Y2
wH2,2
tritangle
..
..
..
..
H5
x9
???????????????? 5 ????
???????????????? 9 ????
21
Neural Network (NN)

• Classification Problem solved with Neural
  Network, weight tuning

threshold
i1
w1
output
S
T
i2
w2
start Wi 0.5 T I1 W1 I2 W2 (T gt0)
output 1 (T lt0) output 0
weight
input
train adjust w Wia(Desire_Output
Current_Output)Ii a learning rate
22
Back propagation Algorithm

• Back-propagation ?????????????????????????????????
  ?????????????????????????????????????????
  multilayer perceptron ????????????????????????????
  ??????????????????????????? ??????????????????????
  ??????????????????????????????????????????????????
  ?????????????

O neuron output ??? NN T target
?????????? r learning rate 0.1 ???????? Y1
H1wh11H2wh21 ??????? ?????????? Y1 ????????
T wH11 wH11 r(T-O) wH21 wH21
r(T-O) ???????? H1 X1w11X2w21 ??????? w11
w11 r(T-O) w21 w21 r(T-O) ????????
?????????????????????????? ???????????????????????
????????????????
Ref 202.28.94.55/web/320417/2548/work1/g26/Files/
Report_Neural20Network.doc http//en.wikipedia
.org/wiki/Backpropagation
23
(No Transcript)
24
Introduction to Simulated Annealing SA

• .

25
Local Maxima Problem

• ????? local maxima ?????????? search
  ?????????????????????????????? (?????????)
  ???????????????????????????? ??????????? search
  ??? ??????????????????????????????????????????????
  ???????????

The best solution
Local maxima
Start search point
http//www.bloggang.com/viewblog.php?idzoldate0
4-01-2008group10gblog34
26
Simulated Annealing - SA

• ?????????? local search ??????????????????????????
  local maxima ??? ????????????????? anneal
  (??????????????????????? ????????) ??????? ?? 2
  ??????????? ? ??? ??????????? (heating)
  ??????????????????? (slowly cooling)
  ??????????????????????????????????????????????????
  ???????????????
• ????????????????? ????????????????????????????????
  ????????????????? (downhill move)
  ??????????????????????????????????? ?
  ???????????????????????? ?????????????????????????
  ???????????????? (T 0) ?????????????????????

The best solution
Local maxima
Start search point
http//www.bloggang.com/viewblog.php?idzoldate0
4-01-2008group10gblog34
27
Simulated Annealing

• starting from state s0 and continuing to a
  maximum of kmax steps or until a state with
  energy emax or less is found.
• The call neighbour(s) should generate a randomly
  chosen neighbour of a given state s the call
  random() should return a random value in the
  range 0,1).
• The annealing schedule is defined by the call
  temp(r), which should yield the temperature to
  use, given the fraction r of the time budget that
  has been expended so far

http//en.wikipedia.org/wiki/Simulated_annealing
28
????????????????????? ????????????????????
http//www.bloggang.com/viewblog.php?idzoldate0
4-01-2008group10gblog34
29
(No Transcript)