Neural%20Network

1
???? ?????? ?????????
????? ?????? ???????????? ???? ??????? ?????
???????? 2005/2006

• ??? ?? ?????
• Neural Network

2
??? ?????? ??????

• ??????? ????? ???????? ?????? ???????? ?????
  ????
• 1- ??? ?????? ??????? ???????
• 2- ???? ?????

3
3- ???? ??????? ??????? ?? ???????? ??? ????
??????
4- ????? ??????? ???????
4
5- ??????? ???????
5
????? ??????? ???? ?????? ?????? ?? ?????? ?
??????? ??????? ?????????.
6
?????? ????????? ???? ??????? ?? ??? ???? ??????
????? ???????? ?????? ?????? .
7
????? ?? ??????? ??????? ??????????

• ??????? ??????? ??????????
• ?????? ??????
• ????????? ??????????
• ????? ??????? ???????
• ????? ???????
• ?????? ????????? ??????? ?????????
• ?????? ??? ??????? ???????? ????????
• ??????? ??? ??????? ???????
• ??? ????? ?????? ?????????
• ????????? ????? ??????
• ????? ?????? Network Creation

8
??????? ???????? ??????? ???????

• 1- set of simple processing
• 2- a pattern of connectivity
• 3-a rule for propagating signal through the
  network
• 4-a rule for combining input signal
• 5- a rule for calculating an output signal
• 6- learning rule to adapt the weights

9

• Artificial Neural Networks
• ?? ?????? ?????? ????? ??????? ??????? ???? ????
  ??? ?????? ?????? ???? ?????? ???? ?? ???? ??????
  ???? ????? ??? ???????? ?????? ?? ????? ??????
  ?????? ??? ??????? ?? ?? ??? ????? ?????? ????
  ??????? ?? ??? (Nodes , Neurons ) ????? ??? ?????
  ????? , ?? ??? ???? ???? ?????? ??????? ???????
  ?????????? ????????? ??????? ????? ???????? ????
  ?? ???? ??? ??????? ?????.

10

• ?????? ?????? ????????? ??????????
• ??? ????? ?? ??????? ????????? ????? ?? ?????? ??
  ????? ???????? ????? ????? ????? ? ?????? (1)
  ???? ?????? ?? ???? ?????? ??????? ?????????  
• ??? ?? ??????? ????? ????? ????? ??????? ???????
  ??? ????? ?????? ????? ??????? ??????? ?????
  ?????? ????? . ?????? ?????? ??? ???? ??????
  ?????? ???? ??? ??????? ? ???? ?? ?????? ??? ???
  ????? ???????? ??? ???? ?? ???????? ?????? .
  ?????? ??????? ????? ???? ???? ???? ????????? ?
  ????? ???????? ????? ???? ???????? ??? ???? ????
  ????? ??????. ???? ?? ???? ?? ??? ??????? ????
  ???? ?? ??????? ??????? ???? ???? ?? ???? ???????
  ???? ????? ????? ??? ???? ??? ??????? ?????? ???
  ???? ?????? ?????? ?????? ??? ???? ????? ??? ??
  ????? ???????(??????) , ?????? ?? ????? ??? ????
  ?? ???? ?? ????? ????????(????????? ????????).

11

• ????? ?? ????? (2) ?? ??????? ????? ??
• 1 - ?????? ????? ( Input )  a1,a2,a2,.an 2- ???
  ??????? (Weights )  Wj1, Wj2, Wj3,Wjn ??? ????
  ????? ?? ??? ??????? ??? ???? ???? ????? ???? . 3
  - ???? ???????? J   (Processing Element )
• ???? ?????? ???? ??? ????? 
• ? - ?????? (Adder ) ???? ???????? ?? ?????
  ??????? .
• ? - ???? ????? ?? ???? ??????? (Activation
  Function ) 
• ???? ?????? ??? ?? ??? ??????? ??? ???? ?????
  ??????? Squashing ??? ???? ????? ??? ?????? 0,1
  ?? ??? ??????-1,1 ???? ????? ??? ?????? ??????.
• 4 - ????? (Output) (Xj ) .

12
????? ??????? ??????? 

• ??? 
• X1,X2------Xm   ?????? ????? . Wk1, W
  k2------Wkm  ??????? ??????? ??????? k. UK  
  ????? ????? ?????? . bk ???????? .
• ???? ??????? .

13

• ??????  ???? ?????? ???????? bk ??? ??? ???
  ??????? W0 ?????X0 1 ????? ????? ???????
  ???????

14
????? ???????

• ????? ??????? ????? ??????? ??? ??????? ????????
  ???? ???? ???? ??????
• ??? ?? ??? ??????? . ???? ?? ????? ??????
  ???????
• ?? ???? ????? ?????? . ?? ???? ????? ????????
  ?????? ??? ?????? .
• ?? ???? ???????? ??? ??????.
• ????? ????? ????? ?????? ???????

15
1- ???? ?????? ?? ???? ??????

• ??? ??? ?????? ?? ??? ??????? ???? ???? ?????
  ?????? ?????? ??? ??? ????? ???? ?? ?????? ?????
  ????? ????? ?????? ????? ??? ??? ????? ???? ??
  ?????.
• ?????? ?? ????? ????????? ???? ??????? (??????)

16
2- ?????? ?????? ????? ?? ???? ???????

• ?????? ??? ?????? ?? ????????? ????????? ??
  ???????? ????????? ?????? .

17
3- ?????? ?????? Sigmoid

• ???? ??? ?????? ??? ????? ???????? ??? 8 ?? ? 8
  ????? ????? ?????? ??? 0 ? 1 ... ??? ???? ???????
  ???????? ???? ????? ??????? ????? ?????? .

18
?????? ????????? ??????? ?????????

• ??????? ?????? ??????? ??????????, ?? ???????
  ???? ????? ??? ????????? ?? ????? ????? ??????
  ?????? , ???? ????? ????????? ??????? .
  1-4-3?????? ??? ?????? ??????? ????????  ??
  ????? (8) ????? ?? ????? ?? ?????? ???? ????? P
  ??? ????? ?? ???? ?????? ????? W .

19
?????? ???? ????? ???? ??? ?????? ?? ???? ??????
??????? ???????

• ?????? ????? ?????? ??? ???????? ??? ??? ???????
  ????? ??? ?????? ?????? ??? ?????? ????? ?????? .
  ?? ?? ???????? ?? ?????? W1,2 ??? ??? ?? ???
  ????? ????? ???????? ????? , ??? ????? ????? ????
  ??????? ?? ??????? ??????? .

20
?????? ??? ??????? ???????? ???????? 
21

• ????? ?? ???? ?? ??? ?????? ????? ???? ??? ????
  ????? .?????? ???? ???? ????? ???? ???? ????? ,
  ??? ????? ??? ????? ???? , ????? ??????? ????
  ??????? ?????? . ???? ?? ???? ?????? ????????
  ??????? ?? ????? ?????? ???????? ????? ???????
  ?????? 

22
??????? ??? ??????? ???????

• ??? ????? ???? ??? ????? ???? ????? ????? ????? ,
  ????? ?? ????? ?? ???? ????? ?? ????????? ???
  ????? ???? ???? ??? ??? ?? ????????? ????????
  .??? ???? ???? ????? ????? ????? ?? ?? ???
  ??????? ???? ??? ???? ???? ??? ??????? ?????
  ????????? ?? ?????? ?????? ????? ?????? ?????
  ??????? ??????? ?? ???? ????? ?????? .

23
??? ????? ?????? ?????????

• ????? ?????? ?? ???? ??????? ?????? ?? ????????
  ???? ??? ?? ???? ?????? ??????? ??? ??? ?????? ??
  ???? ???? ??????. ??????? ??????? ??? ???? ???
  ???????.
• ?????? ??? ????? ???? ????? ??? ????? ??? ???
  ??????? ???? ???? ??? ?????? ????

24
????? ?????? ???????

• 1- ??????? ????? ???? ??? ?????
• 2- ??????? ????? ???? ??????? ????????? ??????
  ????? ????? ?????? ????? ??????? ?? ??????
• 3- ????? ?????? ??????? ???? ????????? ??????
  ???? ?????? brain maker

25
????? ????? ?????? ???????

• ?? ????? ??????? ?????? ??????? ???? ?????
  ??????? ????????? ????? ?? ??????? ??????? ??
  ??????? ??????? ??? ??? ??????? ??? ??????? ???
  ?????? ???????
• ??? ???? ?????? ?????? ??????? ?? ??? ???????

26
?????? ???????? ?????? ???????

• 1- ?? ????? ??? ????? ????? ???????
• 2- ???? ????? ?\??? ????????? ?? ???? ??????
• 3- ?? ????? ??? ?????? ????? ?? ???? ????? ?????
  ??? ?????? ???? ??? ????? ???????
• 4- ?? ???? ????????? ?? ????? ?????? ????? ???
  ?? ????? ?? ???????? ??????? ???????

27
??????? ??????? (?????? ????) Supervised Learning
of ANNs

• ??????? ??? ??????? ( ???? ???? ) Unsupervised
  learning

28
????????? ????? ??????

• 1- ????? ???????? ??????? Feed forward Back
  Propagation
• 2- ????? ???????? ?????? Back Propagation

29
????? ???????? ?????? Back Propagation

• ??? ????? ??? ????? ?????? . ?? ???????? ????????
  ?????? ???????? ?? ???????? ???????? ????????
  gradient descent algorithm
• ???? ??????? ????? ???????? ???????? 
• ????? -?????? ???????? Incremental mode
• ?????? - ???? ?????? ??????? Batch mode

30
????? ?????? Network Creation

• ?? ??? ???? ?? ????? ?????? ?? ????? ?????? ????
  ???????? ??? ????? . ???? ???? ???? ????? ????
  ?????? ???? ?????? ?????? newff ???? ????? ???
  ????? ?????? ??? ??? 
• 1- ?????? ????? ??? ????? ?????? ??????? ??? ????
  ?? ????? ???? ????? ????? ?? ?????? ????
  ??minmax(p) ???? ???? ?????? ???? ????? ???? ??
  ???? ?????.
• 2- ?????? ????? ??? ??? ????????? ?? ?? ???? ??
  ????? ?????? .
• 3- ?????? ????? ??? ????? ????? ??????? ??? ????
  .
• 4- ??? ???? ??????? ???????? .

31

• ???? 
• (network1newff( 0 5,10,6,2,tansig,logsig,pur
  lin, traingd
• ?? ??? ???????? ???? ?????? ???? ?????? ???
  ???????? ?????? ? ??? ??? ???? ????? ??? ????????
  0 ? 5 ? ?????? ??? ?????? ?? ?????? ?????? ?????
  ??? ? ?????? ?????? ?????? ???? ???? ???????
  ????? ?????? ?????? ??????? ???? ??? ??????? ?
  ??? ???? ????? ?????? ?? ?????? ??? , ??????
  ??????? ???? ??????? ?? tansig ?????? ??????
  ?????? ? ? logsig ??????? ? ? purlin ????? ?????
  ? ??? ???? ??????? ???????? ?? ??? ?????? ??
  traingd .

32
???? ??????? traingd ???????????

• ?????? ?? ???? ????? ?????? ?? ????? Batch mode
  ???? ?????? ??? ??? ????.???? ??? ?????????
  ?????? traingd ???? ??????????? ???? ????????
  ???
• 1- ???? ?????? tr ???? ??? ????? ???? ????
  ????? ??????????? .
• Show - 2 ??? ?????? ???? ??????? .
• Epoch - 3 ??????? ?????? ????? ??????? ? ???
  ????? ?????? ?? ??????? ??? ??? ??? ????????? ???
  ?? epochs ??????.
• Goal - 4 ?????? ???? ????? ??????? .
• min_grad ????? ??????? ???? ??? ???? ??????? .

33
??? ??????? ?????????? Initializing weights

• ??? ????? ?????? ??? ?? ???? ????? ????????
  ??????? ???????????. ?? ???????? ??????? newff
  ??? ???? ???????? ??????? ??????????? ???? ??? ?
  ???? ?? ??? ??????? ????? ??? ????? ????? ???
  ????? ???? ??? ??? ??????? ?? ???? ???????? init
  ??? ???? ??? ???????? ?????? ???? ??????? ????
  ??? ???
• (Network1init ( Network1

34
??????? training

• ??? ????? ????? ?????????? ??????? ???????????
  ???? ?????? ????? ??????? ? ????? ??????? ?????
  ??? ??????? ??????????? ???? ?????? ????? ??????
  ??? ?????? ?????? ????? ?????? ?? ?? ???? ????
  ?????? performance function.
• ?? ???? ?????? ????????? ?????? ??????? ????????
  ?? ????? ???? ????? (mse).

35
?????? ?????? Simulation

• ???? ?????? ?????? ?? ?????? ?????? ??? ???? ???
  ???????? ????? ? ????? ???? ?????? ??????? ????
  ??? ?????? p ?? ?????? ????????T ? ????? ???
  ?????? A ??? ?? ?????? ??????
• (Asim(network1, T

36
?????

• ????? ?? ??????? ??????? ?????????? - ???????? -
  ??? ?????? ???????.

37
?????? ??????? ????????? ????????

• ??? ??????
• ???? ???? ?? ????
• ????? ??????? ?????????
• ???? ???? ????????? ????