NEURAL NETWORKS FOR DATA MINING

1
Chapter 8

• NEURAL NETWORKS FOR DATA MINING

2
Learning Objectives

• ????????????????????????????????????????????
  (artificial neural networks (ANN))
  ???????????????????????
• ????????????????????????????????????? ANN
• ??????????????? back propagation neural networks
  ??????????????????
• ??????????????????????????????????????????????????
  ?
• ??????????????????????????????????????????????????
  ???

3
Basic Concepts of Neural Networks

• ?????????????? (NN) ???? ???????????????????
  (artificial neural network (ANN))
• ?????????????????????????????????????????????????
  ??????????????????????????????????
  ??????????????????????????????????????????????????
  ??????????????????????????????????????????????????
  ???

4
Basic Concepts of Neural Networks

• ????????????????????? (Neural computing)
• ?????????????????????????????????????????????????
  ??????????????????????????????????????????????????
  ??????????????????
• Perceptron
• ????????????????????????????????????????????????
  (hidden layer)

5
Basic Concepts of Neural Networks

• ??????????????????????????????? (Biological and
  artificial neural networks)
• ?????? (Neurons)
• ????? (????????????? (processing elements)) ???
  biological ???? artificial neural network
• ????????? (Nucleus)
• ?????????????????????????
• ???????? (Dendrite)
• ??????? biological neuron ???????????????????????
  ?

6
Basic Concepts of Neural Networks

• ??????? (Axon)
• ?????????????(i.e., terminal) ??? biological
  neuron
• ??????? (Synapse)
• ???????????? (?????????????????????????
  (weights)) ??????????????????????? ?
  ????????????????

7
Basic Concepts of Neural Networks
8
Basic Concepts of Neural Networks
9
Basic Concepts of Neural Networks

• ????????????? ANN
• ???????? (Topology)
• ??????????????????? ? ???????????????????????????
  ?????????? ?
• ??????????? (Back propagation)
• ?????????????????????????????????????????????????
  ???????????????? ?????????????????????????????????
  ???????????????????????????????????????????
  (????????????????????)

10
Back Propagation
11
Basic Concepts of Neural Networks

• ????????????? (Processing elements (PEs))
• ??????????????
• ????????????????
• ???????????????????? (??? 3 ???? ???? three
  layers)
• ?????? (Input)
• ???????? (Intermediate layer) ???? ???????
  (hidden layer)
• ???????? (Output)

12
Basic Concepts of Neural Networks
13
Basic Concepts of Neural Networks

• ?????????????????? (Parallel processing)
• ?????????????????????????????????????????????????
  ????????????????????????? ? ??????????????????????
  ? ???? ?????????????????????????? ?

Six specialized vector processors (SPUs)
14
Basic Concepts of Neural Networks

• ???????????????????????????
• ?????? (Inputs)
• ???????? (Outputs)
• ????????????????????????????? (Connection
  weights)
• ?????????????? (Summation function) ????
  ??????????????????????(Transformation function)
  ???? ?????????????????? (Transfer function)

15
Basic Concepts of Neural Networks

• ????????????????????????????? (Connection
  weights)
• ????????????????????????????? link
  ?????????????????????? ???????????????????????????
  ???????????????????????????????? (neural networks
  learning algorithm)
• ?????????????? (Summation function) ????
  ??????????????????????(Transformation function)
  ???? ?????????????????? (Transfer function)
• ???????????????????? ?????????????? (???????)
  ?????????????? ( transform) ?????????????????????
  fire (????????????????) ??????????????????????????
  ? internal activation level ??????????????????????
  ?????

16
Basic Concepts of Neural Networks
17
Basic Concepts of Neural Networks

• ???????????????? (Sigmoid (logical activation)
  function)
• S-shaped transfer function ??????????????????? 0
  ??? 1
• ??????????? (Threshold value)
• ????????????????????????????????????????
  (trigger) ??????????????????? ????????????????????
  ??????????????????????? ??????????????????????????
  ???????????????????? (???????????????
  ???????????????????????????)
• ??????? (Hidden layer)
• ???????? (middle layer) ?????????????????????????
  ???????????????????????????

18
(No Transcript)
19
Sigmoid Function
20
Basic Concepts of Neural Networks
Y Sum of (wixi) (3(0.2) 1(0.4) 2(0.1))
1.2 Transfer function 1/(1exp (-x)) YT
1/(1exp(-1.2)) 0.77
21

• ???????????????????? ?????????????? Threshold
  0.8 ???? ?????????????? Output ?????
• ????????????????? Threshold 0.8 ????
  ??????????? Output ?????

22
Basic Concepts of Neural Networks

• ????????????????????????????
• ???????????????????????????????? ? ????????????
• Back propagation
• Feed forward (or associative memory)
• Recurrent network

23
Basic Concepts of Neural Networks
24
Basic Concepts of Neural Networks
25
Learning in ANN

• Learning algorithm
• ??????????????????????????????? artificial
  neural network
• Supervised learning
• ???????????????? (training) ?????????????????????
  ??????????????????????????????????????????????????
  ???????? weights ?????????????????????????????????
  ?????????????????

26

• Unsupervised learning
• ?????????????????????????????????????????????????
  ????????????????????????????????
  ??????????????????????????????????????????????????
  ?????

27
Learning in ANN

• Self-organizing
• ?????????????????????????????????????????????????
  ? unsupervised learning
• Adaptive resonance theory (ART)
• ???????? unsupervised learning ???????????????
  Stephen Grossberg ????????????????????????????????
  ??????????????????????????????????????????????????
  ???????????????? unsupervised mode
• Kohonen self-organizing feature maps
• ??????????????????????????????????????? machine
  learning

28
Learning in ANN
29
Learning in ANN

• ???????????????????? ANN ?????????
  ???????????????????????
• ????? temporary outputs
• ??????????? temporary outputs ????????? desired
  targets
• ????????? weights ?????????????????

30
Learning in ANN
31
Learning in ANN

• ?????????????? (Pattern recognition)
• ??????????????????? matching ???????????????
  (external pattern) ???????????????????????????????
  ????????????????? ???????????? inference
  engines, image processing, neural computing, ???
  speech recognition (????????????????????????????
  ????????????? classify data ??????? predetermined
  categories)

32
Learning in ANN

• ???????????????????????
• ???????????????? (Learning rate)
• ???????????????????????????????????????
  ??????????????????????????????????????????????????
  ???????? offset ?????
• ???????? (Momentum)
• ??????????????????? (learning parameter)
  ??????????????????? feedforward-backpropagation

33
Learning in ANN

• Backpropagation
• ?????????????????????????????????????????????????
  ???????????????????? ?????????????????????????????
  ???????????????????? (computed output)
  ????????????????????? (desired outputs)
  ??????????? historical cases

34
Learning in ANN

• ??????????????????????????
• ????????????? learning algorithm
• ??????????????????? weights ?????? random values
  ?????????????????? ? ?????????????
• ???? input vector ??? desired output
  ????????????????
• ???????????????????????????????????????? layer
  ???? ? ???????????????????????? ? ????? (actual
  output)
• ????????????????????? (error)
• ????????????????????????? ? (weights)
  ??????????????????? (working backward) ??? output
  layer ????? hidden layers

35
Developing Neural NetworkBased Systems
36
Developing Neural NetworkBased Systems

• ?????????????????????????????? (Data collection
  and preparation)
• ??????????????????????????????????? (training and
  testing) ?????????????????????????????????????????
  ??????????
• ???????????????????????????? (Selection of
  network structure)
• ??????????? topology ????? ?
• ???????? (Topology)
• ?????????????????????????????????????????????????
  ????????? (???????????????????????????????????????
  ????????????????)

37
Developing Neural NetworkBased Systems

• ???????? topology
• ?????????????????
• ?????????????????? (Input nodes)
• ???????????????????? (Output nodes)
• ????? hidden layers
• ???????????????? hidden layer

38
Developing Neural NetworkBased Systems

• ???????????????????????????? (Learning algorithm
  selection)
• ??????? set of connection weights ???????????
  training data ???????? ????????? best predictive
  accuracy
• ?????????????? (Network training)
• ?????????????? ? ??????????? a random set of
  weights ??? ???? ? ???????????????????????????????
  ???????????????????????????????????????????
  (??????? ??????????????????? ?????????????????????
  ??????)
• ??????????????????????????????????????????
  (?????????????????????????????????????????????????
  ?????????) ???????????????????????????????????????
  ?????????

39
Developing Neural NetworkBased Systems

• ???????? (Testing)
• Black-box testing
• ?????????????????????????????????????????????????
  ?????????????????????
• ????????????????????? routine cases ???
  potentially problematic situations
• ????????????????????????? ?? large deviations
  ??????????????????????????????????????????????????
  ?????? ???????????????????????????????????????????
  ????????????????????????????????

40
Developing Neural NetworkBased Systems

• ????? ANN ????????
• ??????????????????????????????????????????????????
  ???????????????????????? ? ???????????????????????
  ??????????????
• ???????? Ongoing monitoring ??? feedback
  ??????????????????????????????????????????????????
  ??????????????
• ??????????????????????????????????????????????????
  ??????????????????????????????????????????????????
  ??????????????????????????????????????????????????
  ?

41
Developing Neural NetworkBased Systems
42
A Sample Neural Network Project
43
Other Neural Network Paradigms

• Hopfield networks
• A single large layer of neurons
  ????????????????????????????????? (total
  interconnectivity) ??????? ???????????????????????
  ???????????? ? ??????
• ????????????????????????????????????????????????
• ????????????????? Hopfield networks ?????
  ?????????????????? constrained optimization
  problem ???? classic traveling salesman problem
  (TSP)

44
Other Neural Network Paradigms

• Self-organizing networks
• Kohonens self-organizing network ??????????????
  unsupervised mode
• ????????????? Kohonen ??????????????? feature
  maps ??????? neighborhoods of neurons
  ??????????????
• Neighborhood ???????????????????????????????
  topology ?????????????????????????????????????????
  ????????????????????
• Self-organizing maps ???? self organizing feature
  maps ?????????????????????????? some early
  insight into the data

45
(No Transcript)
46
Applications of ANN

• ANN ??????????????????? ? ?????????????????
  categorical ??? numeric ??????????????????????????
  ??????????????????????????????????? ????
  ???????????????????????????????????????

47
??????????? 8

• ???????????????.