NEURAL NETWORKS

1
NEURAL NETWORKS

• IS3301 Session 19a
• Prof. Mark Nissen

2
Agenda

• Vignette John Deere
• Machine Learning
• Neural Nets
• Target ID/Tracking Example
• Learning Exercise
• Summary

3
Vignette John Deere

• 5B pension fund
• Improve returns via ANN
• Data from Fortune-1000
• Predict rank stock performance
• Rarely hear about successful trading ANNs?
• Many successful trading ANNs

4
Machine Learning

• Increase knowledge in systems
• Generalize across specific instances
• Knowledge enhances problem solving
• Learning enhances knowledge
• Rule-based problems
• Expert not always available/willing/able
• Knowledge engineering bottleneck
• No one knows how to solve problem
• Static knowledgebase, dynamic environ

5
Machine Learning Approaches

• Symbolic
• Induction - rules from examples (ID3/VPX)
• Chunking - generalize rules (Soar)
• Analogy - reason from past cases (CBR)
• Explanation - generalize with theory (EBL)
• Numerical
• Statistics - fit models to data (regression)
• Neural nets - fit network to data (ANN)
• Genetic algorithms - evolve pop to fit data
• Supervised training most common

6
Neural Nets

• Brain metaphor - parallel computing
• Subsymbolic representation
• Excellent at pattern matching
• Solve complex problems
• Too complex for people (experts)
• Highly nonlinear dynamic
• Adapt to changing environment (slowly)
• Intensive data time needs
• Development as art not craft/science

7
ANN Topology
Data Input Hidden
Output Solution
WI11
WM11
WM31
WI63
8
Target ID/Tracking Example

• Target ID/tracking, coordinate grid
• Image data input (grid cells, pixels)
• Train ANN to model image data
• Predict outcome - head-on or side
• also speed, IFF, next maneuver, etc.

1 2 3 4 5 6 7 8
9
9
Aspect Example
Data Input Hidden
Output Solution grid-darkness
1 head-on
(0,1)
0 side view
WI42
4
G4
2
WI52
5
G5
WI62
6
G6
Simple ANN spreadsheet implementation a
10
Learning Example

• Learn English vowels (5 letters)
• Binary grid representation (9 squares)
• Data vectors (e.g., 010010010 I)
• Make ANN representation
• Which links/weights most discriminating?
• Simplify network
• How many values needed to ID letter?
• Work in project teams
• Write as decision tree or IF-THEN rules

11
Summary

• Learning d/dx knowledge
• Increase knowledge in systems
• Overcome rule-based system problems
• Solve very difficult problems
• ANN uses neural metaphor
• Subsymbolic representation
• Supervised learning
• Excellent at pattern recognition
• Military, investment, other examples