Shreekanth Mandayam

1
Artificial Neural Networks ECE.09.454/ECE.09.560
Fall 2008
Lecture 11November 17, 2008

• Shreekanth Mandayam
• ECE Department
• Rowan University
• http//engineering.rowan.edu/shreek/fall08/ann/

2
Plan

• ANN Pre-processing
• Feature Extraction
• Approximation Theory
• Universal approximation
• Final Project Discussion

3
Feature Extraction

• Objective
• Increase information content
• Decrease vector length
• Parametric invariance
• Invariance by structure
• Invariance by training
• Invariance by transformation

4
Approximation TheoryDistance Measures

• Supremum Norm
• Infimum Norm
• Mean Squared Norm

5
Recall Metric Space

• Reflexivity
• Positivity
• Symmetry
• Triangle Inequality

6
Approximation Theory Terminology

• Compactness
• Closure

F
7
Approximation Theory Terminology

• Best Approximation
• Existence Set

E
ALL f
M
min
u0
8
Approximation Theory Terminology

• Denseness

9
Fundamental Problem
E
M
min
g
?

• Theorem 1 Every compact set is an existence set
  (Cheney)
• Theorem 2 Every existence set is a closed set
  (Braess)

10
Stone-Weierstrass Theorem

• Identity
• Separability
• Algebraic Closure

F
afbg
11
Final Project Discussion