Where are we going

1
Where are we going?

• In most cases, and pretty much everything we will
  do in this class, NNs are used as either
  classifiers or function approximators.
• Classifier
• For an object with a given set of input
  features, what class is it in?
• Function Approximator
• For an input X(a feature vector), what should be
  the output as a function of X?

2
Hyperplanes Dot Products

• In an n-dimensional space, a hyperplane is a
  function of n variables, e.g.
• F(x,y,z) ax by cz d 0
• i.e. any point (x,y,z) on the plane evaluates to
  F(x,y,z) of 0
• Thus, we say that the hyperplane function is
  defined by specifying n1 parameters. In this
  example, giving a value for a, b, c, and d
  defines the hyperplane

3
Hyperplanes Dot Products

• Note that up to n-1 of these parameters can be 0.
  
• While both xy3 and xy-30 are equivalent, we
  usually use xy-30
• A hyperplane defines a set of values (infinite)
  for the n dimensions for which the function
  evaluates to a 0

4
Hyperplanes Dot Products

• For a point A (x,y,x) and a hyperplane
  F(x,y,z),
• if F(A)gtF(x,y,z) 0, then the point
  (x,y,z) must lie on the
• hyperplane.
• The two other possibilities are that
• F(x,y,z)gt0 or
• F(x,y,z)lt0
• Note points on the same side of the hyperplane
  will give a value for F of the same sign

5
Hyperplanes Dot Products

• How do we know where a point is in relation to a
  hyperplane?
• Given a 2D space and a hyperplane defined as
• F(x,y)7.5x 2.3y -18 0
• Where is the point P(1.2, 8.3)?

6
Hyperplanes Dot Products

• By substituting P(1.2, 8.3) into the equation
  F(1.2,8.3)7.5(1.2) 2.3(8.3) -18 we get
• 10.09
• This tells us that at least P is on the
  positive side of F

7
Question

• What happens, or what is the effect of
  multiplying F by -1?
• Just swaps the positive and negative sides

8
Dot Product

• Generally, linear equations for hyperplanes are
  represented as vectors.
• F(x,y,z)6x2y-3z-150 becomes
• F6
• 2
• -3
• -15

9
Dot Product

• Using vector form we can determine the sign for
  a point with respect to a hyperplane as follows
• F(P) FT P gt dot product
• Tgt transpose
• F(P) 6 2 3 -15 1.2 61.228.33(-
• 8.3 1.2)(-15)1 5.2
• -1.2
• 1

10
Neurons and Hyperplanes

• Consider a neuron whose activation function is a
  step function

11
Hyperplanes

• Consider a line defined as F(x,y)gt y-2x0
• What is the hyperplane

12
Transfer Functions

• In addition to the step functions, the more
  common ones are the log sigmoid and the
  hyperbolic tangent

13
Stochastic transfer Function
14
Distance Measures

• How does one measure the similarity between two
  feature vectors?
• Euclidean distance
• Dot product