Olshausens Demo

1
Olshausens Demo
2
How Important Is

• The Training set ?
• Natural Images (Olhausens database)
• How much do we learn ?
• face database and car database
• The Sparseness term ?
• Prior steepness
• Sparseness function
• Natural encoding or hacking?
• Whitening the data
• Non-stationary hypothesis

3
Training with Natural Images

• Training 10 images (512x512)
• 10,000 presentations

• Batch size 100
• Basis Function 16x16

4
Face Database

• Training 100 images (100x100)
• 10,000 presentations

• Batch size 100
• Basis Function 16x16

5
Encoding Properties
Original
10 basis
20 basis
50 basis
30 basis
40 basis
6
Car Database

• Training 200 images (128x128)
• 10,000 presentations

• Batch size 100
• Basis Function 16x16

7
Comments

• The algorithm seems to capture the structure of
  the images (cf car)
• Learning is experience-dependent
• Basis functions found in good agreement with
  properties of neurons in visual cortex
• Receptive fields are localized, oriented, bandpass

8
How Important Is

• The Training set ?
• Background, face and car databases
• The Sparseness term ?
• Prior steepness
• Sparseness function
• Natural encoding or hacking?
• Whitening the data
• Non-stationary hypothesis

9
Prior Steepness
Steepness 2.2
Steepness 5
Steepness 10
Steepness 100
10
Prior Steepness
Steepness 2.2
Steepness 1.5
Steepness 0.2
11
Sparseness Function
12
Sparseness Function
13
Sparseness Function
S(x)log(1x2)
S(x)x
14
Sparseness Function

• batch of 100 samples
• Mean Error abs.471 / log .504

15
How Important Is

• The Training set ?
• Background, face and car databases
• The Sparseness term ?
• Prior steepness
• Sparseness function
• Natural encoding or hacking?
• Whitening the data
• Non-stationary hypothesis

16
Whitening the Data
Data are filtered with whitening/low-pass filter

• How important is it for the convergence of the
  algorithm?
• The question is to know whether it is just a
  speed-up or is it required for convergence?

17
Non-preprocessed Car Images

• Training 100 images (100x100)
• 30,000 presentations

• Batch size 100
• Basis Function 16x16

18
Non-stationary HypothesisEncoding the Full Face
After few iterations
19

• Code images available http//web.mit.edu/serre/
  www/