Nonlinear models for Natural Images

1
Nonlinear models for Natural Images

• Urs Köster Aapo Hyvärinen
• University of Helsinki

1
2
Overview

1. Limitations of linear models
2. A hierarchical model learns Complex Cell pooling
3. A Horizonal product model for Contrast Gain
   Control
4. A Markov Random Field generalizes ICA to large
   images

2
3
1. Limitations of ICA image models

• Natural images have complex structure, cannot be
  modeled as superpositions of basis functions
• Linear models ignore much of the rich
  interactions between units
• Modeling the dependencies leads to more abstract
  representations
• Variance dependencies are particularly obvious
  structure not captured by ICA
• Model with (complex cell) pooling of filter
  outputs - hierarchical models
• Alternative Model dependencies by gain control
  on the pixel level

Schwartz Simoncelli 2001
4
Gain Control in Physiology

• Divisive normalization is common throughout the
  brain, see e.g. the normalization model for
  primary visual cortex
• Previous work has analyzed the effect of gain
  control on the cortical level
• We use a statistical model for gain control on
  the LGN level
• Important effects on subsequent cortical
  processing

Normalization modelCarandini and Heeger,1994
5
A hierarchical model estimated with Score
Matching learns Complex Cell receptive fields
6
2. Two Layer Model estimated with Score Matching

• Define an energy based model of the form
• Squaring the outputs of linear filters
• Second layer linear transform v
• Nonlinearity that leads to a super-gaussian pdf.
• Cannot be normalized in closed form. Estimation
  with Score Matching makes learning possible
  without need for Monte Carlo methods or
  approximations

6
7
Results

• The second layer learns to pool over units with
  similar location and orientation, but different
  spatial phase
• Following the energy model of Complex Cells
  without any assumptions on the pooling
• Estimating W and V simultaneously leads to a
  better optimum and more phase invariance of the
  higher order units

Some pooling patterns
8
Learning to perform Gain Control with a
Horizontal Product model
9
Multiplicative interactions
A horizontal network model
Horizontal layers

• Two parallel streams or layers on one level of
  the hierarchy
• Unrelated aspects of the stimulus are generated
  separately
• Observed data is generated by combining all the
  sub-models

• Data is described by element-wise multiplying
  outputs of sub-models
• Can implement highly nonlinear (discontinuous)
  functions
• Combine aspects of a stimulus generated by
  separate mechanisms

10
The model

• Definition of the model
• Likelihood
• Constraints B and t are non-negative, W
  invertible
• g(.) is a log-cosh nonlinearity (logistic
  distribution)
• t has a Laplacian sparseness prior

10
11
Results
First layer W 4 units in B
First layer W 16 units in B
12
Second Layer Contrast Gain Control

• Emergence of a contrast map in the second layer
• It performs Contrast Gain Control on the LGN
  level (rather than on filter outputs)
• Similar effect to performing divisive
  normalization as preprocessing
• The model can be written as
• Something impossible to do with hierarchical
  models

True image patches
Reconstruction from As only
Modulation from Bt
13
The big picture A Markov Random Field
generalizes ICA to arbitrary size images
14
4. Markov Random Field

• Goal Define probabilities for whole images
  rather than small patches
• A MRF uses a convolution to analyze large images
  with small filters
• Estimating the optimal filters in an ICA
  framework is difficult, the model cannot be
  normalized
• Energy based optimization using Score Matching

15
Model estimation

• The energy (neg. log pdf) is
• We can rewrite the convolution
• where xi are all possible patches from the image,
  wk are the different filters
• We can use score matching just like in an
  overcomplete ICA model
• The MRF is equivalent to overcomplete ICA with
  filters that are smaller than the patch and
  copied in all possible locations.

16
Results

• We can estimate MRF filters of size 12x12 pixels
  (much larger than previous work, e.g. 5x5)
• This is possible from 23x23 pixel images, but
  the filters generalize to images of arbitrary
  size
• This is possible because all possible overlaps
  are accounted for in the (2 n -1) size
  image
• Filters similar to ICA, but less localized
  (since they need to explain more of the
  surrounding patch)
• Possible applications in denoising and filling-in

16