Nonlinearity in Primary Visual Cortex

1
Nonlinearity in Primary Visual Cortex
Informal talk
Figure Vision A window on consciousness,
Scientific American, Nov 1999

• Mark V. Albert nonlinear systems IGERT group
  Feb 4, 2005

2
Why learn about visual cortex?

• More efficient object recognition
• Fast within 200 ms
• Robust invariant to lighting, location, size,
  viewing angle
• Energy consumption roughly 20 Watts (entire
  brain)
• Medical application
• Basic research/philosophical reasons

3
Intro to visual cortex

• Retina ? LGN ? V1
• Primary visual cortex
• Dorsal stream
• where pathway
• Ventral stream
• What pathway

4
More ventral stream
Wiskott, L. (2004). How Does Our Visual System
Achieve Shift and Size Invariance? In Problems in
Systems Neuroscience, eds. J. L. van Hemmen and
T. J. Sejnowski, publ. Oxford University Press
Adapted from Oram and Perrett, 1994
5
Why V1?

• First stage of processing in visual cortex
• Easiest to characterize
• Minimal attention effects from feedback
• Visual stimuli easy to manipulate
• Sufficient complexity
• Even the most linear neurons have complex
  nonlinearities, as we will see

6
First insights into V1 neural responses

• Receptive field basics
• Response to
• Points
• Bars
• Sinusoidal gratings
• Gaussian noise
• Simple vs complex cells
• (Hubel Wiesel, 1960s)

7
Linear receptive field model
2-D gabor functions explain linear properties
Figure Jones, J. P. Palmer, L. A. ,The
two-dimensional spectral structure of simple
receptive fields in cat striate cortex." In
Journal of Neurophysiology, vol. 58, no. 6, pp.
1187-1211, December 1987
8
Simple cells from natural image statistics
Olshausen BA, Field DJ (1997). Sparse Coding with
an Overcomplete Basis Set A Strategy Employed by
V1? Vision Research, 37 3311-3325.
9
Not the end of the story

• Linear models do not fully predict V1 stimulus ?
  response
• Neurons cannot respond negatively.
• The limited dynamic range of a neuron

10
Response Saturation
Figure Albrecht, D.G., Geisler, W.S., Crane,
A.M. (in press). Nonlinear properties of visual
cortex neurons Temporal dynamics, stimulus
selectivity, neural performance
11
But what about noise?

• Neurons in V1 spontaneously fire
• Phototransduction isnt perfect
• Effects at low contrast?...

12
Response expansion
Figure Albrecht, D.G., Geisler, W.S., Crane,
A.M. (in press). Nonlinear properties of visual
cortex neurons Temporal dynamics, stimulus
selectivity, neural performance
13
Are output nonlinearities the solution?

• No.
• Sub-optimal stimuli elicit responses that
  asymptote well before the top of the dynamic
  range

14
Contrast Gain Control
Figure Albrecht, D.G., Geisler, W.S., Crane,
A.M. (in press). Nonlinear properties of visual
cortex neurons Temporal dynamics, stimulus
selectivity, neural performance
15
Question the receptive field

• Classic Receptive Field
• The area in which stimuli can elicit responses
• But this does not define the area than can affect
  a neurons response!...

16
Contextual effects
Figure A Knierim JJ, Van Essen DC (1992)
Neuronal responses to static texture patterns in
area V1 of the alert macaque monkey. J
Neurophys., 67, 961-980.
Sillito AM, Grieve KL, Jones HE, Cudeiro J, Davis
J (1995) Visual cortical mechanisms detecting
focal orientation discontinuities. Nature, 378,
492-496.
17
Time

• Neurons are not meant to encode stationary
  stimuli
• Not efficient to respond constantly
• Changes elicit greater response.
• Temporal aspects should be included in a linear
  model

18
Spatiotemporal linear receptive field models
Gifs from Izumi Ohzawa, http//neurovision.berke
ley.edu/Demonstrations/VSOC/teaching/AA_RFtutorial
.html
19
Temporal nonlinearities?

• Onset of stimuli ? greater response
• Response time of V1 neurons depends on contrast

20
Latency shift
Figures Albrecht, D.G., Geisler, W.S., Crane,
A.M. (in press). Nonlinear properties of visual
cortex neurons Temporal dynamics, stimulus
selectivity, neural performance. Albrecht, D.G.,
Geisler, W.S., Frazor, R.A, Crane, A.M. (2002).
Visual cortex neurons of monkeys and cats
Temporal dynamics of the contrast response
function. Journal of Neurophysiology, 88, 888-913
21
Other types of cells in V1

• Nonlinearities thus far belong to simple cells as
  well as other types
• Simple cells
• excitatory bright and dark regions
• Orientation, position, frequency and phase
  selective
• There are other types of cells in V1
• Complex cells and Hypercomplex cells

22
Phase invariance traditional complex cells

• Complex cells
• Simple cells phase invariance

Figure Albrecht, D.G., Geisler, W.S., Crane,
A.M. (in press). Nonlinear properties of visual
cortex neurons Temporal dynamics, stimulus
selectivity, neural performance.
23
A Model Topographic ICA complex cells

• Hyvarinen and Hoyer

A. Hyvärinen P. O. Hoyer (2001) A Two-Layer
Sparse Coding Model Learns Simple and Complex
Cell Receptive Fields and Topography from Natural
Images. Vision Research, 41(18)2413-2423
24
hypercomplex cells one example End-stopping
End-stopping cell
Complex cell
Figures Pack, C.C., Livingstone, M.S., Duffy,
K.R., and Born, R.T. (2003) End-stopping and the
aperture problem Two-dimensional motion signals
in macaque V1. Neuron, 39, 671-680.
25
Nonlinearities in review
Simple Cells

• Complex Cells
• Phase invariance

• Response nonlinearities
• Saturation and expansion
• Spatial nonlinearities
• Contrast gain control
• Contextual surround effects
• Latency Shift

• Hypercomplex cells
• End-Stopping

Certainly not a complete list!