Visual Object Recognition as Paradigm of Brain Function

1
Visual Object Recognition as Paradigm of Brain
Function
Christoph von der Malsburg Inst. f.
Neuroinformatik, Ruhr-Universität
Bochum, Computer Science, Neuroscience, USC, Los
Angeles FIAS, Frankfurt
2
Mind function
Autonomous Goal Pursuit Instincts, drives,
emotions anima Intentionality relating to the
environment Perception representation,
interpretation, epistemology Action Consciousness
Subsystem Coordination Intelligence
Generalization abstract schemata applied to
concrete situations
3
Matter material implementation
How does Matter. (molecules, synapses, neurons,
circuits, systems, gross anatomy) .Correspond to
Mind? What is the Data Structure of Brain
State? of Memory?
4
Organization
Coordination of molecules, cells, into one
coherent system Evolution beyond intelligent
design and blind search Ontogenesis little
genetic information for much structure 109 bits
in the genome, 1016 bits in the wiring diagram
parametrically controlled sequence of events
of SO! Learning accumulation of memory traces
unsolved problem!!
5
Organization (contd.)
State organization SO of scene representation synt
hesized from memory traces quick form of
ontogenesis input as analogon to control
genes Challenges coordination of subsystems
(consciousness!) generalization (intelligence!)
6
Discours de la méthode
Science has to be portable, simple!! teachable,
paraphrasable, reproducible, structurally
stable The importance of paradigms free fall,
two-body motion, ideal gas, harmonic oscillator,
hydrogen atom, regular crystal, Ising model
(phase transition) Invariant object recognition
as paradigm Mathematics essence bridging
description levels 1) differential equations plus
stability analysis 2) statistical mechanics
7
Computer and Brain
Computer simulation as tool as acqua regia to
separate the wheat from the chaff connecting
mind-phenomena and brain-phenomena immediate
technical applications Computer as model for the
brain Artificial Intelligence whose
intelligence machine or man? Turing
universality ready to be programmed Brain
universality autonomous problem solving The
computer is not an end but a tool
8
Feynmans 1959 Lecture
There's Plenty of Room at the Bottom An
Invitation to Enter a New Field of Physics
Miniaturizing the computer If I look at your
face I immediately recognize that I have seen it
before. (....) Yet there is no machine which,
with that speed, can take a picture of a face and
say even that it is a man and much less that it
is the same man that you showed it
before---unless it is exactly the same picture.
If the face is changed if I am closer to the
face if I am further from the face if the light
changes---I recognize it anyway. Now, this little
computer I carry in my head is easily able to do
that. The computers that we build are not able to
do that. The number of elements in this bone box
of mine are enormously greater than the number of
elements in our wonderful'' computers. But our
mechanical computers are too big the elements in
this box are microscopic. I want to make some
that are submicroscopic.
9
Invariant Object (Face) Recognition
Very competitive field (benchmarks, indust.
appl.) Attention control Figure-ground
separation Correspondence finding under
variance position, orientation, scale, pose,
deformation, illumination, surface markings,
partial occlusion, background, noise Learning
10
Attention, Figure-Ground Separation
Xiangyu Tang
11
Object recognition 1
Image Domain
Model Domain
Model Window
12
van Essen - Felleman
13
Graph matching
Correspondence-based Object Recognition
a, b, c, d,.. feature types
Image Domain
Model Domain
14
Devalois 1
15
Gabors
16
Devalois 2
17
Image-to-jets
18
Maryl-representation
19
Maryl-reconstruction
20
Similarity matrix
Laurenz Wiskott
21
2D map formation
Junmei Zhu
22
Maryl-match
23
Bunch graph method
Laurenz Wiskott
24
Bunchgraph Rekonstrution
Reconstruction from 70 Reference faces
Left Original Right Reconstruction
25
Bunch graph gender
26
Visual Learning Aspects of the Problem

• Identification of significant patterns
• Extraction of examples
• Finding more examples
• Consolidation of the representation

27
One-Shot Learning
Hartmut Loos
28
Image to graph
Hartmut Loos
29
Bottles found
30
One person found
Hartmut Loos
31
More persons
Hartmut Loos
32
Face Finding
33
Analysis
34
Synthesis
Parameterized model
35
PCA schema
Principal Component Analysis (PCA)
36
PCA faces
Jan Wieghardt
37
Pose-estimation
38
PCA Nonlin
Kazunori Okada
39
Pose-reconstruction
Kazunori Okada
40
Gestures Samples
Hai Hong
41
Parameterized Gesture Model
Andreas Tewes
42
Object Recognition as Paradigm
Implementation Perception Action Goals
Intelligence Consciousness Organization
43
The Importance of Dynamic Links
Classical Data Structure Vector of Unit
Activities Units (groups of) neurons as
elementary symbols Problem No Structure! Dynamic
Link to represent Relatedness a and b relate
to the same object relate to neighboring points
(in an object) correspond to each other Physical
Representation Environment causal
connection Brain fiber connection Primitive
Observation temporal signal correlation Links
have to be dynamic as part of brain state
44
Graph matching
How implement dynamic links?
45
Temporal binding
Dynamic Links I

• Temporal binding

Time
10 msec

• Rapid, Reversible Synaptic Plasticity

46
Network Self-Organization
Network
Signals
Signal Dynamics
Synaptic Plasticity
The brain is dominated by attractor
networks Among them are 2D Aspects of
objects Homomorphic (Correspondence) mappings
47
DLM vs. MCU

• Implementation
• Dynamic Links Represented by
• I Switching Synapses
• Based on basic mechanisms of organization
• Simple structural preconditions
• Possible early in ontogeny
• Slow, fragile (very large search space)
• II Switching Neurons (control units)
• Fast, reliable (small search space)
• Needs very specific connectivity patterns
• Storage and retrieval of DLM Networks

48
Link control units
Dynamic Links II
Jörg Lücke
49
Perception, Action
50
Goals
51
Intelligence
Application of abstract schemata to concrete
situations
52
Raven 1
53
Raven 2
54
Raven 3
55
Graph matching
Concrete situation
Abstract schema
56
Consciousness
I am conscious if I can speak about it remember
it picture it act on it in short have all my
five senses together Consciousness is tantamount
to coordination (interfacing) of
subsystems! Vision (recognition) is prime example
of an interface
57
Analysis
58
Synthesis
59
Autonomy of organization
Complete organizational chain (evolution)
ontogenesis, learning, state organization Ultrasta
bility Autonomous regulation of control
parameters Definition of Organization