Title: Perceptual Organization of Curvilinear Structures
1Perceptual Organization of Curvilinear Structures
Laurent Alquier Research Director Chabane
Oussalah Professor Thesis Advisor Philippe
Montesinos Assistant Professor
- UNIVERSITE MONTPELLIER II - - Laboratoire de
Génie Informatique et d 'Ingénierie de
Production, Nîmes - - September, 30th 1998 -
2Image Analysis
Construction of a symbolic representation
Detection of visual cues
3Visual Perception
- Theories of Visual Perception
- Continuous flow of visual information
- Necessity to guide perception
4Perceptual Organization
- Principles
- Pre-attentive phenomenon of grouping
- Global visual properties Saliency
- Imposed to perception before interpretation
- Elementary groups
- Principle of "good shape"
- Simplicity, closure, familiarity Stability
- Useful properties
- Generic, robust - Qualitative organization
References
- Gestalt Theory Wertheimer, Koffka - 1923
5Overview
- Introduction.
- Context and Objectives.
- Organization using Saliency Networks.
- High Levels of Organization.
- Contributions and Perspectives.
6Objectives
- Complete system of image analysis
- Segmentation - hypotheses - interpretation
- Application of psycho-vision principles
- Analysis of complex scenes
- Restriction to shapes from contours
- Validation with artificial scenes
- Application to images of real scenes
- Goal
- Extract a set of elements of representation
- Remain open to possible needs from future
applications
7Overview of the system
- Edge detection
- Three levels of organization
- Selection of salient structures
- Extraction of elementary hypotheses
- Organization into complex hypotheses
- Hierarchic relationship between hypotheses
- Application
- Detection of junctions
- Structural matching
8Overview
- Introduction.
- Context and Objectives.
- Organization using Saliency Networks.
- High Levels of Organization.
- Contributions and Perspectives.
9Detection of Curvilinear Structures
- Goal
- Select the most regular contours
- Complete discontinuities
- Extract stable structures ( noise, scale )
10Grouping by Saliency
- Principles
- Estimate the visual quality of a structure
- optimise this quality function
- Direct Approach
- Extension Fields (Guy and Medioni, 1996)
- Stochastic Completion Fields (Williams and
Jacobs, 1994) - Approaches by optimisation
- Neural Networks (F. Mangin, 1994)
- Mean Field Annealing (L. Hérault, 1991)
- Saliency Network (Shashua et Ullman, 1989)
11Saliency Network
12Saliency Network (2)
- Selection of a Primitive
- Definition of visual properties.
-
- Definition of Local Neighborhood
- Network of locally connected elements.
- Quality function for a group
- Estimate compatibility between elements of a
group. - "Extensible" functions.
- Definition of Saliency
- Quality of the best possible group for a given
primitive
13Local Neighborhood
- Relationships between primitives.
- Primitives linked using Elements of Connection.
- Properties of Proximity and Compatibility.
- Method suited for curved groups.
- Importance of neighborhood.
- Initialize optimisation of the network.
- Sets complexity of optimisation.
- Influence visual quality of groups.
14Quality Function
- Properties of Continuity
- Proximity, smoothness, similarity.
- Formalism
- Linear combination of opposed constraints.
- "Internal" relationships.
- Visual properties of groups expected.
- "External" relationships.
- Imposed by contours from image.
Ã¥
Ã¥
G
a
a
)
F(
R
R
j
j
k
k
ÃŽ
ÃŽ
external
internal
j
k
15Structural Saliency
- Recursive expression
- F is supposed to be an "extensible function"
- Formal definition.
- Best sum of contributions around a primitive
according to two directions.
16Structural Saliency (2)
- Example of recursive expression
- Local term.
- Contribution of neighbors.
- Iterative optimisation.
- Research of best neighbors.
- Update of contributions
17Structural Saliency (3)
- Properties
- Local measurements - Global optimisation.
- Completion of discontinuities.
- Saliency Map.
- Grouping possible by following connections.
- Only one optimal group for each primitive.
18Structural Saliency (4)
- Application
- Organization of Pixels.
- Static neighborhood.
- Heavy computation.
- Slow optimisation.
- Organization of Chains.
- Dynamic Neighborhood.
- Reduced complexity.
- Fast optimisation.
19Consequence of grouping
Organization of chains
Normal edge linking
20Detection of Salient Groups
21Elementary Groups
- Classes of groups
- Starting points along a global structure.
- "Attraction" phenomenon.
- Evaluation of groups
- Local saliency.
- Global saliency ( sum of saliency of primitives )
- Accumulation of votes
- Selection
- Threshold from evaluations.
22Elementary Groups (2)
23Overview
- Introduction.
- Context and Objectives.
- Organization using Saliency Networks.
- High Levels of Organization.
- Primary Hypotheses.
- Complex groups and applications.
- Contributions and perspectives.
24Primary Hypotheses
- Structural Hypotheses
- Straight parts Segments.
- Curved parts Arcs.
- Special points Junctions, inflection points,
corners. - Principles of extraction
- Detection from each elementary group
- Scale, sensitiveness.
- Fusion of primary hypotheses.
- Similarity - elimination of duplicates.
25Primary Hypotheses (2)
- Elements of symbolic representation
- Set of Hypotheses
- Role of ambiguities and errors.
- Certain amount of duplicates tolerated.
- Problems to solve
- Discretization
- Structures with different scales within the same
scene. - Output
- Similar groups superimposed
- Exceptions to properties of continuity
(occlusions and junctions)
26Hypotheses - Segments
Salient Groups
Recursive division (tolerance Es)
Elementary segments
Fusion of similar segments (length,orientation)
Points of interest
Grouped segments
27Segments ( before organization )
28Segments ( after organization )
29Hypotheses - Arcs
Salient Groups
Division according to Extremes of Curvature
(Scale Ea)
Classes of Elementary Arcs
Fusion of similar arcs (classe,superposition)
Points of Interest
Grouped Arcs
30Arcs ( after organization )
31Points of Interest
32Overview
- Introduction.
- Context and Objectives.
- Organization using Saliency Networks.
- High Levels of Organization.
- Primary Hypotheses.
- Complex groups and applications.
- Contributions and perspectives.
33Complex groups
- Example of application
- Extraction of multiple junctions
- Structural matching of junctions
- Motivations
- Rich structural information
- Location of center, orientation of branches
- Robust matching possible
- Precise location possible after matching
- Structures difficult to extract
- Alterations from contour detection
- Few publications in that area
34Organization of junctions
Segments
Points
Intersections
Confirmation of center
Detection of Double Junctions
Fusion of Similar Junctions (proximity of
centers, similar branches)
Multiple Junctions
35Detection of double junctions
36Fusion into multiple junctions
37Structural Matching
- Principle
- Consistent labeling of two sets of junctions
- Requires
- Direct comparison of junctions
- Evaluate motion (transformation)
- Between two images
- Within a single image
- Conditional comparison
38Structural Matching (2)
- Method in two stages
- Temporal Matching
- Elimination of improbable matches
- Spatial Matching
- Elimination of improbable groups
- Properties
- Matching as perceptual grouping
- Tolerate important differences between images
- Mutual reinforcement of two type of organization
39Structural Matching (3)
40Structural Matching (4)
41Overview
- Introduction.
- Context and Objectives.
- Organization using Saliency Networks.
- High Levels of Organization.
- Contributions and perspectives.
42Contributions
- Saliency Networks
- Generic Formalism
- Generalized to Organization of Chains
- Configurable Quality Function
- More stable optimisation
- Choice of the most important connections
- Extraction of final groups
- Selection of the most salient structures
References
Shashua and Ullman, 1991 - Alter and Basri, 1997
43Contributions (2)
- Organization of Geometric Hypotheses
- Segments
- Arcs
- Points of Interest
- Modular approach
- Global strategy of organization
- Specialized modules
- Hypotheses defined for only one scale
- Application to numerous types of scenes
References
Mohan and Nevatia, 1992 - Sarkar and Boyer, 1993
- Gao and Wong, 1993
44Contributions (3)
- Perceptual Organization of Junctions
- Detection of Elementary Junctions
- Organization into Multiple Junctions
- Structural Matching
- Cooperation between Spatial Perceptual
Organization and Temporal Matching.
References
Matas and Kittler, 1993 - Chang and Aggarwal, 1997
45Results
- Stable results
- Generic parameters for classes of scenes
- Robust in case of perturbations
- Reasonable computation time on low-end systems
- Example
- PC - Pentium 100 - 65 Mo RAM
- Image 800x600 pixels
- 500 chains in a scene
- Grouping with Saliency Network 30s
- Global computation time 5 min
46Application Outdoor Scenes
47Application Satellite Imaging
48Junction Matching
49Junction Matching (2)
50Short term extensions
- Organization of Generic Objects
- Quantitative evaluation of results
- Qualitative results only for now
- Numerous parameters
- Empirical definition for now
- Automatic validation of hypotheses
- Multi-scale detection
- Hierarchical Structural Matching
- Top-down approach
51Perspectives
- Integration with other visual cues
- Visual attention - tracking
References
Laurent Iti, 1997 Caltech - Roch and Ullman, 1985
52Perspectives (2)
- Automatic Indexation of Models
- Aspect graphs
References
Pope and Lowe, 1996