Object Recognition Based on Shape Similarity

1
Object Recognition Based on Shape Similarity

• Longin Jan Latecki
• Computer and Information Sciences Dept. Temple
  Univ., latecki_at_temple.edu
• Collaborators
• Zygmunt Pizlo, Psychological Sciences Dept.,
  Purdue Univ.,
• Nagesh Adluru, Suzan Köknar-Tezel, Rolf
  Lakaemper, Thomas Young, Temple Univ.,
• Xiang Bai, Huazhong Univ. of Sci. Tech. Wuhan,
  China

2
Object Recognition Process
Source 2D image of a 3D object
Object Segmentation
Contour Extraction
Contour Cleaning, e.g., Evolution
Contour Segmentation
Matching Correspondence of Visual Parts
3
Motivation

• Once a significant visual part is recognized the
  whole recognition process is strongly constrained
  in possible top-down object models.
• (H1) object recognition is preceded by, and based
  on recognition of visual parts.
• (H2) contour extraction is driven by shape
  similarity to a known shape.

4
What do you see?
5
With grouping constraints we can see (i.e.,
recognize the object).
6
Object contours

• Psychophysical and neurophysiological studies
  provide an abundance of evidence that contours of
  objects are extracted in early processing stages
  of human visual perception.
• Contours play a central role in the
  Gestalt-theory.

7
Salient visual parts can influence the object
recognition (Singh and Hoffman 2001)
8
Salient visual parts can influence the object
recognition (Singh and Hoffman 2001)
9
Salient visual parts can influence the object
recognition (Singh and Hoffman 2001)
10
Visual parts and shape similarity

• (H1) object recognition is preceded by, and based
  on shape recognition of visual parts.
• (H2) contour extraction is driven by shape
  similarity to a known shape. becomes
• (H2) Contour extraction is based on grouping of
  contour parts to larger contour parts with
  grouping assignments driven by shape familiarity.

11
Contour detection is a difficult inverse problem

• A given image could be produced by infinitely
  many possible 3D scenes. In order to produce a
  unique, stable and accurate interpretation, the
  visual system must use a priori constraints (see
  Pizlo, 2001 for a review).
• The solution is obtained by optimizing a cost
  function which consists of two general terms 1.
  how close the solution is to the visual data 2.
  how well the constraints are satisfied

12
Partial shape similarity
13
Partial shape similarity
Given only a part (of a shape ), find similar
shapes

• (1) length problem,
• (2) scale problem,
• (3) distortion problem

Query Shape
Target Shape
Target Shape
14
Partial Shape Similarity

• We reduce partial shape similarity to subsequence
  matching
• This is done by computing a curvature like value
  at every contour point.
• We do this for complete contours of known objects
  in our database
• and for query contour parts extracted from edge
  images

15
Subsequence Matching (shape similarity)
Database contours
Query contours
16
Motivation for subsequence matching
The top (red) and bottom (blue) sequences
represent parts of contours of two different but
very similar bone shapes
17
Motivation(2)
Example sequences a 1, 2, 8, 6, 8 b 1, 2,
9, 15, 3, 5, 9
18
OSB Algorithm

• Goal given two real-valued sequences a and b,
  find subsequences a of a and b of b such that
  a best matches b
• Possible to skip elements in both a and b
• The ability to exclude outliers
• Preserve the order of the elements
• A one-to-one correspondence

19
OSB Algorithm (2)

• Create a dissimilarity matrix
• No restrictions on the distance function d
• We used d(ai,bj) (ai bj)2
• To find the optimal correspondence, use a
  shortest path algorithm on a DAG

20
OSB Algorithm (3)

• The nodes of the DAG are all the index pairs of
  the matrix (i,j)?1,,m?1,,n
• The edge weights w are defined by
• C is the jump cost (the penalty for skipping an
  element)

21
OSB Algorithm (4)

• The edge cost may be extended to impose a warping
  window
• Set a maximal value for k i 1 and l j - 1
• This definition of the edge weights is our main
  contribution

22
A Simple Example
a 1, 2, 8, 6, 8 b 1, 2, 9, 15, 3, 5, 9
b b b b b b b
1 2 9 15 3 5 9
a 1 0 1 64 196 4 16 64
a 2 1 0 49 169 1 9 49
a 8 49 36 1 49 25 9 1
a 6 25 16 9 81 9 1 9
a 8 49 36 1 49 25 9 1
The dissimilarity matrix
d(ai,bj) (ai bj)2
23
The DAG
24
Experimental results on MPEG 7 dataset, 1400
targets in 70 classes
25
How to find contour parts in images?

• Humans group contours automatically
• An adaptive, probabilistic process to perform
  grouping
• All shapes contain local symmetry ? exploit
  local symmetry

26
Shape model
27
(No Transcript)
28
(No Transcript)
29
Play movie
30
Contour Grouping as Robot Mapping

• Rao-Blackwellized particle filter is an adaptive,
  probabilistic approach
• Frequently utilized in SLAM approaches to Robot
  Mapping
• Each particles successor is its most likely
  successor
• Particles are resampled to eliminate poorly
  performing particles

31
Traversal space generated as discrete center
points
32
Center Points

• Center points act as center points for maximal
  radius disk between the two sample points
• Full set of center points gives full set of
  maximal radius disks
• Entire set of potential skeletal points
• Want to generate a skeletal path that best groups
  the segments for a given shape model

33
Center points and particle paths
34
Shape model

• System needs to utilize reference model
• Some a priori knowledge to discover the proper
  shape
• Model is a sequence of radii at sample skeleton
  points
• Position in reference model determined by
  triangulation

35
Contour Smoothness

• Smoothness as a criterion for segment selection
• Smoothness is the measure of the amount of turn
  and the distance between segments
• Use least sum of distance to determine both
  distance and the segment pairing
• Smoothness as Gaussian mixture of distance and
  angle

36
SLAM framework

• Obtain particles by sampling from the maximum
  posterior probability
• x is the path traversal
• m is the contour grouping model
• z is the observations
• u is the reference model

37
Particle filter

• Sampling The next generation of particles x(i)t
  is obtained from
• the current generation x(i)t-1 by sampling from a
  proposal distribution

for
38
2) Importance Weighting An individual importance
weight w(i) is assigned to each particle,
according to
39
log(M(c2)) log of pdf that a given pixel is a
center point of radius 10
40
log(M(c2)) log of pdf that a given pixel is a
center point of radius 10
41
3) Resampling Particles with a low importance
weight w(i) are typically replaced by samples
with a high weight. Residual resampling was used
4) Contour Estimating For each pose sample
x(i)t, the corresponding contour estimate m(i)t
is computed based on the trajectory and the
history of observations according to
42
Results

• Grouping performed on several pictures
• Useful groupings on many images
• Little or no noise grouped
• Few structural particles missed

43
(No Transcript)
44
(No Transcript)
45
Future Work

• Integration of the shape similarity of parts and
  the contour grouping
• Learning good contour parts
• Further improvements to the contour grouping to
  make it more robust