3D Object Representation and Matching

1
3D Object Representation and Matching

• Spin images
• Shape histograms/distributions
• Context Shapes

2
Spin Images(Jonhson, Hebert, 1997)

• A surface representation that uses 2D images to
  describe 3-D oriented points
• It allows to apply powerful techniques from 2-D
  template matching and pattern classification to
  the problem of 3-D surface recognition.

3
Overview

• A spin image provides a 2D object centered
  description of a 3D mesh object.
• Essentially, a spin image represents the radial
  and elevation distances to every other vertex on
  the mesh.

4
Spin image computation

• a perpendicular distance from surface normal
• b signed perpendicular distance to the plane p

p oriented point n surface normal x
position of another vertex b difference of x
- p, then dot product with surface normal to
project on normal. a length of vector from x to
p, subtract b component
5
Spin Images

ai

bj
Restriction b gtThreshold
6
Comparing spin images

• How can we determine similarity of spin images?
• Euclidean distance
• Correlation

7
Statistical correlation

• A statistical method used to determine whether a
  linear relationship between variables exists
• The correlation coefficient cor computed from
  data measures the strength and direction of a
  linear relationship between two variables.

8
Range of the correlation

• The range of the correlation coefficient is from
  -1 to 1.
• If there is a strong positive linear
  relationship
• between the variables, the value of r will be
  close to 1.
• If there is a strong negative linear
  relationship between the variables, the value of
  r will be close to -1.

9
Examples with scatter plots

• A scatter plot is a graph of the ordered pairs
  (x,y)

10
Correlation of Spin images

• Given two spin-images P and Q with N bins each,
  the correlation coefficient R(P,Q) is computed by
  the equivalent formula

11
Geometric Matching

• A two-step procedure
• Establish individual point correspondences based
  on the correlation of the spin image
• 2. Group point correspondences using a geometric
  consistency criterion

12
Grouping Point Correspondences
The grouping criterion is the Geometric
Consistencyof distances and angles of
corresponding points
13
Consistency Criterion

• A correspondence C (P, P) between two points
  P and P is geometrically consistent with the
  group of already established correspondences
• C1 (Q1, Q1), . . . , Cn (Qn, Qn)
• if
• 1. the spin images of P and P are highly
  correlated
• 2. for every i 1, . . . , n, the distances
  between P and Qi and between P and Qi are
  within some user-defined tolerance
• 3. for every i 1, . . . , n, the angle between
  the normals at P and Qi is the same as the angle
  between the normals at P and Qi within some
  user-defined tolerance.

14
Grouping correspondences

• Basic Idea
• Select a correspondence as a seed to start a
  group
• Grow a solution around the seed by adding
  geometrically consistent correspondences
• .

15
A greedy Algorithm

• L is the list of correspondences sorted in
  decreasing order with respect to correlation
  values.
• The top element of the list L forms the seed of
  a group of correspondences
• Remove the seed from L and scan L

16
Algorithm (continued)

• if a correspondence is found that is
  geometrically consistent with those already in
  the group, then it is added to group and removed
  from L.
• When no more correspondences can be added, but
  the list L is not empty, start over again to
  create a new group.

17
Scoring the groups

• Different criteria
• Number of pairs of points in the group
• RMSD of the rigid transformation that best
  overlaps the corresponding points of the group

18
Surface Matching- Block Diagram
19
Why spin images?

• Spin-images are useful representations the
  following reasons.
• invariant to rigid transformation
• object-centered
• simple to compute
• scalable from local to global representation
• applicable to problems in 3-D computer vision

20
Shape histograms
Based on a partition of the space in which the
objects/molecules reside, i.e. a decomposition
into cells corresponding to the histogram bins.
Shell bins
Sector bins Combined bins
21
Euclidean Distance

• Given two N-dimensional vectors p and q
• deuclid(p, q) S i1,N (pi qi)2 (pq)
  (pq)T

22
Shortcomings of Euclidean distance
23
Example Proteins
24
Quadratic distance

• If AI then the quadratic distance coincides with
  the Euclidean distance

25
Similarity Weights

• Setting the elements of A
• d(i,j) represents the distance of bins i and j
• The higher s the more similar A to I

26
Generalization of geometric histogram Shape
Distribution

• build shape distributions from 3D polygonal
  models and compute a measure of their
  dissimilarities.
• Shape matching problem is reduced to sampling,
  normalization, and comparison of probability
  distributions,

27
Shape DistributionSelecting a Shape Function

• A3 Measures the angle between three random
  points on the surface of a 3D model.
• D1 Measures the distance between a fixed point
  and one random point on the surface. We use the
• centroid of the boundary of the model as the
  fixed point.
• D2 Measures the distance between two random
  points on the surface.
• D3 Measures the square root of the area of the
  triangle between three random points on the
  surface.
• D4 Measures the cube root of the volume of the
  tetrahedron between four random points on the
• surface.

28
(No Transcript)
29
Three steps

• 1) to select discriminating shape functions
• 2) to efficiently construct shape functions for
  each 3D model,
• 3) to compute a dissimilarity measure for pairs
  of distributions.

30
(No Transcript)
31
Comparing Shape Distributions

• f and g are two functions representing
  probability distributions
• Minkowski LN norm
• D( f , g) Integral (f g N)1/N

32
Experimental setting

• 10 representative 3D models
• 8 transformations applied to each of them
• Scale
• Grow by a factor of 10 in every dimension.
• Anisotropic scale
• Grow by 5 in the Y dimension and 10 in the Z
  dimension.
• Rotation
• Rotate by 45 degrees around the X axis, then the
  Y axis, then the Z axis.
• ......
• The resulting database had 9 versions of each
  model, making 90 models in all.

33
Experimental Results
Plot of D2 values scaled to align their mean
values
34
Shape discriminationSimilarity matrix
35
2D Shape Contexts

• Take a random point on the shape

Image source Belongie02
36
2D Shape Contexts

• Compute the offset vectors to all other samples

37
2D Shape Contexts

• Histogram the vectors against sectors and shells
• Perform this for a large sampling of points

38
Extension to 3D

• Step 1 pick random points on surface

Image source Koertgen03
39
Extension to 3D

• For each point, compute and histogram offsets

Image source Koertgen03
40
Extension to 3D

• For each point, compute offsets

Image source Koertgen03
41
Extension to 3D

• Now we histogram the offset vectors.
• The 3D histogram of looks like

Image source Frome04
42
Context Shapes
The support region is divided into bins by
equally spaced boundaries into azimuth,
elevation, and radial dimensions R (R0, . . .
,RJ ) q (q0, . . . , q k) f (q0, . . . , q
l) Some Ln difference of the histogram vector
can be used to compare two contexts
43
Extension to 3D

• Shells are spaced logarithmically apart
• Histogram votes are weighted by the volume of the
  bin
• Some Ln difference of the histogram vector can be
  used to compare two contexts

Image source Frome04, Koertgen03
44
Initial histogram orientation

• Align the objects north-pole to the surface
  normal
• Problems
• One degree of freedom remains
• Histogram values depend on the precision of the
  surface normals

45
Local descriptors
Variation of shape descriptors For each feature,
represent its local geometry in cylindrical
coordinates about the normal
Image from Kazhdan, 2005
46
Challenges

• How do we orient the histogram spheres?
• How do we compute distance between a model and
  one of its subsets?
• Speed?

47
Extended Gaussian Image
48

• Better tesselations may be found by projecting
  regular polyhedra onto the unit sphere after
  bringing their center to the center of the sphere
  
• Regular polyhedra are uniform and have faces
  which are all of one kind of regular polygon.
  (They are also called the Platonic solids)

49

• (tetrahedron, hexahedron, octahedron,
  dodecahedron, and icosahedron).
• One can go a little further by considering
  semi-regular polyhedra. A semi-regular polyhedron
  has regular polygons as faces, but the faces are
  not all of the same kind
• (They are also called the Archimedean polyhedra.)

50
(No Transcript)
51
Desirable property of a tessellation

• 1. all cells should have the same area
• 2. all cells should have the same shape
• 3. the cells should have regular shapes that are
  compact
• 4. the division should be fine enough to provide
  good angular resolution
• 5. for some rotations, the cells should be
  brought into coincidence with themselves.