Methods for 3D Shape Matching and Retrieval

1
Methods for 3D Shape Matching and Retrieval

• Marcin Novotni Reinhard Klein
• University of Bonn
• Computer Graphics Group

2
Our Aim 1

• Given an example

• Find the most
• similar object(s)
• in a database

,
,
,
3
Motivation

• Lots of 3D archives
• WWW
• Proprietary databases
• ...
• Search engines for data
• Text, 2D images, music (MIDI),
• Emerging since 1998 for 3D

4
Our Aim 2

• Direct matching
• Alignment
• Establishing correspondences

5
Motivation

• Partial matching/retrieval

6
Motivation

• Partial matching/retrieval
• Statistical shape analysis
• Morphing
• Texture transfer
• Registration

7
General Problem

• Abstract representation facilitating
• identification of salient features of 3D objects
• description of features
• comparison (matching)

8
Overview

• Matching for 3D Shape Retrieval
• Correspondence Matching

9
Matching for 3D Shape Retrieval
10
General Problem

• We need a Descriptor

? D( )
D
11
General Problem

• We need a Distance Measure

d( , )
d( , )
D( )
D( )
12
General Problem

• We need a Distance Measure
• Close to (application driven) notion of
  resemblance
• Computationally cheap and robust

d( , )
d( , )
d( , )


13
3D Zernike Descriptors

• Feature vectors
• Xi 3D Zernike Descriptors
• Canterakis 99, Novotni Klein 03, 04
• Distance Measure Euclidean Distance

14
3D Zernike Descriptors

• Retrieval performance Novotni Klein 03 04
• Slightly better than Funkhouser et al. 02
• Object class dependent performance!
• Class dependent coefficient importance!

15
3D Zernike Descriptors
Faces
Chairs
Airplanes
Importance
Coeff No. (Frequency)
16
3D Zernike Descriptors

• Relevance feedback
• User selects relevant / irrelevant items
• Distance measure is tuned
• Learning Machines
• SVM (Support vector machines) Vapnik 95
• One class SVM Schölkopf et al. 99
• (K)BDA ((Kernel) Biased Discriminant Analysis)
  Zhou et al. 01

17
Correspondence Matching
18
Geometric Similarity Estimation

• Idea Novotni Klein 2001
• Definition of geometric similarity in terms of
  a geometric distance

• Intuitive, simple, robust.

19
Geometric Similarity Estimation
Database objects
example
Normalized volumetric error
6.78
8.85
30.29
38.09
67.53
0.00
20
Geometric Similarity Estimation

• Classification by user set threshold

21
Geometric Similarity Estimation

• Measures deformation magnitude

22
Correspondence Matching
?
23
Correspondence Matching
?
24
Correspondence Matching
?
25
Correspondence Matching
?
26
Correspondence Matching

• Ideally dense mapping

?
27
Correspondence Matching

• Ideally dense mapping
• Deformation by mapping semantics

DArcy Thompson 1917 On Growth and Form
28
Correspondence Matching

• Ideally dense mapping
• Easier mapping salient points
• Curvature extremes
• Corners (Harris points in 2D)
• Etc
• Scale space extremes

29
Correspondence Matching

• Ideally dense mapping
• Easier mapping salient points
• Curvature extremes
• Corners (Harris points in 2D)
• Etc
• Scale space extremes

30
Correspondence Matching

• Scale Space extremes Lindeberg 94

31
Correspondence Matching

• We have
• Salient points
• Spatial position
• Size of local blobs
• How to match???

32
Correspondence Matching

• Criteria for correspondences
• Similar
• Local geometries
• Constellations of points

33
Correspondence Matching

• Criteria for correspondences
• Similar
• Local geometries
• Constellations of points

34
Correspondence Matching

• Local description

35
Correspondence Matching

• Local description

36
Correspondence Matching

• Assumption
• Similar local descriptors
• Similar local geometries

37
Correspondence Matching

• Criteria for correspondences
• Similar
• Local geometries
• Constellations of points

38
Correspondence Matching

• Similar constellations of points
• Smooth mappings leave constellations consistent
• Idea
• Constellations are consistent if mapping is smooth

39
Correspondence Matching
40
Correspondence Matching
41
Correspondence Matching
42
Correspondence Matching
43
Correspondence Matching
44
Correspondence Matching

• Similar constellations of points
• Idea
• Constellations are consistent if mapping is
  smooth
• Thin Plate Spline interpolation Brookstein 89
• ? minimize

Total curvature
45
Correspondence Matching

• ? minimize
• Minimizer (Thin Plate Spline interpolator)

Affine part
Nonlinear deformation
46
Correspondence Matching

• ? minimize
• Minimizer (Thin Plate Spline interpolator)

2D Thin Plate Spline
47
Correspondence Matching

• ? minimize
• Minimizer (Thin Plate Spline interpolator)

Can be computed by a (N4)x(N4) matrix inversion
48
Correspondence Matching

• Find (sub)sets of correspondences
• Small local descriptor distances
• Small deformation energy
• Hierarchical pruning and clustering
• Using
• Local descriptors
• Geometrical constellation consistency

49
Correspondence Matching
50
Correspondence Matching
51
Correspondence Matching
52
Correspondence Matching
53
Correspondence Matching
54
Correspondence Matching
55
Correspondence Matching
56
Correspondence Matching
57
Correspondence Matching
58
Correspondence Matching

• New avenues
• Local Descriptions for retrieval
• Online Learning for local descriptions
• Dense matching from salient points
• Etc.

59

• Danke,
• DFG!

60
3D Zernike Descriptors

• Basis functions in the unit sphere

61
3D Zernike Descriptors

• Basis functions in the unit sphere

SH on the sphere
Function of the radius
Rotation invariant!
62
3D Zernike Descriptors

• Basis functions in the unit sphere
• 3D Zernike Moments Canterakis 99

Object function, e.g. voxel grid
63
3D Zernike Descriptors

• 3D Zernike Descriptors
• Amplitudes of the Zernike decomposition
• Rotation invariant

64
3D Zernike Descriptors

• Basis functions in the unit sphere

65
3D Zernike Descriptors

• Basis functions in the unit sphere

SH on the sphere
Function of the radius
Rotation invariant!
66
3D Zernike Descriptors

• Basis functions in the unit sphere
• 3D Zernike Moments Canterakis 99

Object function, e.g. voxel grid
67
3D Zernike Descriptors

• 3D Zernike Descriptors
• Amplitudes of the Zernike decomposition
• Rotation invariant

68
3D Zernike Descriptors

• For N22 155 floats as search key
• Timings (1.8 GHz Pentium)
• Voxelization 0.3 10.0 sec / object
• Computation 0.2 sec / object
• Retrieval (1814 objects) 0.3 sec

69
3D Zernike Descriptors

• Retrieval performance Novotni Klein 04
• Slightly better than Funkhouser et al. 02
• Object class dependent performance!
• Class dependent coefficient importance!

70
3D Zernike Descriptors
Faces
Chairs
Airplanes
Importance
Coeff No.
71
3D Zernike Descriptors

• 3D Zernike functions Canterakis 99
• are polynomials such that are
  orthonormal within the unit ball

72
3D Zernike Descriptors

• 3D Zernike functions Canterakis 99
• are polynomials such that are
  orthonormal within the unit ball
• 3D Zernike Moments

73
3D Zernike Descriptors

• 3D Zernike Descriptors
• Amplitudes of the Zernike decomposition
• Rotation invariant

74
3D Zernike Descriptors

• For N20 121 floats as search key
• Timings (1.8 GHz Pentium)
• Voxelization 0.3 10.0 sec / object
• Computation 0.2 sec / object
• Retrieval (1814 objects) 0.3 sec

75
3D Zernike Descriptors

• Retrieval performance Novotni Klein 04
• Slightly better than Funkhouser et al. 02
• Object class dependent performance!
• Class dependent coefficient importance!

76
Correspondence Matching

• Matching should be
• Independent of topology
• Robust
• Suitable for partial matching

77
Correspondence Matching

• Local description
• Local shape histograms
• ? Not rotation invariant
• Rotation invariance
• ? Amplitudes of the Fourier Transform

78
3D Zernike Descriptors

• 155 floats as search key
• Retrieval (1814 objects) 0.3 sec

79
Correspondence Matching

• Stuff to remember
• Salient points simplify the problem
• Smooth mapping iff consistent constellations

80
Correspondence Matching

• Stuff to remember
• Salient points simplify the problem
• Volumetric
• On the surface

81
Correspondence Matching

• Stuff to remember
• Salient points simplify the problem
• Smooth mapping iff consistent constellations

82
Correspondence Matching

• New avenues
• Local Descriptions for retrieval
• Retrieval by part selection recognition
• Retrieval from large scenes

83
Correspondence Matching

• New avenues
• Local Descriptions for retrieval
• Online Learning for local descriptions
• Adopting pattern recognition methods

84
Correspondence Matching

• New avenues
• Local Descriptions for retrieval
• Online Learning for local descriptions
• Dense matching from salient points
• Morphing, registration, object statistics

85
Our Aim 2

• Direct matching
• Alignment

86
Correspondence Matching

• Scale space extremes Lindeberg 94
• Blob detection by
• localizing extremes of Laplacian
• in scale and space

Position of the blob
Size of the blob
87
Correspondence Matching

• Maxima of Laplacian over scales

88
Correspondence Matching

• Spatial maxima