Title: Digital Camera and Computer Vision Laboratory
1Computer and Robot Vision II
- Chapter 18
- Object Models And Matching
Presented by ??? ??? 0911 246
313 r94922093_at_ntu.edu.tw ???? ??? ??
218.1 Introduction
- object recognition one of most important aspects
of computer vision
318.2 Two-Dimensional Object Representation
- 2D shape analysis useful in machine vision
application - medical image analysis
- aerial image analysis
- manufacturing
418.2 Two-Dimensional Object Representation
- 2D shape representation classes
- global features
- local features
- boundary description
- skeleton
- 2D parts
518.2.1 Global Feature Representation
- 2D object can be thought of as binary image
- value 1 pixels of object
- value 0 pixels outside object
- 2D shape features area, perimeter, moments,
circularity, elongation
618.2.1 Global Feature Representation
- Shape Recognition by Moments
- f binary image function
- 2D shape
- digital th moment of S
-
- area of S number of pixels of S
718.2.1 Global Feature Representation
- moment invariants functions of moments invariant
under shape transform - prefer moment invariants under translation,
rotation, scaling - skewing center of gravity of S
818.2.1 Global Feature Representation
- central th moment of S
- central moments translation invariant
- normalized central moments of S
918.2.1 Global Feature Representation
- seven functions that are rotation invariant
1018.2.1 Global Feature Representation
- Shape Recognition with Fourier Descriptors
- Fourier descriptors another way for extracting
features from 2D shapes - Fourier descriptors defined to characterize
boundary - The main idea is to represent the boundary as a
function of one variable , expand in
its Fourier series, and use the coefficients of
the series as Fourier descriptors (FDs). - finite number of FDs can be used to describe the
shape
1118.2.1 Global Feature Representation
1218.2.1 Global Feature Representation
1318.2.1 Global Feature Representation
1418.2.2 Local Feature Representation
- 2D object characterized by local features,
attributes, relationships - most commonly used local features holes, corners
- holes found by connected component procedure
followed by boundary tracing - holes detected by binary mathematical
morphology, if hole shapes known - hole properties areas, shapes
- corner detection can be performed on binary or
gray tone image - corner property angle at which lines meet
15 1618.2.3 Boundary Representation
- boundary representation most common
representation for 2D objects - 3 main ways to represent object boundary
- 1. sequence of points
- 2. chain code
- 3. sequence of line segments
1718.2.3 Boundary Representation
- The Boundary as a Sequence of Points
- boundary points from border-following or
edge-tracking algorithms - interest points boundary points with special
property useful in matching
1818.2.3 Boundary Representation
- The Chain Code Representation
- chain encoding can be used at any level of
quantization - chain encoding saves space required for row and
column coordinates - boundary encoded first quantized by placing over
square grid - grid side length determines resolution of
encoding - marked points grid intersections closest to
curve and used in encoding - marks starting point of curve
1918.2.3 Boundary Representation
- chain encoding of boundary curve
2018.2.3 Boundary Representation
- line segments links to be used to approximate
the curve - encoding scheme eight possible directions
assigned integer between 0, 7 - chain chain encoding in the form
- or
2118.2.3 Boundary Representation
- length of chain code with n chains can be simply
estimated as n - number of odd chain codes
- number of even chain codes
- number of corners
- unbiased estimate of perimeter length
- Freeman suggested
2218.2.3 Boundary Representation
- The Boundary as a Sequence of Line Segments
- line segment sequence after boundary segmented
into near-linear portion - line segment sequence used in shape
recognition or other matching tasks - coordinate location where pair of
lines meet - angle magnitude where pair of lines meet
- sequence of junction
points to represent line segment sequence
2318.2.3 Boundary Representation
- sequence of junction
points representing test object T -
an association - goal given O, T, to find F satisfying i lt j
F(i) lt F(j) or F(i) missing or F(j) missing
2418.2.4 Skeleton Representation
- strokes long, sometimes thin parts forming
shapes - line segments that characterize the strokes of
set of characters
2518.2.4 Skeleton Representation
- symmetric axis transform set of maximal circular
disks inside object - symmetric axis locus of centers of these maximal
disks - symmetric axes of the characters
2618.2.4 Skeleton Representation
- symmetric axis one example of skeleton
description of 2D object - symmetric axis of rectangle consists of five
line segments not single line - symmetric axis extremely sensitive to noise
- symmetric axis difficult to use in matching
27(No Transcript)
2818.2.4 Skeleton Representation
- axis of smoothed local symmetries separate
definition for skeleton - local symmetry midpoint P of line segment BA
joining pair of points A, B - angle between BA and outward normal at
A - angle between BA and inward normal at
B
2918.2.4 Skeleton Representation
- point P that is local symmetry with respect to
boundary points A and B
3018.2.4 Skeleton Representation
- axes spines loci of local symmetries maximal
w.r.t. forming smooth curve - cover of axis portion of shape subtended by axis
- axis cover properly contained in another cover
second axis subsumes first
3118.2.4 Skeleton Representation
- symmetric axes of local symmetry of a rectangle
3218.2.4 Skeleton Representation
- axes of smoothed local symmetries of several
objects
3318.2.5 Two-Dimensional Part Representation
- parts, attributes, interrelationships form
structural description of shape - nuclei regions where primary convex subset
overlap - nuclei shaded areas of overlap
3418.2.5 Two-Dimensional Part Representation
- decomposition of shape into primary convex
subsets and nuclei
3518.2.5 Two-Dimensional Part Representation
- near-convexity allows noisy distorted instances
to have same decompositions - , two points on object boundary
- relation visibility relation
- if line completely interior to object
boundary, - the graph-theoretic clustering to determine
clusters of visibility relation
3618.2.5 Two-Dimensional Part Representation
- decomposition of three similar shapes into
near-convex pieces
37 3818.3 Three-Dimensional Object Representations
3918.3.1 Local Features Representation
- range data obtained from laser range finder,
light striping, stereo, etc. - from depth, try to infer surfaces, edges,
corners, holes, other features - 3D matching more difficult than 2D because of
occlusion
4018.3.2 Wire Frame Representation
- wire frame model 3D object model with only edges
of object
4118.3.2 Wire Frame Representation
- two-color hyperboloid and its line drawing
4218.3.2 Wire Frame Representation
4318.3.2 Wire Frame Representation
- Necker cube lower-vertical face or upper
vertical face closer to viewer - Schroder staircase viewed either from above or
from below
44- two well-known ambiguous line drawings
45- two well-known ambiguous line drawings
46- two well-known ambiguous line drawings
47- inherent ambiguity of line drawing owing to
complete loss of depth
4818.3.2 Wire Frame Representation
- general-viewpoint assumption none of the
following situations - 1. two vertices of scene objects represented at
same picture point - 2. two scene edges seen as single line in picture
- 3. vertex seen exactly in line with unrelated
edge
4918.3.2 Wire Frame Representation
- general-viewpoint assumption heart of
line-drawing interpretation - viewpoint in perspective projection center of
projection - viewpoint in orthographic projection direction
of projection
50- subjective contours of Kanizsa white occluding
triangle in space
51(No Transcript)
5218.3.2 Wire Frame Representation
- line labels for visible projections of
surface-normal discontinuities
53(No Transcript)
54- four basic ways in which three planar surfaces
can form polyhedron vertex
55- all possible distinct appearances of trihedral
vertices of polyhedra
56- complete junction catalog for line drawings of
trihedral-vertex polyhedra
57- complete junction catalog for line drawings of
trihedral-vertex polyhedra
58 5918.3.3 Surface-Edge-Vertex Representation
- VISIONS system Visual Integration by Semantic
Interpretation of Natural Scenes - PREMIO system Prediction in Matching Images to
Objects - PREMIO 3D object model hierarchical, relational
model with five levels - 5 levels world, object, face/edge/vertex,
surface/boundary, arc/2D, 1D piece
6018.3.3 Surface-Edge-Vertex Representation
- world level arrangement of different objects in
world - object level arrangement of different faces,
edges, vertices forming objects - face level describes face in terms of surfaces
and boundaries - surface level specifies elemental pieces forming
surfaces
6118.3.3 Surface-Edge-Vertex Representation
- 2D piece level describes pieces and specifies
arcs forming boundaries - 1D piece level describes elemental pieces
forming arcs - SDS spatial data structure
- A/V attribute-value table
62(No Transcript)
6318.3.4 Sticks, Plates, and Blobs
- sticks, plates, blobs model rough models of 3D
objects used in rough-matching - sticks long, thin parts with only one
significant dimension - stick cannot bend very much
- stick has two logical endpoints, set of interior
points, center of mass
6418.3.4 Sticks, Plates, and Blobs
- plates flattish wide parts with two nearly flat
surfaces - plates have two significant dimensions
- plate surfaces cannot fold very much
- plate has set of edge points, set of surface
points, center of mass - blobs parts with three significant dimensions
- blob can be bumpy but cannot have concavities
- blob set of surface points and center of mass
- sticks, plates, blobs near-convex
65(No Transcript)
6618.3.4 Sticks, Plates, and Blobs
- attribute-value table contains global attributes
- simple-parts relation lists the parts and their
attributes - connects-supports relation gives connections
between pairs of parts - triples relation specifies connections between
three parts at a time - parallel relation lists pairs of parts that are
parallel - perpendicular relation lists pairs of parts that
are perpendicular - TYPE 1 for stick, 2 for plate, 3 for blob
67- full relational structure of sticks-plates-blobs
model of chair object
68(No Transcript)
6918.3.5 Generalized Cylinder Representation
- generalized cylinder volumetric primitive
defined by axis and cross-section - cross section swept along axis, creating a solid
- e.g. actual cylinder generalized cylinder whose
axis is straight-line segment and whose cross
section is circle of constant radius - e.g. cone generalized cylinder whose axis is
straight-line segment and cross section is circle
with radius initially zero to maximum
7018.3.5 Generalized Cylinder Representation
- e.g. rectangular solid generalized cylinder
whose axis is straight line segment and cross
section is constant rectangle - e.g. torso generalized cylinder whose axis is
circle and whose cross section is constant circle - generalized cylinder representation uses
generalized cylinders as primitives
7118.3.5 Generalized Cylinder Representation
- surface-edge-vertex model very precise
- sticks-plates-and-blobs model very rough
- generalized cylinder model somewhere in between
7218.3.5 Generalized Cylinder Representation
- person modeled roughly as cylinders for head,
torso, arms, legs - dotted lines axes of cylinders
7318.3.6 Superquadric Representation
- superquadrics lumps of clay deformable and can
be glued into object models - superquadric models mainly used with range data
7418.3.6 Superquadric Representation
- Superquadrics are a flexible family of
3-dimensional parametric objects, useful for
geometric modeling. By adjusting a relatively few
number of parameters, a large variety of shapes
may be obtained.
75- range data image of (a) a doll, (b) its
superquadric fit (c), (d) wire frame
76 7718.3.7 Octree Representation
- octree encoding geometric modeling technique
used to represent 3D objects - octree encoding used in computer vision,
robotics, computer graphics - octree hierarchical 8-ary tree structure
- each node in octree corresponds to cubic region
of universe - full if cube is completely enclosed by 3D object
- empty if cube contains no part of object
7818.3.7 Octree Representation
- partial if cube partly intersects object
- full, empty, partial correspond to black,
white, gray in quadtrees - node with label full or empty has no children
- partial has eight children representing
partition of cube into octants
79(No Transcript)
8018.3.8 The Extended Gaussian Image
- 3D object collection of surface normals, one at
each point of object surface - planar surface all points on surface map to same
surface normal - convex with positive curvature everywhere
distinct surface normal everywhere - Gaussian sphere unit sphere
- set of surface normals mapped to Gaussian sphere
tail at center head outward - Gaussian image of object resultant set of points
on Gaussian sphere
81(No Transcript)
8218.3.8 The Extended Gaussian Image
- for planar objects Gaussian image not
invertible, not precise enough for use - small surface patch of object
- corresponding surface patch on Gaussian
sphere - Gaussian curvature K
8318.3.8 The Extended Gaussian Image
- point on Gaussian sphere
corresponding to point (u, v) on object surface - extended Gaussian image
- planar region Gaussian curvature 0, point mass
in extended Gaussian image
8418.3.9 View-Class Representation
- view classes each representing set of viewpoints
sharing some property - property e.g. same object surfaces visible
- property e.g. same line segments visible
- property e.g. relational distances between
relational structures are similar - characteristic views sets producing
topologically isomorphic line drawings
8518.3.9 View-Class Representation
- three view classes of cube producing
topologically isomorphic line drawings
86(No Transcript)
8718.3.9 View-Class Representation
- aspect graph of object graph structure where
- 1. each node represents topologically distinct
view of object - 2. a node for each such view of object
- 3. each arc represents a visual event at
transition - 4. there is an arc for each such transition
88(No Transcript)
8918.4 General Frameworks for Matching
- matching finding correspondence between two
entities - consistent labeling procedures examples of
matching algorithms
9018.4.1 Relational-Distance Approach to Matching
- relational distance compares two structures and
determines similarity - Relational-Distance Definition
- relational description
- sequence of relations
- set of parts of entity being described
- relation indicating various relationships
among parts - relational description with part set A
- relational description with part set B
9118.4.1 Relational-Distance Approach to Matching
- assumption A B, otherwise add dummy parts
to smaller set - f any one-one, onto mapping from A to B
- N positive integer
- composition R f of relation R with
function f
9218.4.1 Relational-Distance Approach to Matching
- f maps parts from set A to parts from set B
- structural error of f for ith pair of
corresponding relations in , - total error of f with respect to ,
- relational distance between
,
9318.4.1 Relational-Distance Approach to Matching
- best mapping from to mapping f
that minimizes total error
9418.4.1 Relational-Distance Approach to Matching
- Relational-Distance Examples
- best mapping from to
is - for this mapping
9518.4.1 Relational-Distance Approach to Matching
- two digraphs whose relational distance is 3
96(No Transcript)
9718.4.1 Relational-Distance Approach to Matching
- Relational Distance as a Metric
- relational distance used to determine similarity
of unknown object to model - relational distance used to compare object
models to group models - f relational isomorphism if f one-one, onto from
A to B and E(f) 0 - f A B relational isomorphism ,
isomorphic - GD relational-distance measure
9818.4.1 Relational-Distance Approach to Matching
- arbitrary relational descriptions
- , , metric property of GD
- 1. isomorphic
- 2.
- 3.
9918.4.1 Relational-Distance Approach to Matching
- Attributed Relational Descriptions and Relational
Distance - extend relational description and relational
distance to include - properties of parts
- properties of the whole
- properties of these relationships
100 10118.4.2 Ordered Structural Matching
- definition of ordering on primitives greatly
reduces complexity of search
10218.4.3 Hypothesizing and Testing with Viewpoint
Consistency Constraint
- viewpoint consistency constraint
- The locations of all projected model features
in an image must be consistent with projection
from a single viewpoint.
10318.4.4 View-Class Matching
- if 3D object represented by view-class model,
matching divided into 2 stages - 1. determining view class of object
- 2. determining precise viewpoint within that view
class
10418.4.4 View-Class Matching
- Determining View Class
- relational pyramid hierarchical relational
structure to represent view class - Level-1 primitives straight- and curved-line
segments - Level-2 relations junctions and loops
- Level-3 relations adjacency, collinearity,
junction parallelness, loop-inside-loop
10518.4.4 View-Class Matching
- Pose Determination within View Class
- relational pyramid hierarchical, relational
structure to constrain matching
10618.4.5 Affine-Invariant Matching
- set of interest points
lying in plane - rotation matrix relating model reference frame to
camera reference frame - translation of object reference frame to camera
reference frame
10718.4.5 Affine-Invariant Matching
- f distance between image plane and center of
perspectivity - observed image data points
by perspective projection - when translation in z-direction large
compared with -
10818.4.5 Affine-Invariant Matching
- A 2 x 2 (scaling, rotation, skewing) matrix
- b 2D (translation) vector
- affine 2D correspondence Aw b
10918.4.5 Affine-Invariant Matching
- Affine Transformation of Points in a Plane
- necessary and sufficient to define plane
uniquely 3 noncollinear points
11018.4.5 Affine-Invariant Matching
- The Hummel-Wolfson-Lamdan Matching Algorithm
- to match noncollinear triplets in model interest
points with scene - preprocessing convert model interest points into
affine-invariant model - recognition match model against image using
affine representation
11118.4.5 Affine-Invariant Matching
- Shortcomings of the Affine-Invariant Matching
Technique - affine-invariant matching technique
mathematically sound in noiseless case - shortcomings of affine-invariant matching in
practice - 1. if three noncollinear points not numerically
stable, points not reliable - 2. coordinates of detected interest points noisy
in real image - 3. partial object symmetries may cause wrong
matching
11218.4.5 Affine-Invariant Matching
- An Explicit Noise Model and Optimal Voting
11318.5 Model Database Organization
- organize database of models to allow rapid
access to most likely candidate - group similar relational models into clusters and
choose representative - arrows indicate mapping from parts of object 2
to parts of other objects - cluster of object models whose representative is
object 2
114(No Transcript)
115