Digital Camera and Computer Vision Laboratory - PowerPoint PPT Presentation

About This Presentation
Title:

Digital Camera and Computer Vision Laboratory

Description:

Title: 1 Author: boss Last modified by: mufan Created Date: 2/27/2004 9:53:19 AM Document presentation format: Company: CMT – PowerPoint PPT presentation

Number of Views:145
Avg rating:3.0/5.0
Slides: 116
Provided by: Boss8
Category:

less

Transcript and Presenter's Notes

Title: Digital Camera and Computer Vision Laboratory


1
Computer and Robot Vision II
  • Chapter 18
  • Object Models And Matching

Presented by ??? ??? 0911 246
313 r94922093_at_ntu.edu.tw ???? ??? ??
2
18.1 Introduction
  • object recognition one of most important aspects
    of computer vision

3
18.2 Two-Dimensional Object Representation
  • 2D shape analysis useful in machine vision
    application
  • medical image analysis
  • aerial image analysis
  • manufacturing

4
18.2 Two-Dimensional Object Representation
  • 2D shape representation classes
  • global features
  • local features
  • boundary description
  • skeleton
  • 2D parts

5
18.2.1 Global Feature Representation
  • 2D object can be thought of as binary image
  • value 1 pixels of object
  • value 0 pixels outside object
  • 2D shape features area, perimeter, moments,
    circularity, elongation

6
18.2.1 Global Feature Representation
  • Shape Recognition by Moments
  • f binary image function
  • 2D shape
  • digital th moment of S
  • area of S number of pixels of S

7
18.2.1 Global Feature Representation
  • moment invariants functions of moments invariant
    under shape transform
  • prefer moment invariants under translation,
    rotation, scaling
  • skewing center of gravity of S

8
18.2.1 Global Feature Representation
  • central th moment of S
  • central moments translation invariant
  • normalized central moments of S

9
18.2.1 Global Feature Representation
  • seven functions that are rotation invariant

10
18.2.1 Global Feature Representation
  • Shape Recognition with Fourier Descriptors
  • Fourier descriptors another way for extracting
    features from 2D shapes
  • Fourier descriptors defined to characterize
    boundary
  • The main idea is to represent the boundary as a
    function of one variable , expand in
    its Fourier series, and use the coefficients of
    the series as Fourier descriptors (FDs).
  • finite number of FDs can be used to describe the
    shape

11
18.2.1 Global Feature Representation
12
18.2.1 Global Feature Representation
13
18.2.1 Global Feature Representation
14
18.2.2 Local Feature Representation
  • 2D object characterized by local features,
    attributes, relationships
  • most commonly used local features holes, corners
  • holes found by connected component procedure
    followed by boundary tracing
  • holes detected by binary mathematical
    morphology, if hole shapes known
  • hole properties areas, shapes
  • corner detection can be performed on binary or
    gray tone image
  • corner property angle at which lines meet

15
  • joke

16
18.2.3 Boundary Representation
  • boundary representation most common
    representation for 2D objects
  • 3 main ways to represent object boundary
  • 1. sequence of points
  • 2. chain code
  • 3. sequence of line segments

17
18.2.3 Boundary Representation
  • The Boundary as a Sequence of Points
  • boundary points from border-following or
    edge-tracking algorithms
  • interest points boundary points with special
    property useful in matching

18
18.2.3 Boundary Representation
  • The Chain Code Representation
  • chain encoding can be used at any level of
    quantization
  • chain encoding saves space required for row and
    column coordinates
  • boundary encoded first quantized by placing over
    square grid
  • grid side length determines resolution of
    encoding
  • marked points grid intersections closest to
    curve and used in encoding
  • marks starting point of curve

19
18.2.3 Boundary Representation
  • chain encoding of boundary curve

20
18.2.3 Boundary Representation
  • line segments links to be used to approximate
    the curve
  • encoding scheme eight possible directions
    assigned integer between 0, 7
  • chain chain encoding in the form
  • or

21
18.2.3 Boundary Representation
  • length of chain code with n chains can be simply
    estimated as n
  • number of odd chain codes
  • number of even chain codes
  • number of corners
  • unbiased estimate of perimeter length
  • Freeman suggested

22
18.2.3 Boundary Representation
  • The Boundary as a Sequence of Line Segments
  • line segment sequence after boundary segmented
    into near-linear portion
  • line segment sequence used in shape
    recognition or other matching tasks
  • coordinate location where pair of
    lines meet
  • angle magnitude where pair of lines meet
  • sequence of junction
    points to represent line segment sequence

23
18.2.3 Boundary Representation
  • sequence of junction
    points representing test object T

  • an association
  • goal given O, T, to find F satisfying i lt j
    F(i) lt F(j) or F(i) missing or F(j) missing

24
18.2.4 Skeleton Representation
  • strokes long, sometimes thin parts forming
    shapes
  • line segments that characterize the strokes of
    set of characters

25
18.2.4 Skeleton Representation
  • symmetric axis transform set of maximal circular
    disks inside object
  • symmetric axis locus of centers of these maximal
    disks
  • symmetric axes of the characters

26
18.2.4 Skeleton Representation
  • symmetric axis one example of skeleton
    description of 2D object
  • symmetric axis of rectangle consists of five
    line segments not single line
  • symmetric axis extremely sensitive to noise
  • symmetric axis difficult to use in matching

27
(No Transcript)
28
18.2.4 Skeleton Representation
  • axis of smoothed local symmetries separate
    definition for skeleton
  • local symmetry midpoint P of line segment BA
    joining pair of points A, B
  • angle between BA and outward normal at
    A
  • angle between BA and inward normal at
    B

29
18.2.4 Skeleton Representation
  • point P that is local symmetry with respect to
    boundary points A and B

30
18.2.4 Skeleton Representation
  • axes spines loci of local symmetries maximal
    w.r.t. forming smooth curve
  • cover of axis portion of shape subtended by axis
  • axis cover properly contained in another cover
    second axis subsumes first

31
18.2.4 Skeleton Representation
  • symmetric axes of local symmetry of a rectangle

32
18.2.4 Skeleton Representation
  • axes of smoothed local symmetries of several
    objects

33
18.2.5 Two-Dimensional Part Representation
  • parts, attributes, interrelationships form
    structural description of shape
  • nuclei regions where primary convex subset
    overlap
  • nuclei shaded areas of overlap

34
18.2.5 Two-Dimensional Part Representation
  • decomposition of shape into primary convex
    subsets and nuclei

35
18.2.5 Two-Dimensional Part Representation
  • near-convexity allows noisy distorted instances
    to have same decompositions
  • , two points on object boundary
  • relation visibility relation
  • if line completely interior to object
    boundary,
  • the graph-theoretic clustering to determine
    clusters of visibility relation

36
18.2.5 Two-Dimensional Part Representation
  • decomposition of three similar shapes into
    near-convex pieces

37
  • joke

38
18.3 Three-Dimensional Object Representations
39
18.3.1 Local Features Representation
  • range data obtained from laser range finder,
    light striping, stereo, etc.
  • from depth, try to infer surfaces, edges,
    corners, holes, other features
  • 3D matching more difficult than 2D because of
    occlusion

40
18.3.2 Wire Frame Representation
  • wire frame model 3D object model with only edges
    of object

41
18.3.2 Wire Frame Representation
  • two-color hyperboloid and its line drawing

42
18.3.2 Wire Frame Representation
43
18.3.2 Wire Frame Representation
  • Necker cube lower-vertical face or upper
    vertical face closer to viewer
  • Schroder staircase viewed either from above or
    from below

44
  • two well-known ambiguous line drawings

45
  • two well-known ambiguous line drawings

46
  • two well-known ambiguous line drawings

47
  • inherent ambiguity of line drawing owing to
    complete loss of depth

48
18.3.2 Wire Frame Representation
  • general-viewpoint assumption none of the
    following situations
  • 1. two vertices of scene objects represented at
    same picture point
  • 2. two scene edges seen as single line in picture
  • 3. vertex seen exactly in line with unrelated
    edge

49
18.3.2 Wire Frame Representation
  • general-viewpoint assumption heart of
    line-drawing interpretation
  • viewpoint in perspective projection center of
    projection
  • viewpoint in orthographic projection direction
    of projection

50
  • subjective contours of Kanizsa white occluding
    triangle in space

51
(No Transcript)
52
18.3.2 Wire Frame Representation
  • line labels for visible projections of
    surface-normal discontinuities

53
(No Transcript)
54
  • four basic ways in which three planar surfaces
    can form polyhedron vertex

55
  • all possible distinct appearances of trihedral
    vertices of polyhedra

56
  • complete junction catalog for line drawings of
    trihedral-vertex polyhedra

57
  • complete junction catalog for line drawings of
    trihedral-vertex polyhedra

58
  • joke

59
18.3.3 Surface-Edge-Vertex Representation
  • VISIONS system Visual Integration by Semantic
    Interpretation of Natural Scenes
  • PREMIO system Prediction in Matching Images to
    Objects
  • PREMIO 3D object model hierarchical, relational
    model with five levels
  • 5 levels world, object, face/edge/vertex,
    surface/boundary, arc/2D, 1D piece

60
18.3.3 Surface-Edge-Vertex Representation
  • world level arrangement of different objects in
    world
  • object level arrangement of different faces,
    edges, vertices forming objects
  • face level describes face in terms of surfaces
    and boundaries
  • surface level specifies elemental pieces forming
    surfaces

61
18.3.3 Surface-Edge-Vertex Representation
  • 2D piece level describes pieces and specifies
    arcs forming boundaries
  • 1D piece level describes elemental pieces
    forming arcs
  • SDS spatial data structure
  • A/V attribute-value table

62
(No Transcript)
63
18.3.4 Sticks, Plates, and Blobs
  • sticks, plates, blobs model rough models of 3D
    objects used in rough-matching
  • sticks long, thin parts with only one
    significant dimension
  • stick cannot bend very much
  • stick has two logical endpoints, set of interior
    points, center of mass

64
18.3.4 Sticks, Plates, and Blobs
  • plates flattish wide parts with two nearly flat
    surfaces
  • plates have two significant dimensions
  • plate surfaces cannot fold very much
  • plate has set of edge points, set of surface
    points, center of mass
  • blobs parts with three significant dimensions
  • blob can be bumpy but cannot have concavities
  • blob set of surface points and center of mass
  • sticks, plates, blobs near-convex

65
(No Transcript)
66
18.3.4 Sticks, Plates, and Blobs
  • attribute-value table contains global attributes
  • simple-parts relation lists the parts and their
    attributes
  • connects-supports relation gives connections
    between pairs of parts
  • triples relation specifies connections between
    three parts at a time
  • parallel relation lists pairs of parts that are
    parallel
  • perpendicular relation lists pairs of parts that
    are perpendicular
  • TYPE 1 for stick, 2 for plate, 3 for blob

67
  • full relational structure of sticks-plates-blobs
    model of chair object

68
(No Transcript)
69
18.3.5 Generalized Cylinder Representation
  • generalized cylinder volumetric primitive
    defined by axis and cross-section
  • cross section swept along axis, creating a solid
  • e.g. actual cylinder generalized cylinder whose
    axis is straight-line segment and whose cross
    section is circle of constant radius
  • e.g. cone generalized cylinder whose axis is
    straight-line segment and cross section is circle
    with radius initially zero to maximum

70
18.3.5 Generalized Cylinder Representation
  • e.g. rectangular solid generalized cylinder
    whose axis is straight line segment and cross
    section is constant rectangle
  • e.g. torso generalized cylinder whose axis is
    circle and whose cross section is constant circle
  • generalized cylinder representation uses
    generalized cylinders as primitives

71
18.3.5 Generalized Cylinder Representation
  • surface-edge-vertex model very precise
  • sticks-plates-and-blobs model very rough
  • generalized cylinder model somewhere in between

72
18.3.5 Generalized Cylinder Representation
  • person modeled roughly as cylinders for head,
    torso, arms, legs
  • dotted lines axes of cylinders

73
18.3.6 Superquadric Representation
  • superquadrics lumps of clay deformable and can
    be glued into object models
  • superquadric models mainly used with range data

74
18.3.6 Superquadric Representation
  • Superquadrics are a flexible family of
    3-dimensional parametric objects, useful for
    geometric modeling. By adjusting a relatively few
    number of parameters, a large variety of shapes
    may be obtained.

75
  • range data image of (a) a doll, (b) its
    superquadric fit (c), (d) wire frame

76
  • joke

77
18.3.7 Octree Representation
  • octree encoding geometric modeling technique
    used to represent 3D objects
  • octree encoding used in computer vision,
    robotics, computer graphics
  • octree hierarchical 8-ary tree structure
  • each node in octree corresponds to cubic region
    of universe
  • full if cube is completely enclosed by 3D object
  • empty if cube contains no part of object

78
18.3.7 Octree Representation
  • partial if cube partly intersects object
  • full, empty, partial correspond to black,
    white, gray in quadtrees
  • node with label full or empty has no children
  • partial has eight children representing
    partition of cube into octants

79
(No Transcript)
80
18.3.8 The Extended Gaussian Image
  • 3D object collection of surface normals, one at
    each point of object surface
  • planar surface all points on surface map to same
    surface normal
  • convex with positive curvature everywhere
    distinct surface normal everywhere
  • Gaussian sphere unit sphere
  • set of surface normals mapped to Gaussian sphere
    tail at center head outward
  • Gaussian image of object resultant set of points
    on Gaussian sphere

81
(No Transcript)
82
18.3.8 The Extended Gaussian Image
  • for planar objects Gaussian image not
    invertible, not precise enough for use
  • small surface patch of object
  • corresponding surface patch on Gaussian
    sphere
  • Gaussian curvature K

83
18.3.8 The Extended Gaussian Image
  • point on Gaussian sphere
    corresponding to point (u, v) on object surface
  • extended Gaussian image
  • planar region Gaussian curvature 0, point mass
    in extended Gaussian image

84
18.3.9 View-Class Representation
  • view classes each representing set of viewpoints
    sharing some property
  • property e.g. same object surfaces visible
  • property e.g. same line segments visible
  • property e.g. relational distances between
    relational structures are similar
  • characteristic views sets producing
    topologically isomorphic line drawings

85
18.3.9 View-Class Representation
  • three view classes of cube producing
    topologically isomorphic line drawings

86
(No Transcript)
87
18.3.9 View-Class Representation
  • aspect graph of object graph structure where
  • 1. each node represents topologically distinct
    view of object
  • 2. a node for each such view of object
  • 3. each arc represents a visual event at
    transition
  • 4. there is an arc for each such transition

88
(No Transcript)
89
18.4 General Frameworks for Matching
  • matching finding correspondence between two
    entities
  • consistent labeling procedures examples of
    matching algorithms

90
18.4.1 Relational-Distance Approach to Matching
  • relational distance compares two structures and
    determines similarity
  • Relational-Distance Definition
  • relational description
  • sequence of relations
  • set of parts of entity being described
  • relation indicating various relationships
    among parts
  • relational description with part set A
  • relational description with part set B

91
18.4.1 Relational-Distance Approach to Matching
  • assumption A B, otherwise add dummy parts
    to smaller set
  • f any one-one, onto mapping from A to B
  • N positive integer
  • composition R f of relation R with
    function f

92
18.4.1 Relational-Distance Approach to Matching
  • f maps parts from set A to parts from set B
  • structural error of f for ith pair of
    corresponding relations in ,
  • total error of f with respect to ,
  • relational distance between
    ,

93
18.4.1 Relational-Distance Approach to Matching
  • best mapping from to mapping f
    that minimizes total error

94
18.4.1 Relational-Distance Approach to Matching
  • Relational-Distance Examples
  • best mapping from to
    is
  • for this mapping

95
18.4.1 Relational-Distance Approach to Matching
  • two digraphs whose relational distance is 3

96
(No Transcript)
97
18.4.1 Relational-Distance Approach to Matching
  • Relational Distance as a Metric
  • relational distance used to determine similarity
    of unknown object to model
  • relational distance used to compare object
    models to group models
  • f relational isomorphism if f one-one, onto from
    A to B and E(f) 0
  • f A B relational isomorphism ,
    isomorphic
  • GD relational-distance measure

98
18.4.1 Relational-Distance Approach to Matching
  • arbitrary relational descriptions
  • , , metric property of GD
  • 1. isomorphic
  • 2.
  • 3.

99
18.4.1 Relational-Distance Approach to Matching
  • Attributed Relational Descriptions and Relational
    Distance
  • extend relational description and relational
    distance to include
  • properties of parts
  • properties of the whole
  • properties of these relationships

100
  • joke

101
18.4.2 Ordered Structural Matching
  • definition of ordering on primitives greatly
    reduces complexity of search

102
18.4.3 Hypothesizing and Testing with Viewpoint
Consistency Constraint
  • viewpoint consistency constraint
  • The locations of all projected model features
    in an image must be consistent with projection
    from a single viewpoint.

103
18.4.4 View-Class Matching
  • if 3D object represented by view-class model,
    matching divided into 2 stages
  • 1. determining view class of object
  • 2. determining precise viewpoint within that view
    class

104
18.4.4 View-Class Matching
  • Determining View Class
  • relational pyramid hierarchical relational
    structure to represent view class
  • Level-1 primitives straight- and curved-line
    segments
  • Level-2 relations junctions and loops
  • Level-3 relations adjacency, collinearity,
    junction parallelness, loop-inside-loop

105
18.4.4 View-Class Matching
  • Pose Determination within View Class
  • relational pyramid hierarchical, relational
    structure to constrain matching

106
18.4.5 Affine-Invariant Matching
  • set of interest points
    lying in plane
  • rotation matrix relating model reference frame to
    camera reference frame
  • translation of object reference frame to camera
    reference frame

107
18.4.5 Affine-Invariant Matching
  • f distance between image plane and center of
    perspectivity
  • observed image data points
    by perspective projection
  • when translation in z-direction large
    compared with

108
18.4.5 Affine-Invariant Matching
  • A 2 x 2 (scaling, rotation, skewing) matrix
  • b 2D (translation) vector
  • affine 2D correspondence Aw b

109
18.4.5 Affine-Invariant Matching
  • Affine Transformation of Points in a Plane
  • necessary and sufficient to define plane
    uniquely 3 noncollinear points

110
18.4.5 Affine-Invariant Matching
  • The Hummel-Wolfson-Lamdan Matching Algorithm
  • to match noncollinear triplets in model interest
    points with scene
  • preprocessing convert model interest points into
    affine-invariant model
  • recognition match model against image using
    affine representation

111
18.4.5 Affine-Invariant Matching
  • Shortcomings of the Affine-Invariant Matching
    Technique
  • affine-invariant matching technique
    mathematically sound in noiseless case
  • shortcomings of affine-invariant matching in
    practice
  • 1. if three noncollinear points not numerically
    stable, points not reliable
  • 2. coordinates of detected interest points noisy
    in real image
  • 3. partial object symmetries may cause wrong
    matching

112
18.4.5 Affine-Invariant Matching
  • An Explicit Noise Model and Optimal Voting

113
18.5 Model Database Organization
  • organize database of models to allow rapid
    access to most likely candidate
  • group similar relational models into clusters and
    choose representative
  • arrows indicate mapping from parts of object 2
    to parts of other objects
  • cluster of object models whose representative is
    object 2

114
(No Transcript)
115
  • END
Write a Comment
User Comments (0)
About PowerShow.com