Models and Matching

1
Models and Matching

• Methods of modeling objects and their
  environments
  
• Methods of matching models to sensed data for
  recogniton

2
Some methods to study

• Mesh models (surface)
• Vertex-edge-face models (surface)
• Functional forms superquadrics (surface)
• Generalized cylinders (volume)
• Voxel sets and octrees (volume)
• View class models (image-based)
• Recognition by appearance (image-based)
• Functional models and the Theory of affordances
  (object-oriented)
  

3
Models are what models do
4
What do models do?
5
Vertex-edge-face models

• Polyhedra and extensions

6
Vertex-Edge-Face model
7
Sample object
All surfaces are planar or cylindrical
8
Matching methods

• Hypothesize point correspondences
• Filter on distances
• Compute 3D alignment of model to data
• Verify positions of other model points, edges, or
  faces
  
• LOTS of work in the literature on this! Can work
  for many industrial objects (and human faces
  perhaps!)

9
Triangular meshes
10
Texture-mapped mesh dog
With each triangle is a mapping of its vertices
into pixels r, c of a color image. Thus any
point of any triangle can be assigned a color R,
G, B. There may be several images available to
create these mappings.
Courtesy of Kari Puli
11
Meshes are very general
They are usually verbose and often are too
detailed for many operations, but are often used
in CAD
12
Modeling the human body for clothing industry and

Multiple Structured light scanners used could
this be a service industry such as Kinkos?
Actually cross sections of a generalized cylinder
model.
13
Mesh characteristics
can be easy to generate from scanned data
14
Making mesh models
15
Physics-based models

• Can be used to make meshes
• Meshes retain perfect topology
• Can span spots of bad or no data

16
Physics-based modeling
17
Forces move points on the model halt at scanned
data
18
Fitting an active contour to image data
19
Balloon model for closed object surface
Courtesy of Chen and Medioni
20
Balloon evolution

• balloon stops at data points
• mesh forces constrain neighbors
• large triangles split into 4 triangles
• resulting mesh has correct topology

21
Physics-based models
Can also model dynamic behavior of solids (Finite
Element Methods)
22
Algorithms from computer graphics make mesh
models from blobs

• Marching squares applied to some connected image
  region (blob)
  
• Marching cubes applied to some connected set of
  voxels (blob)
  
• See a CG text for algorithms see the
  visualization toolkit for software
  

23
Volume model voxels, octrees
24
Simple object and its octree
25
Generalized cylinders
26
Generalized cylinders

• component parts have axis
• cross section function describes variation along
  axis
  
• good for articulated objects, such as animals,
  tools
  
• can be extracted from intensity images with
  difficulty

27
Extracting a model from a segmented image region
Courtesy of Chen and Medioni
28
Interpreting frames from video

• Can we match a frame region to a model?
• What about a sequence of frames?
• Can we determine what actions the body is doing?

29
Generalized cylinders
30
View class models

• Objects modeled by the distinct views that they
  can produce

31
aspect model of a cube
32
Recognition using an aspect model
33
View class model of chair
2D Graph-matching (as in Ch 11) used to evaluate
match.
34
Side view classes of Ford Taurus (Chen and
Stockman)
These were made in the PRIP Lab from a scale
model. Viewpoints in between can be generated fr
om x and y curvature stored on boundary.
Viewpoints matched to real image boundaries via
optimization.
35
Matching image edges to model limbs
Could recognize car model at stoplight or gate.
36
Appearance-based models

• Using a basis of sub images
• Using PCA to compress bases
• Eigenfaces (see older .pdf slides 14C)

37
Function-based modeling

• Object-oriented
• What parts does the object have
• What behaviors does it have
• What can be done with it?
• (See plastic slides of Louise Starkss work.)

38
Theory of affordances J.J. Gibson

• An object can be sittable a large number of
  chair types, a box of certain size, a trash can
  turned over,
  
• An object can be walkable the floor, ground,
  thick ice, bridge, ...
  
• An object can be a container a cup, a hat, a
  barrel, a box,
  
• An object can be throwable a ball, a book, a
  coin, an apple, a small chair,