3-D Computer Vision CSc 83020

1
3-D Computer Vision CSc 83020

• Clustering methods and boundary representations

2
Image Segmentation

• Generate clusters (regions) of pixels that
  correspond to meaningful entities.
• Use metrics of closeness between values.
• Use algorithms for combining close values.
• Apply other constraints (connectivity).

3
Example (Forsyth Ponce)
4
Simple Clustering Methods

• Divisive clustering
• Everything is a big cluster at beginning
• Split recursively
• Agglomerative clustering
• Each data (pixel) is a cluster
• Merge

5
Simple clustering
6
Simple clustering

• What is a good inter-cluster distance?
• How many clusters are there?

7
Simple clustering
8
Clustering by K-means

• Assume that the number of clusters (k) is known.
• Each cluster has a center Ci (i1..k)
• Each data-point is a vector xj (j1..Number of
  pixels)
• Examples
• xjx-coord, y-coord, gray-value
• or xjgray-value
• or xjred-value, green-value, blue-value
• Assume that elements are close to center of
  clusters.
• Minimize

9
Clustering by K-means

• Iterative algorithm
• Allocate each point to center of closest cluster
  (assuming centers are known)
• Calculate centers of clusters (assuming
  allocations are known)
• How do we start?

10
Clustering by K-means
11
Example (Forsyth Ponce)
12
Example (Forsyth Ponce)
13
Example (Forsyth Ponce)
14
RANSAC

• RANdom SAmple Consensus
• Model fitting method
• Line-fitting example
• Fitting a line to a set of edges with 50
  outliers
• Least squares would fail
• Solution M-estimator or RANSAC

15
RANSAC (line-fitting example)

• Two edges (wout normal) define a line.
• General idea
• Pick two points.
• Write the equation of the line.
• Check how many other points are close to line.
• If number of close points is above threshold,
  done
• Otherwise, pick two new points.
• Questions
• Which points to pick?
• Complexity in worst case?

16
RANSAC (line-fitting example)
17
RANSAC (line-fitting example)

• How large should k (max. number of iterations)
  be?

18
RANSAC (line-fitting example)

• How large should k (max. number of iterations)
  be?
• Assume that w is the probability of picking a
  correct point (i.e. a point on the line).
• Since we are picking n (2 for lines) points,
  1-wn is the probability of picking n wrong
  points.
• If we iterate k times we want the probability of
  failure to be small i.e.
• (1-wn)k z gt k log(z)/log(1-wn)
• If z0.1 and w0.5 then k8
  (n2)
• If z0.01 and w0.5 then k16 (n2)
• If z0.001 and w0.1 then k 687 (n2)
• How is the formula affected by n?

19
RANSAC (Conclusions)

• When can this method be successful?
• Can we detect circles?
• In that case how many points do you need to fit a
  circle?
• Can we detect other shapes?

20
Boundary representation of regions
21
Representation of 2-D Geometric Structures

• To MATCH image boundary/region with MODEL
• boundary/region, they must represented in the
  same
• manner.
• Boundary Representation
• Snakes Extraction of arbitrary contours from
  image.
• Region Representation

22
Representation Issues

• Compact Easy to Store Match.
• Easy to manipulate compute properties.
• Captures Object/Model shape.
• Computationally efficient.

23
Boundary Representation
Polylines concatenation of line segments.
Breakpoint
Matching on the basis of of line
segments lengths of line segments angle between
consecutive segments
24
Running Least Squares Method
ei
Move along the boundary At each point find line
that fits previous points (Least
Squares) Compute the fit error ESum(ei) using
previous points If E exceeds threshold, declare
breakpoint and start a new running line fit.
25
Approximating Curves with Polylines

• Draw Straight line between end-points of curve
• For every curve point find distance to line.
• If distance is less than tolerance level for all
  points, Exit
• Else, pick point that is farthest away and use as
  breakpoint.
• Introduce new segments.
• Recursively apply algorithm to new segments.

26
?-s Curve
y
?
2p
6
5
p
4
3
?
s
1
2
x
s
y
4
5
3
6
1
2
x
27
?-s Curve
y
?
2p
6
5
p
4
3
?
s
1
2
x
s
y
4
?-s is periodic (2p) Horizontal section in ?-s
curve gt straight line in the Image Non
horizontal section in ?-s gt arc in Image
5
3
6
1
2
x
28
Slope-Density Function
y
H(?)
HISTOGRAM
Lines
?
s
?
x
p
2p
Arcs
29
Slope-Density Function
y
H(?)
HISTOGRAM
Lines
?
s
?
x
p
2p
Arcs
H(?) shifts as objects rotates. H(?) wraps around
30
Fourier Descriptors
y
Find ?(s) Define F(s) ?(s)-(2ps)/P P
Perimeter. 2p Period of F(s)
?
s
x
F(s) is a Continuous, Periodic function. Fourier
Series for Periodic Functions Fourier
Coefficients
Fks capture shape information Match shapes by
matching Fks Use finite number of Fks
31
Fourier Descriptors
Input Shape
Reconstruction
Power Spectrum
of coefficients
Fk
Note Reconstructed shapes are often not closed
since only a finite of Fks are used.
32
B-Splines

• Piecewise continuous polynomials used to
  INTERPOLATE
• between Data Points.
• Smooth, Flexible, Accurate.

x2
Spline X(s)
x1
s2
s1
x0
Data Point
xi
s
s0
x
N
BASIS FUNCTIONS
We want to find X(s) from points xi Cubic
Polynomials are popular
COEFFICIENTS
33
B-Splines
Bi(s) has limited support (4 spans)
i2
i-2
i-1
i
i1
s
Each span (i-gti1) has only 4 non-zero Basis
Functions Bi-1(s), Bi(s), Bi1(s), Bi2(s)
Bi1(s)
Bi(s)
Bi2(s)
Bi-1(s)
i
i1
s
t0
t
t1
34
B-Splines
4/6
Bi(s)
Bi1(s)
Bi-1(s)
Bi2(s)
1/6
i
i1
t0
t
t1
Cubic Polynomials
35
B-Splines
4/6
Bi(s)
Bi1(s)
Bi-1(s)
Bi2(s)
1/6
i
i1
t0
t
t1
Cubic Polynomials
36
B-Splines
If we compute v0, vN gt continuous
representation for the curve.
37
B-Splines
We have our N1 data points
And in matrix form
We can solve for the vis from the xis Boundary
condition for closed curves v0vN, v1vN1.
38
B-Splines
So, for any i we can find
xi(t)
t
xi
xi1
t0
t1
Note Local support. Spline passes through all
data points xi.
B-Spline demo http//www.doc.ic.ac.uk/df
g/AndysSplineTutorial/BSplines.html
39
Snakes
Elastic band of arbitrary shape. Located near the
image contour.
40
Snakes Implementation
Discrete version sum over the points.
How can we minimize e?
Greedy minimization For each point compute the
best move over a small area.
41
Snakes Implementation
Discrete version sum over the points.
How can we minimize e?
Greedy minimization For each point compute the
best move over a small area. Set ßi0 for
points of high curvature (corners)
42
Snakes Implementation
Discrete version sum over the points.
How can we minimize e?
Greedy minimization For each point compute the
best move over a small area. Set ßi0 for
points of high curvature (corners) Stop when a
user-specified fraction of points does not move.

43
Synthetic Results
Real Experiment
Snakes demo http//www.markschulze.net/snakes/
44
Region Representation
Spatial Occupancy Array
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

• Easy to implement.
• Large Storage area.
• Can apply set operations (unite/intersect).
• Expensive for matching!

45
Quad Trees
Efficient encoding of Spatial Occupancy
Array using Resolution Pyramids.
Black Fully Occupied.
White Empty.
Gray Partially Occupied.
46
Quad Tree
Level 0
Level 1
Level 2
Level 3
Quad Tree Generation Start with level 0. If
Black or White, Terminate. Else declare Gray
node expand with four sons. For each son
repeat above step.
NW
NE
SW
SE