Chapter 5' Segmentation

1
Chapter 5. Segmentation

• In image compression or enhancement, the desired
  output is a picture.
• In image analysis or scene analysis, output is a
  description of input picture.
• Examples
• (1) Input text (machine printed or handwritten),
  output ASCII codes of text
• (2) Input a nuclear bubble chamber picture,
  desired to locate and detect certain types of
  events'' (e.g. particle collisions).
  Description consists of a set of coord. and names
  of event types.

2

• (3) Input a TV image of a pile of parts, desired
  output is a plan of action that can be used by a
  robot to assemble a device out of parts. It
  requires identification and location of
  individual parts in the scene.
• The description refers to specific parts (regions
  or objects) in the picture.
• It is necessary to segment the picture into these
  parts.
• Ex. To identify the individual characters in
  text, they must first be singled out.

3

• Ex. To locate bubble chamber events, the bubble
  tracks and their ends and branches must be found.
• There is no single standard approach to
  segmentation
• The perceptual processes involved in segmentation
  by the human visual system, e.g. Gestalt laws of
  organization, are not yet well understood.
• Success of segmentation must be judged by the
  utility of the description.

4
(No Transcript)
5
5.1 Pixel Classification
Threshold a picture into foreground and
background.
Classify pixels into edge and not edge by
thresholding the response of some difference
operator. Suppose classifying two-class pixels
based on gray levels
Let the prob. densities of values of z for two
classes p(z1), p(z2). Let the a priori prob.
of two classes p(1), p(2), p(1)p(2)1
6

• Overall prob. density of values of z for the
  entire picture p(1)p(z1)p(2)p(z2)
• Suppose we classify pixels by thresholding z
• at t if zltt, class 1, if z?t class 2.
• The prob. of misclassifying a class-2 point as
• class 1
• P (t2)
• The prob. of misclassifying a class-1 point as
• class 2
• 1-P (t1)
• Overall misclassification prob.
• p(1)1-P(t1) p(2)P(t2)
  (2)

7
To find the value of t for which this
misclassification prob. is minimum. Differentiate
(2) w.r.t. t and set to zero.
-p(1)p(t1)p(2)p(t2)0
(3) Suppose prob. density p(z1),p(z2) are
Gaussian with means, variances
8
If we set z t in (4), substitute in (3), take
logarithms of both sides
9
5.1.2 Gray Level Thresholding

• Consider the pictures of writing, chromosomes,
  and clouds.
• The histogram having two peaks is called bimodal.
  
• Semi-thresholding - Display the pixels that
  are
• lighter than threshold as white, and leave those
• darker than threshold with their original gray
  levels.
• Threshold Automatic Selection
• (1) Find two local maxima that are at least
• some minimum distance apart, say

10

• Good Threshold Desired Property
• (1) Busyness The number of adjacencies between
• above-threshold and below-threshold pixels.
• A good choice of t will minimize this
  busyness.
• (2) If we know what fraction of pixels should be
• above threshold, we choose t accordingly.
• Ex. What fraction of the page is characters?

11

• (3) The individual sizes of above-threshold
  objects are known, choose t to give the
  desired sizes as measured in the picture.
• Multilevel Thresholding
• If a picture contains more than two types of
  regions, it may still be possible to segment it
  by applying several thresholds.
• Multimodal A histogram having several peaks.
• It is sometimes useful to select a gray level
  range that
• corresponds to a valley, not a peak, on the
  histogram.

12
(No Transcript)
13
5.2 EDGE DETECTION

• Local operations that can be used to detect
• various types of local features, such as edges
• and curves, in a picture.
• Local features usually involve abrupt changes
• in gray level
• (1) Edge the gray level changes abruptly as the
  
• border between the regions is crossed.
• (2) Line (Curve) the gray level is relatively
• constant except along a thin strip.
• (3) Spot the gray level is relatively constant
• except at one location.

14

• 5.2.1 Difference Operators
• Yield high values at places where the gray
• level is changing rapidly.
• a. Gradient
• Construct derivative operators that are
  isotropic,
• i.e. rotation invariant (rotate f then apply
  operator
• gives same results as applying operator then
  rotate)

Y
X
Y
X
15

• Find the direction ? in which the partial
• derivative of f has maximum.

16
(No Transcript)
17

18

19

20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
0 0 0 1 1 1 ... 0 0 0 1 1 1 ...

• These responses can be sharpened by
  suppressing nonmaxima in the direction across the
  edge, i.e. setting a response to zero if there is
  a stronger response sufficiently close to it.

28

• Sufficiently close - close than the size of the
  
• averaging neighborhood.
• Ex. Use 3x3 averages, the responses

29
(No Transcript)
30

• The one based on averages have weaker, blurred
• responses to edges that are not optimally
  oriented.
• Use sum of absolute, or RMS of two operators
• rather than maximum of their absolute values.
• So far, unweighted averages, but weighted
• averages can also be used.

• Give greater weight to points lying closer to
  (x,y).

31

• 5.2.2 Edge Matching and Fitting
• Since an ideal edge is a steplike pattern, one
• approach to detecting edges is to match such
• patterns, in various orientations.
• Take the orientation that gives the best match
• as the edge orientation
• Ex.

32

• Correspond to central differences in four
  directions
• take maximum of absolute as an edge measure.
• Ex. Generalized Prewitt Operator
  (Difference-of-Averages),
• Constant 1/3
• 1 1 1 1 1 0 1 0 -1 0 -1 -1
  -1 -1 -1 -1 -1 0 -1 0 1 0 1 1
• 0 0 0 1 0 -1 1 0 -1 1 0 -1
  0 0 0 -1 0 1 -1 0 1 -1 0 1
• -1 -1 -1, 0 -1-1, 1 0 -1, 1 1 0,
  1 1 1, 0 1 1, -1 0 1 -1 -1 0.

33
Ex Kirsch (constant factor1/15) 5 5 5
5 5 -3 5 -3 -3 -3 -3 -3 -3
-3 -3 -3 -3 -3 -3 -3 5
-3 5 5 -3 0 -3 5 0 -3 5 0
-3 5 0 -3 -3 0 -3 -3
0 5 -3 0 5 -3 0 5 -3 -3
-3, -3 -3 -3, 5 -3 -3, 5 5 -3,
5 5 5, -3 5 5, -3 -3
5, -3 -3 -3. Step Fitting
34
(No Transcript)
35
1
-1
1
-1
1
36
-1
b
1
a
37
(No Transcript)
38

• Take partial derivatives w.r.t. a b to be the
  
• following eq. (12)

39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
5.2.3 Edge Detection

• - Detect edges by thresholding the responses of
  edge operators.
• - This problem can be formulated in
  decision-theoretic terms.
• Ex. Suppose prob. of two types of regions P(1)
  and P(2). Prob. of points of two regions being
  border points P(12) and P(21).
• Prob. densities of edge operator responses ? in
  the region interiors and on their borders
• p(?1), p(?2), p(?12), p(?21)
• Then we can compute the prob. densities
  p(1?), p(2?), p(12?), p(21?)

51

• A given edge response arises from an interior or
  border point.
• Simple case suppose ? ?x horizontal first
  difference operator
• Suppose gray levels of adjacent points interior
  to a region are independent and have same prob.
  density q(zr), where r1 or 2
• If possible gray levels are 0, 1, .,k then
  possible values of ?x are -k, ,0, , k.

52
(No Transcript)
53
Ex. Yakimovsky (1976) - Decide whether a given
piece S of a picture is interior to a
homogeneous region or overlaps two such
regions. - Suppose each region has normally
distributed gray levels with means and
standard deviation and points are
mutually independent.
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
5.3 FEATURE DETECTION
Thin vertical line Detect line segments that are
darker (higher value) than their background.
Interested in finding the shape (thin, vertical)
rather than specific gray levels of line
background, it makes sense to convert f into
outline form, say by differentiation or high-pass
filtering, before cross-correlation.
58
(No Transcript)
59

• Output has high value 3(a-b) at points of the
  line.
• Disadvantage respond some patterns not linelike
  more strongly than thin line.
• Ex. A vertical step edge
• c c d d 0
  3(c-d)/2 3(c-d)/2 0
• c c d d ? 0
  3(c-d)/2 3(c-d)/2 0
• c c d d 0
  3(c-d)/2 3(c-d)/2 0

60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70

• Algorithm (Line Detection)
• 1. Quantize paramenter space between min max
  for (m,c).
• 2. Initialize A(c,m)0.
• 3. If (x,y) is a line element, all points
  A(c,m) for m and c satisfying c-mxy within
  the limits of digitization.
• 4. Local maxima correspond to collinear points.
  The values of accumulator array are the number of
  points on the line.

71
(No Transcript)
72

• Disadvantage Slope becomes infinite for vertical
  lines. To avoid this, use
• (angle, distance from origin)

73

• Disadvantages
• (1) in digital data are less
  accurate.
• (2) Computation and size of accumulator array
  increase exponentially as number of parameters.

74
(No Transcript)
75

• 5.3.4 Edge and Curve Linking
• In practice, we will also obtain many noise''
  edge points, also miss some edge points.
• Methods (1) Apply curve detection in 6.3.2 to
  the edge output eliminate noise while preserving
  the edge points that do lie on curves provided
  the gaps in the curves are not too wide.
• (2) Examine neighbors of each edge point in the
  direction along the edge. If their slopes do not
  differ greatly from the slope of given edge
  point, linked to them. If not, deleted.

76
5.4 SEQUENTIAL SEGMENTATION

• Up to now, parallel segmentation'' - the
  processing that was done at each point did not
  depend on results already obtained at other
  points.
• Seg. segmentation - need not be performed at each
  point, but only at points that extend objects
  that have already been detected. (track)

77

• Raster Scan Track from row-by-row of the
  picture, as we scan the picture row-by-row in the
  manner of a TV raster.
• Raster tracking criterion (Assume d gt t )
• (1) detection criterion (x,y) accepted if (x,y)
  gt d
• (2) tracking criterion check neighbors of (x,y)
• Accept any of the points (x-1,y-1), (x,y-1) and
  (x1,y-1) provided they have gray levels gt t

78

• Disadvantage Results depend on the orientation
  of the raster and the direction in which it is
  scanned.
• Ex. If a strong curve gradually becomes weaker
  from row to row, the acceptance criterion is
  relatively permissive. If curves start out weak
  and get stronger, we might not detect it.

79

• Omni-directional Tracking - scan in both
  direction, carry out tracking procedure for each
  of the scans independently, and combine the
  results.
• Ex. If curves are nearly horizontal, we cannot
  track since the crossings of successive rows are
  many columns apart.
• This directionality problem can be avoided by
  using two perpendicular rasters.
• Search Techniques in Tracking (not only
  look-ahead, allow back-up)
• If an acceptance decision seems to be leading to
  a series of poor subsequent acceptance, one can
  go back and alter decision.

80

• Ex. For any curve C, let be average
  contrast of C minus its average curvature,