Computer Vision Image Features

1
Computer VisionImage Features

• Instructor Dr. Sherif Sami
• Lecture 4

2
Image Features

• Image features may appear in two contexts

• Global properties of the image (average gray
  level, etc) global features
• Parts of the image with special properties (line,
  circle, textured region) local features

• Here, assume second context for image features

• Local, meaningful, detectable parts of the image

• Detection of image features
• Detection algorithms produce feature
  descriptors
• Example line segment descriptor coordinates of
  mid-point, length, orientation

3
Edge Detection

• Definition of edges

• Edges are significant local changes of intensity
  in an image.
• Edges typically occur on the boundary between two
  different regions in an image.

4
Edge Detection

• Examples

5
Edge Detection

• Goal of edge detection

• Produce a line drawing of a scene from an image
  of that scene.
• Important features can be extracted from the
  edges of an image (e.g., corners, lines, curves).
• These features are used by higher-level computer
  vision algorithms (e.g., segmentation,
  recognition).

6
Edge Detection

• Not that easy

7
Edge Detection

• What causes intensity changes?

• Geometric events
• object boundary (discontinuity in depth and/or
  surface color and texture)
• surface boundary (discontinuity in surface
  orientation and/or surface color and texture)
• Non-geometric events
• specularity
• shadows (from other objects or from the same
  object)
• inter-reflections

8
Edge Detection

• Edge descriptors

• Edge normal unit vector in the direction of
  maximum intensity change.
• Edge direction unit vector to perpendicular to
  the edge normal.
• Edge position or center the image position at
  which the edge is located.
• Edge strength related to the local image
  contrast along the normal.

9
Modeling Intensity Changes

• Edges can be modeled according to their intensity
  profiles

• Step edge
• the image intensity abruptly changes from one
  value to one side of the discontinuity to a
  different value on the opposite side.

• Ramp edge
• a step edge where the intensity change is not
  instantaneous but occurs over a finite distance.

10
Modeling Intensity Changes

• Ridge edge
• the image intensity abruptly changes value but
  then returns to the starting value within some
  short distance
• generated usually by lines

11
Modeling Intensity Changes

• Roof edge
• a ridge edge where the intensity change is not
  instantaneous but occurs over a finite distance
• generated usually by the intersection of surfaces

12
Edge Detection

• The four steps of edge detection

• Smoothing suppress as much noise as possible,
  without destroying the true edges.
• Enhancement apply a filter that responds to
  edges in the image
• Detection determine which edge pixels should be
  discarded as noise and which should be retained
  (usually, thresholding provides the criterion
  used for detection).
• Localization determine the exact location of an
  edge (sub-pixel resolution might be required for
  some applications, that is, estimate the location
  of an edge to better than the spacing between
  pixels). Edge thinning and linking are usually
  required in this step.

13
Edge Detection

• Edge detection using derivatives

• Calculus describes changes of continuous
  functions using derivatives.
• An image is a 2D function, so operators
  describing edges are expressed using partial
  derivatives.
• Points which lie on an edge can be detected by
  either

• detecting local maxima or minima of the first
  derivative
• detecting the zero-crossing of the second
  derivative

14
Edge Detection

• Edge detection using derivatives cont.

15
Edge Detection

• Differencing 1D signals

• To compute the derivative of a signal, we
  approximate the derivative by finite differences
• Computing the 1st derivative

16
Edge Detection

• Computing the 1st derivative cont.

Backward difference
Forward difference
Central difference
17
Edge Detection

• Computing the 1st derivative cont.

• Examples using the edge models and the mask -1
  0 1 (centered about x)

18
Edge Detection

• Computing the 2nd derivative

• This approximation is centered about x 1
• By replacing x 1 by x we obtain

19
Edge Detection

• Computing the 2nd derivative cont.

20
Edge Detection

• Computing the 2nd derivative cont.

• Examples using the edge models

21
Edge Detection

• Image derivatives

Image I
22
Edge Detection

• Image derivatives

Image I
23
Derivatives and Noise

• Derivatives are strongly affected by noise
• obvious reason image noise results in pixels
  that look very different from their neighbors
• The larger the noise - the stronger the response

• What is to be done?
• Neighboring pixels look alike
• Pixel along an edge look alike
• Image smoothing should help
• Force pixels different to their neighbors
  (possibly noise) to look like neighbors

24
Derivatives and Noise
Increasing noise

• Need to perform image smoothing as a preliminary
  step
• Generally use Gaussian smoothing

25
Edge Detection

• Possible detectors

• Gradient operators
• Roberts
• Prewitt
• Sobel
• Gradient of Gaussian (Canny)
• Laplacian of Gaussian (Marr-Hildreth)
• Facet Model Based Edge Detector (Haralick)

26
Edge Detection Using the Gradient

• Definition of the gradient

• To save computations, the magnitude of gradient
  is usually approximated by

27
Edge Detection Using the Gradient

• Properties of the gradient

• The magnitude of gradient provides information
  about the strength of the edge
• The direction of gradient is always perpendicular
  to the direction of the edge

• Main idea

• Compute derivatives in x and y directions
• Find gradient magnitude
• Threshold gradient magnitude

28
Edge Detection Using the Gradient

• Estimating the gradient with finite differences

• Approximation by finite differences

29
Edge Detection Using the Gradient

• Using pixel-coordinate notation (remember j
  corresponds to the x direction and i to the
  negative y direction)

30
Edge Detection Using the Gradient

• Example
• Suppose we want to approximate the gradient
  magnitude at z5

• We can implement ?I/?x and ?I/?y using the
  following masks

Note Mx is the approximation at (i, j 1/2) and
My is the approximation at (i 1/2, j)
31
Edge Detection Using the Gradient

• The Roberts edge detector

• This approximation can be implemented by the
  following masks

Note Mx and My are approximations at (i 1/2, j
1/2)
32
Edge Detection Using the Gradient

• The Prewitt edge detector

• Consider the arrangement of pixels about the
  pixel (i, j)

• The partial derivatives can be computed by

• The constant c implies the emphasis given to
  pixels closer to the center of the mask.
• Setting c 1, we get the Prewitt operator

Note Mx and My are approximations at (i, j))
33
Edge Detection Using the Gradient

• The Sobel edge detector

• Setting c 2, we get the Sobel operator

Note Mx and My are approximations at (i, j))
34
Edge Detection Using the Gradient

• Main steps in edge detection using masks

35
Edge Detection Using the Gradient
(an example using the Prewitt edge detector -
dont divide by 2)
36
Edge Detection Using the Gradient

• Example

37
Edge Detection Using the Gradient

• Example cont.

38
Edge Detection Using the Gradient

• Example cont.

39
Edge Detection Using the Gradient
40
Edge Detection Using the Gradient
41
Edge Detection Using the Gradient

• Isotropic property of gradient magnitude

• The magnitude of gradient is an isotropic
  operator (it detects edges in any direction !!)

42
Edge Detection

• Practical issues

• Differential masks act as high-pass filters
  tend to amplify noise.
• Reduce the effects of noise - first smooth with a
  low-pass filter.

1. The noise suppression-localization tradeoff

• a larger filter reduces noise, but worsens
  localization (i.e., it adds uncertainty to the
  location of the edge) and vice-versa.

43
Edge Detection

1. How should we choose the threshold?

44
Edge Detection

1. Edge thinning and linking

• required to obtain good contours

45
Edge Detection

• Criteria for optimal edge detection

• Good detection the optimal detector must
  minimize the probability of false positives
  (detecting spurious edges caused by noise), as
  well as that of false negatives (missing real
  edges)
• Good localization the edges detected must be as
  close as possible to the true edges.
• Single response constraint the detector must
  return one point only for each true edge point
  that is, minimize the number of local maxima
  around the true edge

46
Edge Detection

• Examples

True edge
Poor robustness to noise
Poor localization
Too many responses
47
The Canny Edge Detector

• The Canny edge detector

• This is probably the most widely used edge
  detector in computer vision.
• Canny has shown that the first derivative of the
  Gaussian closely approximates the operator that
  optimizes the product of signal-to-noise ratio
  and localization.
• His analysis is based on "step-edges" corrupted
  by "additive Gaussian noise".

48
The Canny Edge Detector
49
The Canny Edge Detector

• The derivative of the Gaussian

50
The Canny Edge Detector

• Canny smoothing and derivatives

51
The Canny Edge Detector

• Canny gradient magnitude

image
gradient magnitude
52
Edge Detection

• Non-maxima suppression

• To find the edge points, we need to find the
  local maxima of the gradient magnitude.
• Broad ridges must be thinned so that only the
  magnitudes at the points of greatest local change
  remain.
• All values along the direction of the gradient
  that are not peak values of a ridge are
  suppressed.

53
Edge Detection

• Non-maxima suppression cont.

54
Edge Detection

• Non-maxima suppression cont.

• What are the neighbors?
• Look along gradient normal
• Quantization of normal directions

55
Edge Detection

• Canny Non-maxima suppression

gradient magnitude
thinned
56
Edge Detection

• Hysteresis thresholding / Edge linking

• The output of non-maxima suppression still
  contains the local maxima created by noise.
• Can we get rid of them just by using a single
  threshold?
• if we set a low threshold, some noisy maxima will
  be accepted too.
• if we set a high threshold, true maxima might be
  missed (the value of true maxima will fluctuate
  above and below the threshold, fragmenting the
  edge).

• A more effective scheme is to use two thresholds
• a low threshold tl
• a high threshold th
• usually, th 2tl

57
Edge Detection

• Hysteresis thresholding / Edge linking cont.

• The algorithm performs edge linking as a
  by-product of double-thresholding !!

58
Edge Detection

• Canny Hysteresis thresholding / Edge linking

thinned
59
Edge Detection
60
Edge Detection