Local invariant features - PowerPoint PPT Presentation

1 / 48
About This Presentation
Title:

Local invariant features

Description:

Scale & affine invariant ... affine (valide for local planar objects) ... in case of an affine transformation. Normalization of the image patch ... – PowerPoint PPT presentation

Number of Views:130
Avg rating:3.0/5.0
Slides: 49
Provided by: movi8
Category:

less

Transcript and Presenter's Notes

Title: Local invariant features


1
Local invariant features
  • Cordelia Schmid
  • INRIA, Grenoble

2
Overview
  • Introduction to local features
  • Harris interest points SSD, ZNCC, SIFT
  • Scale affine invariant interest point detectors
  • Evaluation and comparison of different detectors
  • Region descriptors and their performance

3
Introduction
  • Local invariant photometric features

local descriptor
Local robust to occlusion/clutter no object
segmentation Photometric distinctive Invariant
to image transformations illumination changes
4
Partial visibility/occlusion
5
Clutter (additional objects)
6
Image transformation rotation
7
Image transformations scale change
8
Illumination variations
9
Viewpoint changes
10
Local features - history
  • Line segments Lowe87, Ayache90
  • Interest points cross correlation Z. Zhang et
    al. 95
  • Rotation invariance with differential invariants
    SchmidMohr96
  • Scale affine invariant detectors Lindeberg98,
    Lowe99, TuytelaarsVanGool00,
    MikolajczykSchmid02, Matas et al.02
  • Dense detectors and descriptors LeungMalik99,
    Fei-Fei Perona05, Lazebnik et al.06
  • Contour and region (segmentation) descriptors
    Shotton et al.05, Opelt et al.06, Ferrari et
    al.06, Leordeanu et al.07

11
Local features
  • 1) Extraction of local features
  • Contours/segments
  • Interest points regions
  • Regions by segmentation
  • Dense features, points on a regular grid
  • 2) Description of local features
  • Dependant on the feature type
  • Segments ? angles, length ratios
  • Interest points ? greylevels, gradient histograms
  • Regions (segmentation) ? texture color
    distributions

12
Local features Contours/segments
13
Local features interest points
14
Local features segmentation
15
Application Matching
Find corresponding locations in the image
16
Application Image retrieval
Search for images with the same/similar object in
a set of images
17
Overview
  • Introduction to local features
  • Harris interest points SSD, ZNCC, SIFT
  • Scale affine invariant interest point detectors
  • Evaluation and comparison of different detectors
  • Region descriptors and their performance

18
Harris detector Harris Stephens88
Based on the idea of auto-correlation
Important difference in all directions gt
interest point
19
Harris detector
Auto-correlation function for a point
and a shift
20
Harris detector
Auto-correlation function for a point
and a shift

small in all directions
? uniform region
? contour
large in one directions
? interest point
large in all directions
21
Harris detector
22
Harris detector
Discret shifts are avoided based on the
auto-correlation matrix
23
Harris detector
Auto-correlation matrix
24
Harris detector
  • Auto-correlation matrix
  • captures the structure of the local neighborhood
  • measure based on eigenvalues of this matrix
  • 2 strong eigenvalues
  • 1 strong eigenvalue
  • 0 eigenvalue

gt interest point
gt contour
gt uniform region
25
Harris eigenvalues
26
Harris detector
  • Cornerness function
  • Interest point detection
  • Treshold (absolut, relatif, number of corners)
  • Local maxima

27
Harris - invariance to transformations
  • Geometric transformations
  • translation
  • rotation
  • similitude (rotation scale change)
  • affine (valide for local planar objects)
  • Photometric transformations
  • Affine intensity changes (I ? a I b)

28
Harris detector
Interest points extracted with Harris ( 500
points)
29
Comparison of patches - SSD
Comparison of the intensities in the neighborhood
of two interest points
image 2
image 1
SSD sum of square difference
Small difference values signifies similar patches
30
Comparison of patches
SSD
Invariance to photometric transformations?
Intensity changes (I ? I b)
Intensity changes (I ? aI b)
31
Cross-correlation ZNCC
zero normalized SSD
ZNCC zero normalized cross correlation
ZNCC values between -1 and 1, 1 when identical
patches in practice threshold around 0.5
32
Cross-correlation matching
Initial matches (188 pairs)
33
Global constraints
Robust estimation of the fundamental matrix
99 inliers
89 outliers
34
Local descriptors
  • Greyvalue derivatives
  • Differential invariants Koen87
  • SIFT descriptor Lowe99
  • Moment invariants Van Gool et al.96
  • Shape context Belongie et al.02

35
Greyvalue derivatives Image gradient
  • The gradient of an image
  • The gradient points in the direction of most
    rapid increase in intensity
  • The gradient direction is given by
  • how does this relate to the direction of the
    edge?
  • The edge strength is given by the gradient
    magnitude

Source Steve Seitz
36
Differentiation and convolution
  • Recall, for 2D function, f(x,y)
  • We could approximate this as
  • Convolution with the filter

Source D. Forsyth, D. Lowe
37
Finite difference filters
  • Other approximations of derivative filters exist

Source K. Grauman
38
Effects of noise
  • Consider a single row or column of the image
  • Plotting intensity as a function of position
    gives a signal

Source S. Seitz
39
Solution smooth first
f
Source S. Seitz
40
Derivative theorem of convolution
  • Differentiation is convolution, and convolution
    is associative
  • This saves us one operation

Source S. Seitz
41
Local descriptors
  • Greyvalue derivatives
  • Simple difference filters (-1,1)
  • Convolution with Gaussian derivatives

42
Local descriptors rotation invariance
Notation for greyvalue derivatives
43
Local descriptors rotation invariance
  • Invariance to image rotation differential
    invariants Koen87

gradient magnitude
Laplacian
44
Laplacian of Gaussian (LOG)
45
Local descriptors - rotation invariance
  • Estimation of the dominant orientation
  • extract gradient orientation
  • histogram over gradient orientation
  • peak in this histogram
  • Rotate patch in dominant direction

46
Local descriptors illumination change
  • Robustness to illumination changes

in case of an affine transformation
  • Normalization of derivatives with gradient
    magnitude
  • Normalization of the image patch with mean and
    variance

47
SIFT descriptor Lowe99
  • Approach
  • 8 orientations of the gradient
  • 4x4 spatial grid
  • soft-assignment to spatial bins, dimension 128
  • normalization of the descriptor to norm one
  • comparison with Euclidean distance

3D histogram
image patch
gradient
x
?
?
y
48
Invariance to scale changes
  • Scale change between two images
  • Scale factor s can be eliminated
  • Support region for calculation!!
  • In case of a convolution with Gaussian
    derivatives defined by
Write a Comment
User Comments (0)
About PowerShow.com