SIFT: Scale Invariant Feature Transform

1
SIFT Scale Invariant Feature Transform

• David G. Lowe
• Distinctive image features from scale-invariant
  keypoints (IJCV 2004)

Presented By Kirill Dyagilev 317845089 Ayelet
Dominitz 034431304
2
Why Features?

• A brief yet comprehensive representation of the
  image
• Can be used for
• Image alignment
• Object recognition
• 3D reconstruction
• Motion tracking
• Indexing and database search
• More

3
Desired Feature Properties

• Robustness gt Invariance to changes in
  illumination, scale, rotation, affine,
  perspective
• Locality gt robustness to occlusion and clutter.
• Distinctiveness gt easy to match to a large
  database of objects.
• Quantity gt many features can be generated for
  even small objects
• Efficiency gt computationally cheap, real-time
  performance

4
Related Research

• Corner-based local interest points
• Moravec(1981), Harris (1992)
• Descriptors
• Correlation window around each corner
• Zhang (1995).
• Local, rotationally invariant
• Schmid Mohr (1997).
• Scale-invariance
• Crowley Parker (1984), Shokoufandeh et. al.
  (1999), Lindeberg(1993,1994), Mikolakczyk Shmid
  (2002).
• Maximally-Stable Extremal Regions (MSER)
• Matas(2002)

5
Algorithm

• Scale-space extrema detection
• Keypoint localization
• Orientation assignment
• Keypoint descriptor

6
Step 1 Scale-Space Extrema Detection

• Need to find characteristic scale for features
• Scale space representation

7
Step 1 Scale-Space Extrema Detection

• Mikolajczyk (2002) Experimentally, extrema of
  LoG gives best notion of scale

8
Step 1 Scale-Space Extrema Detection

• LoG is computationally expensive
• Approximation DoG
• Smoothed images should be computed anyway gt
  calculation is reduced to image subtraction.

9
Step 1 Scale-Space Extrema Detection
DoG scale space
10
Step 1 Scale-Space Extrema Detection

• X is selected if it is larger or smaller than all
  26 neighbors

11
Algorithm

• Scale-space extrema detection
• Keypoint localization
• Orientation assignment
• Keypoint descriptor

12
Step 2 Keypoint Localization

• (a) 233x189 image
  (b) 832
  DOG extrema
• Too many keypoints, some are unstable
• points with low contrast (sensitive to noise)
• points that are localized along an edge

13
Step 2 Keypoint Localization

• Low contrast points elimination
• Fit keypoint at to nearby data using
  quadratic approximation.
• Calculate the local maxima of the fitted
  function.
• Discard local minima

14
Step 2 Keypoint Localization
729 out of 832 are left after contrast
thresholding
15
Step 2 Keypoint Localization

• Edge keypoints are sensitive to noise, thus
  should be eliminated.
• Solution check cornerness of each keypoint.
• On the edge one of principle curvatures is much
  bigger than another.
• High cornerness ? No dominant principle curvature
  component.

16
Step 2 Keypoint Localization

• Principle curvature is proportional to
  eigenvalues of Hessian matrix
• Harris (1988) Equivalently,

17
Step 2 Keypoint Localization
536 out of 729 are left after cornerness
thresholding
18
Algorithm

• Scale-space extrema detection
• Keypoint localization
• Orientation assignment
• Keypoint descriptor

19
Step 3 Orientation assignment

• Required Rotation invariance of features
• Solution
• Assign orientation to feature based on local
  gradients
• Transform relative data accordingly.

20
Step 3 Orientation assignment

• Create weighted (magnitude Gaussian) histogram
  of local gradient directions computed at selected
  scale
• Assign canonical orientation at peak of smoothed
  histogram
• For location of multiple peaks multiply key point

21
Algorithm

• Scale-space extrema detection
• Keypoint localization
• Orientation assignment
• Keypoint descriptor

22
Step 4 Keypoint descriptor

• We have assigned location, scale, and orientation
  to each keypoint
• ? Impose a repeatable local 2D coordinate system
• ? Provide invariance to these parameters.
• Remaining goal
• Define local descriptor invariant to remaining
  variations
• Illumination
• 3D Viewpoint

23
Step 4 Keypoint descriptor

• Create 16 gradient histograms (8 bins)
• Weighted by magnitude and Gaussian window ( s is
  half the window size)
• Histogram and gradient values are interpolated
  and smoothed

gt Feature vector (128)
24
Step 4 Keypoint descriptor

• Invariance to affine illumination changes
• Gains do not affect gradients
• Normalization to unit length removes contrast
• Non-linear illumination changes
• Saturation affects magnitudes much more than
  orientation
• Threshold gradient magnitudes to 0.2 and
  renormalize

25
Step 4 Keypoint descriptor

• Justification
• Inspired by the human visual system
• Parameters r ( of bins) and nn ( of
  histograms) chosen empirically

26
Keypoint Matching

• Nearest Neighbor algorithm based on L2 distance
• How to discard bad matches?
• Threshold on L2 gt bad performance
• Solution threshold on ratio

27
Applications
28
Typical Usage

• For set of database images
• Compute SIFT features
• Save descriptors to database
• For query image
• Compute SIFT features
• For each descriptor find its match in the
  database
• Verify object recognition by checking feature
  consistency (relative location, scale and
  orientation)
• RANSAC
• Hough transform
• Verify with affine transform

29
Recognition under occlusion
30
Test of illumination Robustness

• Same image under differing illumination

273 keys verified in final match
31
Location recognition
32
Image Registration Results
Brown Lowe 2003
33
Evaluation

• Robustness to
• Viewpoint
• Lighting
• Scale
• Repeatability (50) Lowe
• Textured planar surfaces 50? rotation
• Improve by adding transformed images to database
• 3D objects considerably less
• Weak point Lowe key point detection

34
Evaluation

• Very popular 3000 citations
• Mikolajczyk Schmid
• Close second best descriptor after GLOH
• Moreels Perona
• The best descriptor
• There is a better detector affine-rectified
• K.Mikolajczyk, C.Schmid. A Performance
  Evaluation of Local Descriptors. (CVPR, 2003)
• P.Moreels, P.Perona. Evaluation of Features
  Detectors and Descriptors based on 3D objects.
  (ICCV,2005)

35
Evaluation

• Computationally inexpensive
• Efficient codes available on www
• Simplified implementation gt
• results considerable worse
• Doesnt exploit color information

36
(No Transcript)