Advanced Computer Vision Introduction

1
Advanced Computer VisionIntroduction

• Lecture 02
• Roger S. Gaborski

2
Corner Detection Basic Idea

• We should easily recognize the point by looking
  through a small window
• Shifting a window in any direction should give a
  large change in intensity

Source A. Efros
3
Corner Detection Mathematics
Change in appearance for the shift u,v
Source R. Szeliski
4
Corner Detection Mathematics
Change in appearance for the shift u,v
I(x, y)
E(u, v)
E(3,2)
E(0,0)
Source R. Szeliski
5
Corner Detection Mathematics
Change in appearance for the shift u,v
We want to find out how this function behaves for
small shifts
Second-order Taylor expansion of E(u,v) about
(0,0) (local quadratic approximation)
Source R. Szeliski
6
Corner Detection Mathematics
The quadratic approximation simplifies to
where M is a second moment matrix computed from
image derivatives
Source R. Szeliski
7
Interpreting the second moment matrix
First, consider the axis-aligned case (gradients
are either horizontal or vertical)
?1 and ?2 will be proportional to the principal
curvature of autocorrelation function. If either
eigenvalue ? is close to 0, then this is not a
corner, so look for locations where both are
large.
8
Interpreting the eigenvalues
Classification of image points using eigenvalues
of M
?2
Edge ?2 gtgt ?1
Corner?1 and ?2 are large, ?1 ?2E
increases in all directions
?1 and ?2 are smallE is almost constant in all
directions
Edge ?1 gtgt ?2
Flat region
?1
9
Defining Corner Response Function, R
Recall A a b c d Det(A) ad
bc Then R det(M)- a trace(M)2 ?1 ?2 - a
(?1 ?2) Where a .04 to .06
10
Corner Response Function
R lt 0
R gt 0
Edge
Corners
R small
R lt 0
Flat region
Edge
11
Harris Detector Algorithm

• Compute Gaussian Derivatives at each point
• Compute Second Moment Matrix M
• Compute Corner Response Function
• Threshold R
• Find Local Maxima

12
Harris Corner Detector

• Reference C.G. Harris and M.J. Stephens A
  Combined Corner and Edge Detector
• Code inspired by Peter Kovesi
• Derivative Masks dx -1, 0, 1-1, 0, 1-1, 0,
  1
• dy dx
• Image Derivatives
• Ix imfilter(im, dx, 'conv',same)
• Iy imfilter(im, dy, 'conv',same)
• Gaussian Filter
• g fspecial(gaussian, 6sigma, sigma)

13

• Smooth squared image derivative
• Ix2 imfilter (Ix.2, g, 'conv', same)
• Iy2 imfilter (Iy.2, g, 'conv', same)
• IxIy imfilter (Ix . Iy, g, 'conv', same)
• c (Ix2.Iy2 IxIy.2)./(Ix2Iy2).2

14
Non-maximal Suppression and Threshold

• Extract local maxima gray scale morphological
  dilation
• size 2radius1 radius is parameter
• mx imdilate(c,ones(size)) gray scale dilate
• cc (cmx)(cgtthresh) find maxima
• r,c find(cc) find row, col coordinates
• figure, imagesc( im), colormap(gray)
• hold on
• plot(c,r, rs), title(Corners)

15
100x100 Grid background 1, lines 1
16
(No Transcript)
17
Image rotated 45 Degrees
18
Image rotated 45 Degreessame parameters
19
(No Transcript)
20
(No Transcript)
21
Porsche Image
22
Harris Points
23
1983 Porsche
24
HW2 Due Tuesday, noon

• Work in teams of 2 or 3
• Write a Harris Detector Function (do not simply
  copy one from web, write your own)
• Experiment with grid image and Flower2 image
  and two interesting images of your choice
• Goals - Find all intersections on grid image
• Detect all petal end points on flower image
  better results that class lecture slide
• Email 1- write up including result images,
  observations and 2-MATLAB code

25
Object Recognition

• Issues
• Viewpoint
• Scale
• Deformable vs. rigid
• Clutter
• Occlusion
• Intra class variability

26
Current Work

• Fix
• Viewpoint
• Scale
• Rigid
• Explore affects of
• Intra class variability
• Clutter
• Occlusion

27
Goal

• Locate all instances of automobiles in a
  cluttered scene

28
Acknowledgements

• Students
• Tim Lebo
• Dan Clark
• Images used in presentation
• ETHZ Database, UIUC Database

29
Object Recognition Approaches

• For specific object class
• Holistic
• Model whole object
• Parts based
• Simple parts
• Geometric relationship information

30
Training Images and Segmentation
31
Implicit Shape Model

• Patches local appearance prototypes
• Spatial relationship where the patch can be
  found on the object
• For a given class w
• ISM(w) (Iw ,Pw )
• where Iw is the codebook containing the patches
• and Pw is the probability distribution that
  describes
• where the patch is found on the object
• How do we find interesting patches?

32
Harris Point Operator

• what is it?

33
(No Transcript)
34
Harris Points
35
Segmented Training Mask
Segmented mask ensures only patches containing
valid car regions are selected A corresponding
segmentation patch is also extracted
36
Selected Patches
37
How is spatial information represented?

• Estimate the center of the object using the
  centroid of the segmentation mask
• Displacement between
• Center of patch
• Centroid of segmentation mask

38
Individual Patch and Displacement Information
39
Typical Training Example
40
Typical Training Example
41
Extracted Training Patches
42
Cluster Patches

• Many patches will be visually similar
• Normalized Grayscale Correlation is used to
  cluster patches
• All patches within a certain neighborhood defined
  by the NGC are grouped together
• The representative patch is determined by mean of
  the patches
• The geometric information for each patch in the
  cluster is assigned to the representative patch

43
Patches
44
Wheel Patch Example
45
Clusters
Opportunity for better clustering method
46
Clusters
47
(No Transcript)
48
Object Detection

• Harris point operator to find interesting points
• Extract patches
• Match extracted patches with model patches
• Spatial information predicts center of object
• Create voting space

49
Ideal Voting Space Example
50
Multiple Votes
Multiple geometric interpretations
51
Resolving False Detections
52
Localization Find Corners
53
Localization Model Matching
54
Localization Find Corners
55
Model Matching
56
Spatial Activation(Hough Space)
9000 different locations
57
Hypothesis Candidates
16 candidate locations
58
Hypothesis Candidates
59
References
SEE RESOURCES ON COURSE WEB PAGE Timothy Lebo
and Roger Gaborski, A Shape model with
Coactivation Networks for Recognition and
Segmentation, Eighth International conference on
Signal and image Processing, Honolulu, HI. August
2006. Timothy Lebo, Guiding Object
Recognition A Shape Model with Co-activation
Networks, MS Thesis, RIT, 2005. Daniel Clark,
Object Detection and Tracking using a Parts
Based Approach, MS Thesis, RIT, 2005.
60
References
Bastian Leibe, Ales Leonardis, and Bernt
Schiele, Combined object categorization and
segmentation with an implicit shape model,
ECCV04 Workshop onStatistical Learning in
Computer Vision, May 2004. Shivani Agarwal,
Aatif Awan, and Dan Roth, Learning to detect
objects in images via a sparse, part-based
representation, IEEE Transactions on Pattern
Analysis and Machine Intelligence,
26(11)14751490, 2004.
61
(No Transcript)
62
Voting Space
63
(No Transcript)
64
Model Patches Selected
65
True Object Patches
66
Identified Objects