CS 44957495 Computer Vision

1
CS 4495/7495Computer Vision

• Image Stitching
• Jim Rehg
• Slides from Frank Dellaert, based almost entirely
  on a presentation by Matthew Brown and David Lowe
  at ICCV 2003

2
Introduction

• Are you getting the whole picture?
• Compact Camera FOV 50 x 35

3
Introduction

• Are you getting the whole picture?
• Compact Camera FOV 50 x 35
• Human FOV 200 x 135

4
Introduction

• Are you getting the whole picture?
• Compact Camera FOV 50 x 35
• Human FOV 200 x 135
• Panoramic Mosaic 360 x 180

5
Human FOV
www.inition.co.uk/ inition/guide_hmds_vrar.htm
6
Why Recognising Panoramas?

• 1D Rotations (q)
• Ordering ? matching images

7
Why Recognising Panoramas?

• 1D Rotations (q)
• Ordering ? matching images

8
Why Recognising Panoramas?
9
Why Recognising Panoramas?
10
Why Recognising Panoramas?
11
Overview

• Feature Matching
• SIFT Features
• Nearest Neighbour Matching
• Image Matching
• Bundle Adjustment
• Multi-band Blending
• Results
• Conclusions

12
Invariant Features

• Schmid Mohr 1997, Lowe 1999, Baumberg 2000,
  Tuytelaars Van Gool 2000, Mikolajczyk Schmid
  2001, Brown Lowe 2002, Matas et. al. 2002,
  Schaffalitzky Zisserman 2002

13
SIFT Features

• Invariant Features
• Establish invariant frame
• Maxima/minima of scale-space DOG ? x, y, s
• Maximum of distribution of local gradients ? q
• Form descriptor vector
• Histogram of smoothed local gradients
• 128 dimensions
• SIFT features are
• Geometrically invariant to similarity transforms,
• some robustness to affine change
• Photometrically invariant to affine changes in
  intensity

14
Overview

• Feature Matching
• SIFT Features
• Nearest Neighbour Matching
• Image Matching
• Bundle Adjustment
• Multi-band Blending
• Results
• Conclusions

15
Nearest Neighbour Matching

• Find k-NN for each feature
• k ? number of overlapping images (we use k 4)
• Use k-d tree
• k-d tree recursively bi-partitions data at mean
  in the dimension of maximum variance
• Approximate nearest neighbours found in O(nlogn)

16
Overview

• Feature Matching
• Image Matching
• RANSAC for Homography
• Bundle Adjustment
• Multi-band Blending
• Results
• Conclusions

17
RANSAC for Homography
18
RANSAC for Homography
19
RANSAC for Homography
20
Overview

• Feature Matching
• Image Matching
• RANSAC for Homography
• Finding the panoramas
• Bundle Adjustment
• Multi-band Blending
• Results
• Conclusions

21
Finding the panoramas
22
Finding the panoramas
23
Finding the panoramas
24
Finding the panoramas
25
Overview

• Feature Matching
• Image Matching
• Bundle Adjustment
• Error function
• Multi-band Blending
• Results
• Conclusions

26
Error function

• Sum of squared projection errors
• n images
• I(i) set of image matches to image i
• F(i, j) set of feature matches between images
  i,j
• rijk residual of kth feature match between
  images i,j
• Robust error function

27
Homography for Rotation

• Parameterise each camera by rotation and focal
  length
• This gives pairwise homographies

28
Bundle Adjustment

• New images initialised with rotation, focal
  length of best matching image

29
Bundle Adjustment

• New images initialised with rotation, focal
  length of best matching image

30
Overview

• Feature Matching
• Image Matching
• Bundle Adjustment
• Warping
• Results
• Conclusions

31
Warping

• Take Notes
• Key
• iterate in target image
• calculate corresponding source pixel
• resample
• nearest neighbor (bad)
• bilinear (better)
• bicubic (best)

32
Overview

• Feature Matching
• Image Matching
• Bundle Adjustment
• Multi-band Blending
• Results
• Conclusions

33
Multi-band Blending

• Burt Adelson 1983
• Blend frequency bands over range ? l

34
2-band Blending
Low frequency (l gt 2 pixels)
High frequency (l lt 2 pixels)
35
Linear Blending
36
2-band Blending
37
Overview

• Feature Matching
• Image Matching
• Bundle Adjustment
• Multi-band Blending
• Results
• Conclusions

38
Results
39
Results
http//www.cs.ubc.ca/mbrown/panorama/panorama.ht
ml Autopano based on UBCs code live demo