Image-based walkthroughs from partial and incremental scene reconstructions

1
Image-based walkthroughs from partial and
incremental scene reconstructions

• Kumar Srijan
• Syed Ahsan Ishtiaque
• C. V. Jawahar
• Center for Visual Information Technology,
• IIIT-Hyderabad
• http//cvit.iiit.ac.in

Sudipta N. Sinha Microsoft Research,
Redmond http//research.microsoft.com
2
Introduction
3
Problem

• Efficiently organize and browse these huge image
  collections?
• Keep Incorporating an incoming stream of images
  into an existing framework?

4
Related Work

• World-Wide Media Exchange (WWMX)
• PhotoCompas
• Reality?ythrough
• Aspen Movie Map
• Photowalker
• Sea of Images
• Google Streetview
• Photo Tourism

5
Photo Tourism
Input Images
Incremental SfM
Select a good initial pair to seed reconstruction
Add new images and triangulate new points
Bundle adjust
Full Scene Reconstruction
6
Bottlenecks and Issues

• Global scene reconstruction via incremental
  structure from motion (Sfm)
• Sensitivity to the choice of the initial pair
• Cascading of errors
• O(N4) in the worst case

7
Bottlenecks and Issues

• Timing Breakdown

Full Scene Reconstruction for Trafalgar Square
dataset with 8000 images took gt 50 days
8
Our approach

• In a walkthrough, users primarily observe near
  by overlapping images.
• Advantages
• Robustness to errors in incremental SfM module
• Worst case linear running time
• Scalable
• Incremental

9
Partial Reconstructions
Compute partial Reconstructions
Standard SfM
10
User interface and navigation
11
Global vs. Partial

• Global Allows transition to any image
• Partial Allows transition to a limited number
  of overlapping images
• A -gt B implies B -gt A

A
A
B
B
12
Incremental insertion
Match
Compute Partial Scene Reconstruction
New Image
13
Dataset
Golconda Fort, Hyderabad
Fort Dataset
5989 images
14
Results
15
Results
16
Results

• Courtyard Dataset with 687 images
• Initialized with 200 images
• Added 487 image one by one
• Largest CC of 674 images.

17
Conclusion

• Image navigation system based on partial
  reconstructions can effectively be used to
  navigate through large collections of images.
• Robustness to errors
• Able incorporate more images as they become
  available.

18
Future Work

• Complete automation
• Download images directly from the internet
• Add into the framework

19
Acknowledgements

• Photo tourism Exploring photo collections in
  3D
• Noah Snavely, Cornell University
• Steven M. Seitz, University of Washington
• Richard Szeliski, Microsoft Research

20
Acknowledgements

• Visual Word based Location Recognition in 3D
  models using Distance Augmented Weighting
• Friedrich Fraundorfer, Marc Pollefeys ETH Zürich
• Changchang Wu ,Jan-Michael Frahm ,Marc Pollefeys
  - UNC Chapel Hill

21
Thank You

• Questions