Informal Time Lapse Photography

1
Informal Time Lapse Photography

• Seth Cooper T. Scott Saponas
• CSE 576 Final Project Presentation

2
Problem Statement

• Handheld photographs taken on separate occasions
  (throughout a day, throughout a year)
• Pictures could easily be up to 50 misaligned
• Not enough consistency between frames

3
Problem Statement Cont.

• One subject moves from frame to frame
• Want background aligned
• Fading between images insufficient
• Want to recreate continuous movement

4
Problem Statement Cont.

• Subject Undergoes Continuous Change
• Fading between images insufficient
• Want to recreate continuous movement

5
Problem Statement Cont.

• Historic Images
• Taken with different cameras
• Taken from different viewpoint

6
Informal Time Lapse Photography

• Support someone taking images on multiple
  occasions of similar scene
• Align images
• Create pleasing in-between frames
• Support taking multiple images of moving/changing
  subject
• Create motion from discrete images using morphing

7
Approach - Alignment

• Use SIFT binary to extract features from full
  resolution input images
• Pairwise alignment of consecutive images using
  extracted features (ratio threshold)
• Use RANSAC with two matching features
• Apply 3D rotation motion model to pairs, then
  inliers

8
Approach - Morphing

• Linear Blend
• Triangle Based
• Edge Based

9
Approach - Morphing

• Linear Blend
• Fine when subject is aligned
• Not as good if not aligned, or things are moving

10
Approach - Morphing

• Triangle Based Try 1
• Auto detect features
• Auto triangulate
• Convert to pixel coords to barycentric coords
• Blend feature points

11
Approach - Morphing

• Triangle Based Try 1
• Features good for matching not necessarily good
  for morphing
• Too many interior features, not enough on edges
• Flipping triangles artifact

12
Approach - Morphing

• Triangle Based Try 2
• Allow user to select important features (and
  optionally important edges)
• Augment selection with vision techniques (snap to
  corner, feature match)
• Rest is the same

13
Approach - Morphing

• Triangle Based Try 2
• Auto triangulation not great for morphing
• Better, but flipping triangles still a problem

14
Approach - Morphing

• Edge Based
• Allow user to select important edges
• Blend edges
• Move pixels based on weights from nearby edges
  Beier and Neely

15
Approach - Morphing

• Edge Based
• Less artifacts
• Less lines to cross but bad when they do

16
Approach - Morphing

• Comparison

17
Artifacts
Quad at Sunset
Dancing Book
Clock at Sunrise
Melting Ice
Leaning Tower of Pisa Through 1900s
18
Conclusions

• Our artifacts better?
• Whats an interesting time lapse is subjective
• Good alignment seems like a clear win
• Morphing over blend better in many cases
• Alignment
• Robust same camera, similar viewpoint
• Room for improvement Images through history
• Morphing
• Robust Semi-auto, points not passing through
  lines
• Room for improvement More automatic morphing