Reconstructing depth from 2D images

1
Reconstructing depth from 2D images

• Author Alexei Masterov
• Professor Tony Jebara

2
Organization

• Goal
• Motivation
• Roadmap
• Preliminary Results
• References

3
Goal
Learn to reconstruct depth from 2D data. Use SVM
regression to learn z f (neighborhood of z).
4
Motivation

• Humans are able to reconstruct distance even with
  1 eye
• Still images contain pictorial cues
• a. Interposition
• b. Relative height
• c. Relative size
• d. Linear perspective
• e. Texture Gradient
• f. Shadow
• g. Blurring
• I am hoping to learn those cues from the input
  dataset.

5
Roadmap

• Acquire the dataset Arrange 3D models into
  scenes using AliasWavefront Maya.
• Preprocess the dataset produce the pairs of 2D
  rendered images and Z-Depth maps of those scenes
• Program the converter to prepare the input data
  for use with MySvm
• Learn the regression using MySvm using different
  parameters
• Try different preprocessing techniques such as 2D
  fft in log polar coordinates, and edge detection.
• Acquire real world data of natural scenes using
  laser scanner and high resolution photo camera,
  and try the derived algorithm on it.
• Package the learned SVM into a program, so that
  it can be used to reconstruct depth from
  photographs.

6
MySVM (by Stefan Rüping)

• Kernels
• a. dot
• b. polynomial
• c. RBF
• d. two layered neural net tanh(a xyb)
• e. (RBF) anova kernel

7
Preliminary Results (1)
8
Preliminary Results (2)
Edge Image
61 x 61 Squares
9
Preliminary Results (3)
10
References

• Discriminative Random Fields A Discriminative
  Framework for Contextual Interaction in
  Classification. Sanjiv Kumar, Martial Hebert.
• Mixtures of Eigenfeatures for Real-Time
  Structure from Texture T. Jebara, K. Russell and
  A. Pentland.