Image composition using 3D Textons

1
Image composition using 3-D Textons

• Presented by Howard Zhou
• howardz_at_cc.gatech.edu
• 2003 . 4 . 28

2
Image Composition
Image courtesy of Star Wars Episode II and Mad
Magazine
3
Goal of Image Composition

• Replace or add objects in the scene
• Object 3D model or 2D sprite
• Goal Look real / Undetectable

4
Related Work

• Eigen-Texture Method Appearance compression and
  Synthesis Based on a 3D Model, Ko Nishino, et al.

5
Observation

• We dont need all that
• Color, texture more important
• People use simple local lighting model, so global
  lighting (shadow) can be a little off

6
Who does not fit?
Image courtesy of Mad magazine
7
Whats wrong with this picture?
Impossible lighting is hard to spot in
images (Cavanaugh) Implies that people use simple
local shading and lighting model. - James M.
Rehg
Image courtesy of Patrick Cavanaugh
8
Texture (color) 3D texton

• What is Texton?
• We will characterize a texture by its responses
  to a set of orientation and spatial frequency
  selective linear filters (a filter bank)

9
Texture (color) Texton

• Description from http//www.cs.ucla.edu/cguo/what
  _are_textons.htm
• Textons refer to fundamental micro-structures in
  generic natural images and the basic elements in
  early (pre-attentive) visual perception.
• Firstly, decomposing an image into its
  constituent components reduces information
  redundancy and thus leads to better image coding
  algorithms.
• Secondly, the decomposed image representation
  often has much reduced dimensions and less
  dependence between variables (coefficients),
  therefore it facilitates image modeling which are
  necessary for image segmentation and recognition.
  
• Thirdly, in biologic vision the micro-structures
  in natural images provide an ecologic cue for
  understanding the functions of neurons in the
  early stage of biologic vision system.

10
3D Texton Frame work

• Related work
• Representing and Recognizing the Visual
  Appearance of Materials using 3D Textons, Thomas
  Leung and Jitendra Malik
• Recognition Methods for 3D Textured Surfaces,
  Oana G. Cula and Kristin J. Dana

11
3D Texton Frame work
12
Texture Samples
texture under different lighting condition and
viewing angle
13
Filter banks and filtered texture
The filter bank used in my analysis. Total of 48
filters 36 oriented filters with 6 orientations,
3 scales and 2 phases 8 center surrounded
derivative filters (Laplacian of Gaussian) and 4
low pass Gaussian filters
14
Filtered texture
Filter responses
Original image
15
K-mean clustering
At each pixel, a pyramid of image filters at
various scales and orientations are convolved
with the image. The filter responses are
clustered by a K-mean clustering method.
(vocabulary of 3D textons)
(a) Original image. (b) the K-means centers
reconstructed as local filters. These centers
correspond to the dominant features in the image
16
Texture Recognition with histograms
17
Future work

• Get shadow (lighting) right
• Deriving intrinsic images from image sequences,
  Yair Weiss
• Recovering Intrinsic Images from a Single Image,
  Marshall F Tappen et al.
• Illumination from shadow, imari Sato, et al.

18
Questions ?
19
Deriving Intrinsic Images from image Sequence

• Face, 1 light source, T 10

Images courtesy of Yale Face Database
20
Synthesis

• Work in progress, no synthesized result to show.
• Here are some from Thomas website

Crumpled Paper
Rough Plastic