Interactive Deformation of Light Fields

1
Interactive Deformation of Light Fields

• Billy Chen
• Eyal Ofek
• Heung-Yeung Shum
• Marc Levoy

2
Image-based rendering
Levoy et al., 1996
Shum et al., 1999
Zitnick et al., 2004
Uyttendaele et al., 2004
3
Previous work

• Seitz, S., Kutulakos, K. N. Plenoptic image
  editing, In Proc. ICCV 1998.
• Zhang, Z., Wang, L., Guo, B., Shum, H.-Y,
  Feature-based light field morphing, In Proc.
  SIGGRAPH 2002.
• Weyrich, T., Pfister, H., Gross, M., Rendering
  deformable surface reflectance fields, In IEEE
  Transactions on Computer Graphics and
  Visualization.

4
Light field deformation
5
Outline

• Coaxial light fields
• Framework for deformation
• Splitting
• Deforming
• Rendering
• Results

6
Outline

• Coaxial light fields
• Framework for deformation
• Splitting
• Deforming
• Rendering
• Results

7
Coaxial light fields


8
(No Transcript)
9
(No Transcript)
10
Outline

• Coaxial light fields
• Framework for deformation
• Splitting
• Deforming
• Rendering
• Results

11
Splitting a light field

• A sub-light field consists of
• a bundle of rays
• a deformation box

s
u
12
Sub-light field example
13
Projector-based light field segmentation
14

15
(No Transcript)
16
Outline

• Coaxial light fields
• Framework for deformation
• Splitting
• Deforming
• Rendering
• Results

17
Deformation
18
Ray warping

• preserves mapping specified by the animator
• preserves C0 continuity of points
• maps lines to lines

19
Inverse view ray warping
20
Inverse view ray warping
21
Hardware accelerated warping
front id map
back id map
deformation

• 256 x 256 resolution texture map
• anisoptric filter for mip-mapped texture
• 64 x 64 triangles
• real-time with a NVIDIA GeForce 3

22
Outline

• Coaxial light fields
• Framework for deformation
• Splitting
• Deforming
• Rendering
• Results

23
Rendering and visibility

• For each camera pixel
• Segment ray
• Warp ray
• Render ray
• Mask ray

24
Results
25
Conclusion

• A pipeline for deforming light fields
• Light field segmentation
• Ray warping
• Rendering multiple light fields
• Coaxial light fields
• Enables interactive deformation of
  photo-realistic objects
• Applications photo-realistic rapid prototyping,
  games

26
Thank you

• Reviewers
• NSF (IIS-0219856-001)
• DARPA (NBCH-1030009)
• Elizabeth Chen, Vaibhav Vaish, Gaurav Garg

27
Failure cases
28
Video of acquisition process
29
Deformationand compositing