Fast Imagebased Separation of Diffuse and Specular Reflections

1
(No Transcript)
2
Fast Image-based Separation of Diffuse and
Specular Reflections

• Bruce Lamond, Pieter Peers Paul Debevec
• University of Southern California
• Institute for Creative Technologies

3
Introduction

• Problem Complex to model reflectance
• Specular highlights hamper geometry recovery
• Reflectance component separation useful for
• Image-based reflectance modeling
• Normal/geometry acquisition

4
Overview
5
Outline

• Related work
• Method
• Data capture
• Data processing
• Results extensions
• Conclusions further work

6
Related work 1

• Specular-diffuse separation
• Sato et al. SIGGRAPH 1997
• Surface mesh perpendicular diffuse color
• Only 1 rough specular term recovered

7
Related work 1

• Specular-diffuse separation
• Sato et al. SIGGRAPH 1997
• Surface mesh perpendicular diffuse color
• Only 1 rough specular term recovered
• Nayar et al. IJCV 1997
• Polarization (specular) color (diffuse)
• Not per pixel (spatial coherency)

8
Related work 1

• Specular-diffuse separation
• Sato et al. SIGGRAPH 1997
• Surface mesh perpendicular diffuse color
• Only 1 rough specular term recovered
• Nayar et al. IJCV 1997
• Polarization (specular) color (diffuse)
• Not per pixel (spatial coherency)
• Debevec et al. SIGGRAPH 2000
• Colorspace analysis of per pixel reflectance
  function
• High-res but data intensive

9
Related work 2

• Specular-diffuse separation
• Lin et al. ECCV 2002
• Separation/depth from multi-stereo
• Whole sequence required

10
Related work 2

• Specular-diffuse separation
• Lin et al. ECCV 2002
• Separation/depth from multi-stereo
• Whole sequence required
• Mallick et al. ECCV 2006
• Transform to SUV space PDE erodes specularity
• Correct PDE/spatial coherency

11
Related work 2

• Specular-diffuse separation
• Lin et al. ECCV 2002
• Separation/depth from multi-stereo
• Whole sequence required
• Mallick et al. ECCV 2006
• Transform to SUV space PDE erodes specularity
• Correct PDE/spatial coherency
• Ma et al. EGSR 2007
• High-res normal map from specularity few
  parameters
• Needs each light polarized

12
Method Reflectance function
Images Debevec et al. SIGGRAPH 2000
13
Method Reflectance function
discrete light source
discrete light source
Images Debevec et al. SIGGRAPH 2000
14
Method Reflectance function
discrete light source
discrete light source
Images Debevec et al. SIGGRAPH 2000
15
Method Reflectance function
function for 1 pixel
discrete light source
discrete light source
Images Debevec et al. SIGGRAPH 2000
16
Method Reflectance function
function for 1 pixel
specular
discrete light source
discrete light source
diffuse
Images Debevec et al. SIGGRAPH 2000
17
Method capture hardware

• Reflective Light Stage (Peers et al. USC ICT
  Tech.Rep. 2006)

rough specular reflection
hemispherical dome
object
occluder
18
Method Sharp specular reflectance function
19
Method Sharp specular reflectance function
20
Method Sharp specular reflectance function
21
Method Sharp specular reflectance function
22
Method High frequency lighting
23
Method RLS capture illustration
24
Method RLS capture illustration
25
Method RLS capture illustration
26
Method RLS capture illustration
27
Method RLS capture illustration
28
Method RLS capture illustration
29
Method RLS capture illustration
30
Method RLS capture illustration
31
Method RLS capture illustration
32
Method RLS capture illustration
33
Method RLS capture illustration
34
Method RLS capture illustration
35
Method RLS capture illustration
36
Method compare to related technique

• Nayar et al. SIGGRAPH 2006

37
Method compare to related technique

• Nayar et al. SIGGRAPH 2006

direct
indirect
38
Method compare to related technique

• Nayar et al. SIGGRAPH 2006

• Our technique

direct
indirect
39
Method compare to related technique

• Nayar et al. SIGGRAPH 2006

• Our technique

direct
indirect
diffuse
Specular diffuse
40
Method capturing a real object
41
Method capturing a real object
1
42
Method capturing a real object
2
43
Method capturing a real object
3
44
Method capturing a real object
4
45
Method capturing a real object
Whole illumination (sum of stripes / 2)
46
Method data processing result
specular
min
max
47
Method data processing result
specular
min
max
specular
diffuse
max min
48
Capturing a problematic real object
1
49
Capturing a problematic real object
2
50
Capturing a problematic real object
3
51
Capturing a problematic real object
4
52
Results
diffuse
specular (3 stops)
53
Results highly diffuse case
54
Results highly diffuse case
55
Extension Image-based illumination
original image
stripe 1
56
Extension Image-based illumination
original image
stripe 2
57
Extension Image-based illumination
original image
stripe 3
58
Extension Image-based illumination
original image
stripe 4
59
Extension results
diffuse
specular
60
Extension IBL and complex reflectance
61
Extension IBL and complex reflectance

• Image based illumination

original image
stripe 1
62
Extension IBL and complex reflectance

• Image based illumination

original image
stripe 2
63
Extension IBL and complex reflectance

• Image based illumination

original image
stripe 3
64
Extension IBL and complex reflectance

• Image based illumination

original image
stripe 4
65
Extension results
diffuse
specular (3 stops)
66
Extension 2 surface normals
67
Extension 2 surface normals
68
Extension 2 surface normals
69
Summary

• Conclusion
• Fast image-based diffuse specular separation
• Technically easy to acquire
• Works for sharp specular diffuse objects
• Extends to image-based illumination
• Can be used to extract surface normals
• Further Work
• Investigate image-based modeling of reflectance
  properties
• More surface normals ground truth comparisons

70
Acknowledgements
Tom Pereira, Bill Swartout, Scott Fisher, Randy
Hill, Randolph Hall, and Max Nikias for their
support and assistance with this work. This work
was sponsored by the University of Southern
California Office of the Provost and the U.S.
Army Research, Development, and Engineering
Command (RDECOM).