Appearance Manifolds for Modeling TimeVariant Appearance of Materials

1
Appearance Manifolds for ModelingTime-Variant
Appearance of Materials
Jiaping Wang Xin Tong Stephen Lin Baining
Guo Heung-Yeung Shum Minghao Pan Hujun Bao Chao
Wang
Institute of Computing Technology, CAS Microsoft
Research Asia Zhejiang University
Tsinghua University
2
Motivation

• Modeling and editing of time-variant surface
  appearance

3
Related Work

• Visual simulation of weathering distribution
• Miller94, Hsu95, Wong97, Chen05
• Focus on global distribution
• Physically-based simulation
• Dorsey96, Dorsey99, Merillou01, Paquette02,
  Lu05
• Only for some specific materials
• Directly capture images of real samples
• Georghiades05, Gu06

4
Related Work Image Capture
Georghiades05 Observing and Transferring
Material Histories
Gu06 Time-Varying Surface AppearanceAcquisitio
n, Modeling, and Rendering

• Frame-by-frame capture over time
• ? Realistic appearance from real samples
• ? Laborious capture of full time sequences

5
Our Contributions

• An easy-to-use technique for modeling
  time-variant appearance of materials
• Capture at a single time instant
• Generate realistic appearance
• spatially-variant BRDFs
• surface texture patterns evolve over time
• Visual simulation technique
• not necessarily physically accurate

6
An Example
7
Key Observation
Weathering Degree
Material Sample
8
Key Observation
High-dimensional Appearance Space
Material Sample
9
Key Observation
Appearance Manifold
High-dimensional Appearance Space
Material Sample
10
Key Observation
Weathering Degree
Weathering Degree
Appearance Manifold
Material Sample
11
System Overview
12
System Overview
Material Sample
Appearance Manifold
Construct Appearance Manifold
13
Appearance Space Projection
Material Sample
Appearance Space
14
Construct Appearance Manifold
Material Sample
Appearance Space
15
Construct Appearance Manifold
Material Sample
Appearance Space
16
Determine Weathering Degree
Material Sample
Appearance Space
17
Determine Weathering Degree
Ending Set
Starting Set
Material Sample
Appearance Space
18
Determine Weathering Degree
x
Material Sample
Appearance Space

• Weathering degree of x is

19
Determine Weathering Degree
Material Sample
Appearance Space
20
System Overview
Time-variant Appearance
Material Sample
Appearance Manifold
Synthesize Time-variant Appearance
21
Synthesize Time-Variant Appearance
. . .
time
Target Weathering Degree Sequence
Source Degree Map
. . .
time
Result Texture Appearance Sequence
Source Texture Sample
22
Frame-Coherent Texture Synthesis
Texture Similarity
Monotonic Change

• Similarity texture elements similar to source
• Monotonic monotonic change of appearance

23
Naïve Approach I
Texture Similarity
Monotonic Change

• Similarity texture elements similar to source
• Monotonic monotonic change of appearance

24
Naïve Approach I

• Ignore monotonic change of appearance
• Independently synthesize each frame

Result Sequence
25
Naïve Approach II
Texture Similarity
Monotonic Change

• Similarity texture elements similar to source
• Monotonic monotonic change of appearance

26
Naïve Approach II

• Ignores patterns in texture
• Pixel-wise extrapolation of one frame

27
Naïve Approach II

• Ignores patterns in texture
• Pixel-wise extrapolation of one frame

Pixel-wise Extrapolation
28
Frame-Coherent Texture Synthesis
Texture Similarity
Monotonic Change

• Similarity texture elements similar to source
• Monotonic monotonic change of appearance

29
Frame-Coherent Texture Synthesis
TargetSequence
...
x0
x1
x2
xn

Extrapolated Frames
30
Consistency Constraint
Target Degree Map
Source Texture Source Degree Map
Target Frame xt
31
Monotonicity Constraint
Target Frame xt
32
Comparison to Naïve Approaches
Only Monotonicity
Frame-Coherent Synthesis
Only Texture Consistency
33
Synthesis Result on 3D Surface
34
Weathered Bananas
35
Weathering/Deweathering
Material Sample
36
Weathering/Deweathering
37
Weathered Leaf
38
Weathering Transfer
Degree Map on Surface
Input Weathered Object
39
Rendering Results
40
Conclusion

• An easy-to-use tool for modeling time-variant
  appearance of materials
• Acquisition at a single point in time
• Realistic spatially-variant BRDFs
• Plausible temporal variations
• Frame-coherent texture synthesis
• Tool for editing weathered appearance of existing
  objects

41
Acknowledgements

• Kun Zhou , Dong Xu
• Helpful discussions
• Yuan Tian , Rui Jin
• Geometry models and photography
• Paul Debevec
• Linear light source reflectometry
• HDR environment maps

42
Thanks