By Vanessa Newey

1
Image-Based Modeling, Rendering, and Lighting

• By Vanessa Newey

2
Introduction

• Traditional 3D Modeling and Rendering
• Image-Based Modeling, Rendering, and Lighting
• Applications
• Available Tools
• Summary

3
Traditional 3D Modeling and Rendering
4
Traditional 3D Modeling and Rendering Developments

• Hidden Surface Removal Algorithms
• Shading Algorithms
• Shadow Calculation
• Displaying Curved Surfaces
• Ray tracing and Radiosity
• Powerful Modeling Tools

5
Limitations of Traditional 3D Modeling and
Rendering

• Possible to Visualise Complex 3D Models in Real
  Time
• BUT
• Creating these 3D models requires Expertise and
  Time.
• Rendering is Slow

6
Motivation for Image-based Modeling and Rendering

• Demand for more complex, realistic models
• Traditional methods slow and costly
• Many virtual objects based on real objects
• Advantageous to model directly from real objects

7
Image-based Modeling and Rendering

• Create 3D models from one or more images
• Re-render creating new views
• Easier to model scenes photorealistically

8
Image-based Modeling, Rendering
9
Computer Graphics- Computer Vision
10
Image-based Modeling

• Images (photographs, renderings) are used to
  determine
• Scene Appearance
• Scene Geometry
• Lighting
• Reflectance Characterisics
• Kinematic Properties

11
Image-based Rendering

• Appearance in available views is used to
  determine appearance in novel views
• Not necessary to perform full illumination
  computations
• Rendering is faster

12
Image-based Lighting

• Able to Create a Light-based Model from the
  images
• For each part of the local scene, we know its
  irradiance
• Allows us to combine real and synthetic graphics
  with consistent illumination, using images as
  light sources.

13
Example
14
Example Video
15
Stereo Image Capture Rig
16
Applications

• VR worlds and Games
• Construction of virtual scenes
• Creating terrain from Air-Photos
• Images and Video on the WWW
• Video compression, summarisation, skimming,
  indexing and selective retrieval

17
Tools

• REALVIZ ImageModeler
• http//www.realviz.com/
• ERDAS IMAGINE OrthoMax
• Automated DEM extraction
• ERDAS Stereo Analyst
• Provides tools for collection, interpretation and
  visualisation of 3D geographic information from
  stereo imagery
• 3Q Qlonerator Series
• http//www.3q.com/offerings_tech.htm

18
REALVIZ ImageModeler 1
Start the project

19
REALVIZ ImageModeler 2
Calibrate cameras

20
REALVIZ ImageModeler 3
Create a model

21
REALVIZ ImageModeler 4
Texture a model

22
REALVIZ ImageModeler 5
Export a model

23
ERDAS Stereo Analyst

24
Challenges

• Automatic scene segmentation
• Complex scenes eg. forests
• Non-static scenes
• Inaccuracies in source photos
• Real imagery/computer vision problems
• Camera pose hard to get
• Depth even harder to get
• Data Size may be huge
• Changes Difficult
• lights, geometry, etc.

25
Summary

• Overall exciting developments and much more
  potential for Image-based modeling, rendering and
  lighting.

26
Sources

• http//i31www.ira.uka.de/oel/ibmr-focus/
• http//www.debevec.org/IBMR99/
• http//www.cs.berkeley.edu/debevec/IBMR
• http//research.microsoft.com/szeliski/IBMR98/web
  /sld001.htm
• http//www.esat.kuleuven.ac.be/pollefey/
• 3D modeling from images tutorial
• http//www.esat.kuleuven.ac.be/pollefey/SMILE2/tu
  torial.html
• ERDAS Products
• http//www2.erdas.com/supportsite/apps_gallery/flu
  me_bed/flume_bed.htm
• http//www2.erdas.com/supportsite/apps_gallery/vid
  eography/videography_raw.htm
• http//www.erdas.com/software/ProductModules.asp?S
  ubGroupID513GroupID1
• REALVIZ ImageModeller
• http//www.realviz.com
• http//www.realviz.com/products/stepbystep.php?pro
  ductim
• http//www.debevec.org/Campanile/