MIT Computer Graphics Group Laboratory for Computer Science

1
MIT Computer Graphics GroupLaboratory for
Computer Science

• Automated Acquisition of CAD Models of Urban
  Areas
• Professor Seth Teller
• graphics.lcs.mit.edu
• Site visit / status report to MURI Evaluators
• July 30, 1998

2
Capture

• City Scanning
• Modeling Botanical Structures

How can we import 3D scene data quickly and
automatically?
3
From 2D Images to 3D Site Models
3) Who establish camera control

• How is it done today?

4) Indicate/verify scene structure
5) Control, structure then optimized generalized
triangulation ensues.
2) Processed by human image analysts
Note human(s) in the loop !
1) Images acquired
Implications for scaling, throughput
4
Fundamental limitations of semi-automated
approaches

• Every image is handled by a human
• Semi-automatic algorithms assume
• Small number (tens) of input images
• Images taken from outside the scene
• All pairs of images are correlated O(n2)
• Implications
• Quadratic processing scaling limitation
• Human in loop throughput limitation
• System does not improve w/ technology!

5
Fully Automated Site Modeling

• How it can be done in the future

3) Geometry, reflectance estimated
combinatorially, and with robust statistics
1) Many geo-located images acquired
2) Images are spatially indexed
6
Implications of Novel Approach

• Computation to process a spatial region depends
  only on acquisition density and the size of the
  region (but not on the total number of images)
• System throughput increases as storage,
  processing technology advances
• Quantity/complexity of reconstructed scenes will
  grow steadily with time
• Fusion of spatially distributed models eased
• Quality may be less than that possible with a
  human image analyst

7
Novel problem domain

• Urban exteriors (built structure)
• Tens of thousands of digital still images
• Acquisition near-ground, inside scene
• Absolute, a priori camera pose estimates
• No human in the loop (break scale barrier)

8
Challenges (Research/Engineering/Systems)

• Robust instrumentation for absolute geolocation
• Sparse/dense correspondence algorithms
• Incremental/multiresolution reconstruction
• Scaleable in of images, output features
• Estimation of surface reflectance (texture)
• System assessment (speed, error, cost)

9
Geo-located digital camera
Cheap digital cameras, GPS, MEMS inertial
chipsets soon available
10
Pose-camera Status
Integration of Power, Digital camera PC,
Mass storage, GPS, Inertial sensor,
Inclinometer, Wheel encoders Kalman
filter Field tests in progress Now deployed on
MIT Campus
11
Image acquisition First dataset
Early prototype of pose camera deployed in and
around Tech Square (4 structures) Collected 81
nodes, 4,000 geo-located images
12
Mosaic generation
Each node is 50-70 images tiling a sphere All
nodes correlated to form spherical mosaics Camera
internal parameters auto-calibrated
Computation is automated (no human in loop) Per
node, about 20 CPU-minutes _at_ 200 MHz
13
Mosaics A Closer Look
Each is about 75 Mega-Pixels with
improved cameras, each will be about 300
Mega-pixels
14
Imagery Control
Each node is controlled, or co-situated, in a
common, global (Earth) coordinate system
Current instrumentation requires human assist-
ance (1 hr total, or about 1 second per
image) Mosaicing is significant engineering
advantage Goal full automation of geo-location
instruments
15
Building detection
Sweep-plane technique identifies vertical
façade orientations, locations, boundaries
16
Texture estimation challenge
17
Reflectance (Texture) estimation
Robust, weighted median - statistics algorithm
estimates texture/BRDF for each building facade
weighted median
Algorithm removes structural occlusion foliage
blur (obliquity) color and lighting variations!
18
Texture estimation results
Input Raw imagery
Output Synthetic texture

• Made possible by many observations
• A sensor and system that effectively see
  through complex foliage!

19
Preliminary results
Model represents about 1 CPU-Day at 200 MHz
Next acquire full MIT campus compare to
refer-ence model captured via traditional
surveying
20
From the East
From the South
21
Contributions
Instrumentation, scaleable end-to-end system
design Address scaling with geo-location, spatial
data structures Novel mosaicing, reconstruction,
texturing algorithms Significant step toward
fully automated reconstruction
Goal for 1999 capture MIT Campus (200
structures) from 1 Tb of ground, 1 Tb of
aerial imagery
22
Further information

• graphics.lcs.mit.edu
• graphics.lcs.mit.edu/city/city.html
• graphics.lcs.mit.edu/publications.html \
• LCS Technical Memo 561 (w/ Coorg)
• AI Technical Memo 1593 (w/ Mellor, L.-Perez)
• Proc. 1997 Image Understanding Workshop
• LCS Technical Memo 568 (w/ Coorg, Master)
• LCS Technical Report 729 (w/ Coorg)

23
Exploration

• Immersive Image-Based Virtual Environments
• High Performance Visualization of Urban Scenes
• Hardware for Accelerated Walk-throughs
• Distributed Navigation of Complex Simulation
  Datasets

How can we make viewing and manipulation more
responsive?
24
Design and Simulation

• Modeling and Rendering Weathering and Surface
  Appearance
• Acoustic Design and Modeling of
  ArchitecturalEnvironments
• Interactive High-Fidelity Rendering
• Planning and Design of the MIT CIIS Complex

How can we incorporate simulation into the
design process?
25
Distance Education

• Infrastructure for
• Distributed development of course materials
• Validated self-study of algorithmic concepts
• Collaboration among students, teaching staff
• Rich evaluation, feedback, annotation mechanisms
• Full network and platform transparency

How can we teach computer engineering concepts
at a distance?