Extended Urban Model Acquisition From GeoReferenced Images

1
Extended Urban Model Acquisition From
Geo-Referenced Images

• Seth Teller, MIT Computer Graphics Group
• graphics.lcs.mit.edu

2
Project Goals

• Develop a sensor and algorithms to extract
  geodetic textured CAD models from initially
  uncontrolled imagery, without a human in the loop
• Three parts
• Develop and deploy sensor to acquire
  approximately geo-referenced imagery
• Refine to produce controlled imagery
• Extract geometry, textures (BRDFs)

3
Extended Urban Site Modeling
3) Geometry, reflectance esti- mated with robust
statistics using feature ensembles
1) Many geo-located images acquired
2) Images are indexed controlled
4
Research/Engineering Footprints
1 2

Ascender Façade MIT/City
Number of images Tens Tens Thousands Imagery
type Aerial Ground Ground aerial 6-DOF
camera pose From human From human Instruments
optim. Structure extraction Roof-matching By
human Automatic detection optimization optimi
zation optimization Number of structures Scores
One to Tens Arbitrary Output coord- Specified
by Specified by Geodetic (Earth) inate
system operator operator coordinates Texture
Procedural Manual Automatic w/
matching segmentation robust statistics Scaling
capability Unclear Unclear Spatial
index Parallel model acquis- None None Use of
geodetic ition and merging coordinates, index
1 UMASS 2 Berkeley
5
Geo-referenced digital camera
Cheaper digital cameras, GPS, MEMS inertial
chipsets soon available also MAVs
6
Mosaic generation
Each node is 50-70 images tiling a
sphere about a mechanically fixed optical
center Each node correlated to form spherical
mosaic Camera internal parameters auto-calibrated
Computation is automated (no human in loop) Per
node, about 20 CPU-minutes _at_ 200 MHz
7
Mosaics A Closer Look
Each is about 75 Mega-Pixels with
improved cameras, each will be about 300
Mega-pixels
8
Image acquisition First dataset
Early prototype of pose camera deployed in and
around Tech Square (4 structures) 81 nodes
4,000 geo-located images 20 Gb
9
Imagery Control Exterior Registration
Each node is controlled, or co-situated, in a
common, global (Earth) coordinate system
Instrumentation requires human assistance(lt1
hour total, or about 1 second per
image) Mosaicing significant engineering
advantage Goal full automation of
geo-referencing process
10
Control, detection without correspondence
Histogramming algorithm identifies orientations
of significant vertical façades in vicinity of
camera(s)
11
Control, façade detection II
Sweep-plane algorithm identifies location and
spatial extent of each vertical façade
12
Texture estimation challenge
13
Reflectance (Texture) estimation
Robust, weighted median - statistics algorithm
estimates texture/BRDF for each building façade
weighted xyY median
sharpening
Algorithm removes structural occlusion foliage
blur (obliquity) color and lighting variations!
14
Texture estimation results
Input Raw imagery
Output Synthetic texture

• Made possible by many observations
• A sensor and algorithms that effectively see
  through complex foliage!

15
Preliminary results (with overlaid aerial image)
Model represents about 1 CPU-Day at 200 MHz
Next acquire full MIT campus compare to
refer-ence model captured via traditional
surveying
16
Preliminary results (movie)
17
System overview
18
Limitations
Navigation subsystem under development In
progress integration of GPS, inertial,
etc. Limited spatial extent (200 meters squared)
In progress acquisition through, above
MIT Exterior registration is semi-automated In
progress feature, ensemble correspondence Vertica
l facades only rooftops procedural Under way
polyhedral, quadric shapes Diffuse lighting,
diffuse surfaces Planned directional lighting
(a la Façade) Foliage removed via median stats
not modeled In progress foliage
segmentation, modeling
19
Contributions
Instrumentation, scaleable end-to-end system
design Address scaling with geo-location, spatial
data structures Novel mosaicing, reconstruction,
texturing algorithms Significant step toward
fully automated reconstruction
Next goal capture MIT Campus (200 structures)
from 1 Tb of ground, 1 Tb of aerial imagery
20
Further information

• graphics.lcs.mit.edu
• graphics.lcs.mit.edu/seth
• graphics.lcs.mit.edu/city/city.html
• graphics.lcs.mit.edu/publications.html

21
Toward Automated Exterior Registration
22
From the East
From the South
23
(No Transcript)