Procedural Modeling of Architectures towards 3D Reconstruction

1
Procedural Modeling of Architectures towards 3D
Reconstruction

• Nikos Paragios
• Ecole Centrale Paris / INRIA Saclay Ile-de-France

Joint Work P. Koutsourakis, L. Simon, O. Teboul
N. Komodakis
2
Outline

• The language of Architecture
• Automatic generation of buildings
• Toward single image-based reconstruction
• Discrete MRFs, LP and Duality

04/11/0810/04/08
3
Part IShape Grammars
4
3D reconstruction

• Emerging technology in information society
• Post-production, games, navigation
• Current Methods image based
• Not scalable, computational expensive, low level,
  

04/11/0810/04/08
5
The Geometry of Architecture

• Very complex geometry
• but deeply structured due to
• Laws
• Practical constraints
• Economical constraints
• Aesthetic motivation
• Each architectural style is constrained
  differently. But buildings of the same style obey
  the same rules.

04/11/0810/04/08
6
Simple example of structure
04/11/0810/04/08
7
Architecture as a large scale Lego

• Architecture is made of repetitions of basic
  elements, place holders or real elements.
• Facades, floors, windows, balconies, doors
• A specific style can be describe by
• The specification of the repetitions (geometry)?
• The nature of the elements (semantics)?

04/11/0810/04/08
8
Shape Grammars

• Procedural modeling is a set of techniques for
  creating 3D models from a set of rules.
• Traditional 3D modeling can be very time
  consuming for large models.
• Large scale architectural models are especially
  suited for this kind of techniques.
• There is a strong analogy with classical string
  grammars.

04/11/0810/04/08
9
Basic Shapes and Scopes

• The Basic Shapes are for Shape Grammars what
  Symbols are for String Grammars
• A Basic Shape is composed of
• A Semantic
• A fixed geometry
• Appearance attributes
• A Scope provides the orientation, scaling,
  position of a basic shape

04/11/0810/04/08
10
Shape

• The shape is a hierarchical representation (a
  tree) of the building.
• Each node holds a basic shape with an associated
  scope.

04/11/0810/04/08
11
Rules Interacting with shapes

• A rule can be described as
• precondition LHS ? RHS
• Given a precondition on the context, a LHS shape
  is replaced by the RHS shape. From the
  hierarchical point of view some children are
  added to the node LHS, increasing the depth of
  the tree.
• Rules are built from operators

04/11/0810/04/08
12
The operators of the grammar

• Except the roof operators (which rely on
  computational geometry) the operators are fairly
  simple.
• Transformation rotation,translation, scaling
• Split
• Repeat
• Mirror
• Component Split
• Roof operators hipped, mansard

04/11/0810/04/08
13
Operators continued

• Splits subdivide a scope along an axis
• Component split decomposes a mesh into faces
• Roof operators are based on Weighted Straight
  Skeleton algorithm

04/11/0810/04/08
14
Step by step generation
04/11/0810/04/08
15
Generating from details to large scale
04/11/0810/04/08
16
04/11/0810/04/08
17
04/11/0810/04/08
18
Demos

• Video\ComplexBuilding.ogg
• video\roofsAndTexture.ogg

04/11/0810/04/08
19
Grammar-based reconstruction

• A building is not described by a geometrical
  attributes anymore
• Building sequence of rules
• Goal optimize the sequence of rules that best
  explains a given facade image

04/11/0810/04/08
20
Model Quality Evaluation

• Assuming that the camera has been weakly
  calibrated, the elements of the resulting model
  can be reprojected on the image.
• The score of the model can be defined in terms of
  a distance between the observed regions and the
  known semantics.

04/11/0810/04/08
21
Part IIMRF optimization viathe primal-dual
schema
22
The MRF optimization problem
set L discrete set of labels
23
MRF hardness
MRF hardness
MRF pairwise potential

• Move right in the horizontal axis,

• But we want to be able to do that efficiently,
  i.e. fast

24
The primal-dual schema

• Say we seek an optimal solution x to the
  following integer program (this is our primal
  problem)

(NP-hard problem)?

• To find an approximate solution, we first relax
  the integrality constraints to get a primal a
  dual linear program

primal LP
25
The primal-dual schema

• Goal find integral-primal solution x, feasible
  dual solution y such that their primal-dual costs
  are close enough, e.g.,

primal cost of solution x
dual cost of solution y
Then x is an f-approximation to optimal solution
x
26
The primal-dual schema

• The primal-dual schema works iteratively

unknown optimum
27
The primal-dual schema for MRFs
28
The primal-dual schema for MRFs

• During the PD schema for MRFs, it turns out that

each update of primal and dual variables
solving max-flow in appropriately constructed
graph

• Max-flow graph defined from current primal-dual
  pair (xk,yk)
• (xk,yk) defines connectivity of max-flow graph
• (xk,yk) defines capacities of max-flow graph

• Max-flow graph is thus continuously updated

29
Handles wide class of MRFs

• New theorems- New insights into existing
  techniques- New view on MRFs

primal-dual framework
Approximatelyoptimal solutions
Significant speed-upfor dynamic MRFs
Theoretical guarantees AND tight
certificatesper instance
Significant speed-upfor static MRFs
30
04/11/0810/04/08
31
Future Work

• Efficient optimization framework, through
  neuro-dynamic programming like methods
• Image-based grammar learning
• Augmentation of the expressiveness of the grammar
• Large scale representation of forests of buildings

04/11/0810/04/08
32
Questions ?
04/11/0810/04/08