Minimal Surfaces for Stereo

1
Minimal Surfaces for Stereo

• Chris Buehler, Steven J. Gortler,
• Michael F. Cohen, Leonard McMillan
• MIT, Harvard
• Microsoft Research, MIT

2
Motivation

• Optimization based stereo over greed based
• No early commitment
• Enforce interactions each pixel sees unique item
• Penalize interactions non-smoothness

3
Stereo by Optimization

• Early algorithms dynamic programming
• (Baker 81, Belumeur Mumford 92)
• Dont generalize beyond 2 camera, single scanline

4
Stereo by Optimization

• Recent Algorithms iterative a-expansion
• ( Kolmogorov Zabih 01)
• very general
• NP-Complete
• Local opt found quickly in practice
• Recent algorithms MIN-CUT
• (Roy Cox 96, Ishikawa Geiger 98)
• Polynomial time global optimum
• New interpretation to such methods

5
Contributions

• Stereo as a discrete minimal surface problem
• Algorithms Polynomial time globally optimal
  surface
• Using MIN-CUT (Sullivan 90)
• Build from shortest path
• Applications to stereo vision
• Rederive previous MIN-CUT stereo approaches
• New 3-camera stereo formulation (Ayache 88)

6
Planar Graph Shortest Path

• Given an embedded planar graph
• faces, edges, vertices

7
Planar Graph Shortest Path

• A non negative cost on each edge

57
8
Planar Graph Shortest Path

• Two boundary points on the exterior of the
  complex

9
Planar Graph Shortest Path

• Find minimal curve (collection of edges) with
  given boundary

10
Planar Graph For stereo
11
Algorithms

• Classic Dijkstras
• Works even for non-planar graphs
• Wacky use duality
• But this will generalize to higher dimension

12
Duality
13
Duality

• face vertex
• edge cross edge
• - same cost

57
14
Duality

• Split exterior

15
Duality

• Add source and sink

16
Cuts

• Cuts of dual graph partitions of dual verts
• Cost sum of dual edges spanning the partition
• MIN-CUT can be found in polynomial time

17
Cuts

• Claim Primalization of MIN-CUT will be shortest
  path

18
Why this works

• Cuts of dual graph partitions of dual verts

19
Why this works

• Partition of dual verts partition of primal
  faces

20
Why this works

• Partition of primal faces primal path

21
Why this works

• Cuts in dual correspond to paths in primal
• MIN-CUT in dual corresponds to shortest path in
  primal

22
Same idea works for surfaces!
23
Increasing the dimension
Planar graph verts, edges, faces cost on
edges boundary 2 points on exterior sol min
path
Spacial compex verts, edges, faces, cells
cost on faces boundary loop on exterior
sol min surface
24
Increasing the dimension
Planar graph verts, edges, faces boundary
2 points on exterior sol min path
Spacial compex verts, edges, faces, cells
cost on faces boundary loop on exterior
sol min surface
25
Increasing the dimension
Planar graph verts, edges, faces boundary
2 points on exterior sol min path
Spacial compex verts, edges, faces, cells
cost on faces boundary loop on exterior
sol min surface
26
Dual construction for min surf

• face vertex
• edge cross edge

• cell vertex
• face cross edge

MIN-CUT primalizes to min surf
27
Checkpoint

• Solve for minimal paths and surfaces
• MIN-CUT on dual graph
• Apply these algorithms to stereo vision

28
Flatland Stereo
Geometric interpretation of Cox et al. 96
pixel
Camera Left
Camera Right
29
Flatland Stereo
Geometric interpretation of Cox et al. 96
pixel
Camera Left
Camera Right
30
Flatland Stereo
Cost unmatched/discontinuity, ß
Camera Left
Camera Right
31
Flatland Stereo
Cost correspondence quality
Camera Left
Camera Right
32
Flatland Stereo
33
Flatland Stereo
Uniqueness monotonicity solution is
directed path
34
Flatland Stereo
Note unmatched pixels also function as
discontinuities
Occlusion, discontinuity
Match
35
Flatland to Fatland
Camera Left
Camera Right
36
Flatland to Fatland
Camera Left
Camera Right
37
2 cameras, 3d
38
2 cameras, 3d
39
One Cuboid Among Many
Solve for minimal surface
40
Geometric interpretation IG98
41
Three Camera
Rectification (Ayache 88)
42
Three Camera
43
Three Camera
44
Three Camera
45
Three Camera
46
One cuboid
47
Dual graph of one cuboid
48
One Cuboid Among Many
Solve for minimal surface
49
More divisions of middle cell
50
More expressive decomposition
51
Complexity

• Vertices and edges 20 n d
• n pixels per image
• d max disparity
• Time complexity O((nd)2 log(nd))
• About 1 min

52
Results
LL image
RC
KZ01
MS
53
LL image
RC
KZ01
MS
54
Future

• Application of MS to n cameras
• Monotonicity/oriented manifold enforces more than
  uniqueness
• see Kolmogorov Zabih (today 1100am)
• Other applications of MS