Boundary Extraction in Natural Images Using Ultrametric Contour Maps

1
Boundary Extraction in Natural Images Using
Ultrametric Contour Maps

• Pablo Arbeláez
• Université Paris Dauphine
• Presented by
• Derek Hoiem

2
What is segmentation?
3
What is segmentation?

• Segmentation is a result

4
What is segmentation?

• Segmentation is a result
• Segmentation is a process

5
What is segmentation?

• Segmentation is a result
• Segmentation is a process
• Segmentation is a guide

6
Segmentation as a Guide

• Multiple Segmentations

7
Segmentation as a Guide

• Multiple Segmentations
• Hierarchy of Segmentations

8
Key Concepts/Contributions

• Hierarchical segmentation by iterative merging
• Ultrametric dissimilarities
• Thorough evaluation on BSDS

9
Hierarchical Segmentation
3 Region Image
Dendrogram
Contour Image
10
Ultrametric Contour Map

• Ultrametric
• Definition D(x,y) lt max D(x,z), D(z,y)

The union R12 of two regions R1 and R2 must have
gt distance to adjacent region R3 than either R1
or R2
11
Ultrametric Contour Map
12
Region Dissimilarity

• Dc(R1, R2) mean boundary contrast
• contrast(x) max Lab diff within radius of x
• Dg(R1, R2) mean boundary gradient
• gradient(x) Pb(x)
• Da(R1) Area a3 Scatter (in color space)

a2
D(R1, R2) Dc(R1, R2) a1 Dg(R1, R2) min
Da(R1) , Da(R2)
Learned Parameters xi 4.5 a1 5
a2 0.2 a3 0
13
Examples
Contrast
Contrast Gradient
Contrast Gradient Region
14
Algorithm Summary

• Create Initial Contours
• Extrema in gray channel form regions
• Assign pixels to regions based on above
  ultrametric
• Iteratively merge regions
• Keep adjacency/distance matrix

15
Comparison

• Martin et al. (Pb)
• Canny edge detector
• Hierarchical watersheds (using MFM for gradient)
  Najman and Schmitt 1996
• Variational (global energy minimization)

16
Pb
Brightness Gradient
Oriented Edges
Color Gradient
Texture Gradient
No Boundary
Boundary
Martin Fowlkes Malik 2004
17
Pb
18
Variational Method
Originally Wavelet-based Textons
Koepfler Lopez Morel 1994
19
Comparison

• MFM Martin et al. (Pb)
• Canny Canny edge detector
• WS Hierarchical watersheds (using MFM for
  gradient) Najman and Schmitt 1996
• MS Variational (global energy minimization)

Edge-Based
Region-Based
20
Comparison
21
Results
22
Results
23
Best Results
http//www.ceremade.dauphine.fr/arbelaez/results-
UCM/main.html
24
Best Results
http//www.ceremade.dauphine.fr/arbelaez/results-
UCM/main.html
25
Best Results
http//www.ceremade.dauphine.fr/arbelaez/results-
UCM/main.html
26
Best Results
http//www.ceremade.dauphine.fr/arbelaez/results-
UCM/main.html
27
Median Results
28
Median Results
29
Median Results
30
Median Results
31
Worst Results
32
Worst Results
33
Worst Results
34
Worst Results
35
Hierarchies vs. Multiple Segmentations
36
Revising Segmentation