Sara Vicente, Vladimir Kolmogorov, Carsten Rother

1

• Graph cut based image segmentation with
  connectivity priors

• Sara Vicente, Vladimir Kolmogorov,Carsten
  Rother
• Microsoft Research CambridgeUniversity College
  London

2
Motivation
GrabCut standard(shrinking bias)
GrabCut with Flux
Geodesic Segmentation Bai et. al 07(only
connectivity wrt brushes)
GrabCut low coherency
Our result
3
Defining the Problem
image
User input
GrabCut
Energy minimization
Smootness-term
Data-term
C(x) 0 only if connected otherwise
Valid solution
Properties - higher-order clique (whole
image) - NP hard to solve - Each connected
component is either entirely in- or excluded
from the global optimum
4
Defining the Problem (specific)
image
User input
Valid solution
Energy minimization
C(x) 0 if 2 points connected otherwise
Properties - NP hard to solve - without
pairwise terms not NP hard
5
Defining the Problem (specific ext.)
image
Valid solution
User input
GrabCut
Previous constraint
Energy minimization
C(x) 0 if connected path has a certain width
Properties - NP hard to solve
6
Powerful User Interface (ideal)
Bkg brush
Segmentation is connected
Fgd brush
Segmentation has no holes
Width of last change
Input
Segmentation
7
Specific Problem Two Methods

• DijkstraGC
• heuristic method
• relatively fast (0.2 - 200 seconds)
• Decomposition
• very slow (3 hours)
• gives lower bound and 30 of time the global
  optimum

8
Dijkstra Algorithm
2
1
2
6
2
4
4
6
3
3
1
4
1
Goal Compute shortest path from start node to
any other node
9
Dijkstra Algorithm
8
8
2
1
8
2
6
8
2
4
4
6
3
3
0
8
1
4
8
1
8
10
Dijkstra Algorithm
8
8
2
1
8
2
6
8
2
4
4
6
3
3
0
8
1
4
8
1
8
11
Dijkstra Algorithm
2
6
2
1
8
2
6
8
2
4
4
6
3
3
0
3
1
4
3
1
1
12
Dijkstra Algorithm
2
6
2
1
8
2
6
8
2
4
4
6
3
3
0
3
1
4
3
1
1
13
Dijkstra Algorithm
2
6
2
1
8
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
14
Dijkstra Algorithm
2
6
2
1
8
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
15
Dijkstra Algorithm
2
6
2
1
8
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
16
Dijkstra Algorithm
2
6
2
1
8
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
17
Dijkstra Algorithm
2
4
2
1
3
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
18
Dijkstra Algorithm
2
4
2
1
3
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
19
Dijkstra Algorithm
2
4
2
1
3
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
20
Dijkstra Algorithm
2
4
2
1
3
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
21
Dijkstra Algorithm
2
4
2
1
3
2
6
8
2
4
4
6
3
3
0
3
1
4
2
1
1
22
Dijkstra Algorithm
2
4
2
1
3
2
6
6
2
4
4
6
3
3
0
3
1
4
2
1
1
23
Dijkstra Algorithm
2
4
2
1
3
2
6
6
2
4
4
6
3
3
0
3
1
4
2
1
1
24
Dijkstra Algorithm
2
4
2
1
3
2
6
6
2
4
4
6
3
3
0
3
1
4
2
1
Backward links allow reconstruction of minimum
path
1
25
DijkstraGC Algorithm
p
s
7
2
3
0
0
0
optimal cut
0
3
Goal Compute a minimum cut (segmentation) where
s and p are connected
3
2
1
1
3
0
Idea Replace Distance by cost of cut which
satisfies connectivity
5
3
1
26
DijkstraGC Algorithm
8
8
8
8
1
7
2
3
cut
0
0
0
8
8
0
3
3
2
1
1
3
0
5
3
1
27
DijkstraGC Algorithm
8
8
8
8
1
7
2
3
0
0
0
8
8
0
3
3
2
1
1
3
0
5
3
1
28
DijkstraGC Algorithm
6
8
8
8
1
cut
8
8
8
29
DijkstraGC Algorithm
6
8
8
8
1
cut
8
8
8
30
DijkstraGC Algorithm
6
8
8
8
1
cut
7
8
8
31
DijkstraGC Algorithm
6
8
8
8
1
7
2
3
0
0
0
7
8
0
3
3
2
1
1
3
0
5
3
1
32
DijkstraGC Algorithm
6
6
8
8
1
8
cut
8
7
8
33
DijkstraGC Algorithm
6
6
8
8
1
7
2
3
0
0
0
7
8
0
3
3
2
1
1
3
0
5
3
1
34
DijkstraGC Algorithm
6
6
8
14
1
cut
8
8
8
7
8
35
DijkstraGC Algorithm
6
6
8
14
1
8
cut
8
8
7
8
36
DijkstraGC Algorithm
6
6
8
14
1
7
2
3
0
0
0
7
8
0
3
3
2
1
1
3
0
5
3
1
37
DijkstraGC Algorithm
6
6
8
14
1
13
7
cut
8
8
38
DijkstraGC Algorithm
6
6
8
14
1
7
2
3
0
0
0
13
7
0
3
3
2
1
1
3
0
5
3
1
39
DijkstraGC Algorithm
6
6
8
14
1
cut
10
7
8
8
40
DijkstraGC Algorithm
6
6
8
10
1
10
7
cut
8
8
41
DijkstraGC Algorithm
6
6
8
10
1
7
2
3
0
0
0
10
7
0
3
3
2
1
1
3
0
5
3
1
42
DijkstraGC Algorithm
6
6
8
10
1
7
2
3
0
0
0
10
7
0
3
3
2
1
1
3
0
5
3
1
43
DijkstraGC Algorithm
6
6
8
10
1
10
7
best cut
8
8
Stop with best found cut
44
Suboptimality of DijkstraGC
0
2
0
0
2
1
0
0
3
3
1
2
0
2
0
0
Direction of DijkstraGC matters(in practice less
than 1 of nodes differently labelled)
45
DijkstraGC

• Speed Recycle flow search trees Kohli et al.
  04
• Theorem DijkstraGC gives global optimum if no
  pairwise terms present
• Conjecture DijkstraGC gives global optimum if
  E(1) lt E(0) for all nodes

46
Results - DijkstraGC
User input
GrabCut
size 321x481 time 0.2sec width 1
additional input
DijkstraGC
47
Results - DijkstraGC
User input
GrabCut
size 458x561 time 203sec width 3
additional Input
DijkstraGC
48
Results - DijkstraGC
User input
GrabCut
size 600x450 time 1.5sec width 3
DijkstraGC
additional input
49
Decomposition Approach
Minimization
(Lower bound)
50
Decomposition Approach
Subgradient Optimization Shor 70, Wainwright et
al.05
x1
subgradient
?
x2
?
x3
?
Subproblem 1
Subproblem 2
Subproblem 3

• Guaranteed to converge, close to optimal
  solution of (depending on )
• Lower bound might not be tight

51
Decomposition Approach
Unary terms pairwise terms Global
minimumGraphCut
Unary terms Connectivity constraint Global
minimum Thresholding Dijkstra
Pairwise terms Connectivity constraint Lower
Bound Based on minimal paths on a dual
graphExtra assumption - Planar graph -
Boundary nodes 0
52
Results - Decomposition

• Global optimum 12 out of 40 cases. For those,
  DijkstraGC gives same solution
• Slow (average 3hours)

GlobalMin
GrabCut
Image
Input
53
Conclusions

• Practically Important Problem
• Good results
• Future Work Faster, heuristic methods (visually
  good even if energy is higher)

54
OLD
55
Results - DijkstraGC
Width 2time 2.9sec