October 2003 L1'1

1
Image SegmentationBased on the work of Shi
and Malik, Carnegie Mellon and Berkley and based
on the presentation of Jianbo Shi
2
Edge-based image segmentation

• Edge detection by gradient operators
• Linking by dynamic programming, voting,
  relaxation,
• - Natural for encoding curvilinear grouping
• - Hard decisions often made prematurely

3
Grouping with Bayesian Statistics
Bayes data structure data generation model
segmentation model
Segmentation is to find a partitioning of an
image, with generative models explaining each
partition. Generative models constrain the
observation data, f, and the prior model
constrains the discrete states, X. The solution
sought is the most probable state, or the state
of the lowest energy.
Image asobservation f
Texture models
Grouping asstate X
4
Image segmentation by pairwise similarities

• Image pixels
• Segmentation partition of image into segments
• Similarity between pixels i and j
• Sij Sji 0

Sij

• Objective similar pixels, with large value of
  Sij, should be in the same segment, dissimilar
  pixels should be in different segments

5
Relational Graphs

• G(V, E, S)
• V each node denotes a pixel
• E each edge denotes a pixel-pixel relationship
• S each edge weight measures pairwise similarity

• Segmentation node partitioning
• break V into disjoint sets V1 , V2

6
Solving MRF by Graph Partitioning
Some simple MRF models can be translated into
graph partitioning
pair relationships
data measures
7
Weighted graph partitioning
Pixels i I vertices of graph G Edges ij
pixel pairs with Sij gt 0 Similarity matrix S
Sij di Sj ? G Sij degree of
I deg A Si ? A di degree of A
G Assoc(A,B) Si ? A Sj ? B Sij
8
Cuts in a Graph

• (edge) cut set of edges whose removal makes a
  graph disconnected
• weight of a cut cut( A, B ) Si ? A, Sj ?
  B Sij Assoc(A,B)
• the normalized cut
• Normalized Cut criteria minimum cut(A,A)

NCut( A,B ) cut(A, B)( )
1 deg A
1 deg B
9
Grouping with Spectral Graph Partitioning
SGP data structure a weighted graph, weights
describing data affinity
Segmentation is to find a node partitioning of a
relational graph, with minimum total cut-off
affinity. Discriminative models are used to
evaluate the weights between nodes. The solution
sought is the cuts of the minimum energy.
?
NP-Hard!
10
Normalized Cut and Normalized Association

• Minimizing similarity between the groups, and
  maximizing similarity within the groups are
  achieved simultaneously.

11
Some definitions

• Rewriting Normalized Cut in matrix form

12
Generalized Eigenvalue problem

• after simplification, we get

13
(No Transcript)
14
Brightness Image Segmentation
15
Brightness Image Segmentation
16
(No Transcript)
17
Results on color segmentation
18
Motion Segmentation with Normalized Cuts

• Networks of spatial-temporal connections

• Motion proto-volume in space-time

19
(No Transcript)