Bagofwords model and PLSA David Liu Feb 2006

1
Bag-of-words model and PLSADavid LiuFeb 2006
2
Related work

• ICCV05 Modeling Scenes with Local Descriptors
  and Latent Aspects
• P. Quelhas, F. Monay, J.-M. Odobez, D.
  Gatica-Perez, T. Tuytelaars , and L. Van Gool
  (IDIAP, Katholieke Univ. in Leuven)
• CVPR05 A Bayesian Hierarchical Model for
  Learning Natural Scene Categories
• Li Fei-Fei, P. Perona (UIUC, Caltech)
• ICCV05 Discovering objects and their location
  in images
• J. Sivic, B. C. Russell, A. A. Efros, A.
  Zisserman, W. T. Freeman (Oxford, CMU, MIT)

3
What do you see?
4
(No Transcript)
5
Outline

• Image Representation
• Learning
• Recognition

6
Slide from Li Fei-Fei
7
1.Feature detection and representation
Compute SIFT descriptor Lowe99
Slide credit Josef Sivic
8
2. Codewords dictionary formation
R128
Slide credit Josef Sivic
9
2. Codewords dictionary formation
Vector quantization
R128
Slide credit Josef Sivic
10
Examples of codewords
Fei-Fei et al. 2005
11
Examples of codewords
Sivic et al. 2005
12
3. Image representation bag of words
n(w,d)

w1
w2
w3
13
Outline

• Image Representation
• Learning
• Recognition

14
slide from C. Guestrin
15
slide from K. Murphy
16
slide from K. Murphy
17
slide from M. Jordan
18

• also talk about d-separation, to make conditional
  indep concept clear

19
Graphical representation of a mixture model
z
x
C. Bishop
20
Graphical representation of a mixture model
z
x
Graphical representation of a Gaussian mixture
model
zn
xn
N
C. Bishop
21
zk topic of a word z1 , z2
Model
A document is a mixture of topics. A topic is a
mixture of words.
22
zk topic of the word z1 , z2
Model
d1
d2
d3
w1
w2

w1
w2

w1
w2

23
zk topic of the word z1 , z2
Model
d1
d2
d3
w2
w1
w1
24
zk topic of the word z1 , z2
Model
Data
n(w,d)
25
zk topic of the word z1 , z2
z
w
d
Model
N
D
Things that we have control over
Parameters
Data
n(w,d)
26
zk topic of the word z1 , z2
Model
Things that we have control over
Parameters
P(zd)
P(wz)
Data
n(w,d)
27
zk topic of the word z1 , z2
Model
Things that we have control over
Parameters
P(zd)
P(wz)
Data
n(w,d)
28
d1
d2
d3
d4
w1
w1
w2
w3
Parameters
Data
P(wz)
P(zd)
n(w,d)
29
We have a model well-fitted to the data (Learning
stage)
n(w,d)
Data
Model
Next comes recognition
30
n(w,d)
d
w1
31
n(w,d)
d
w1
P(zd)
P(wz)
keep this fixed
32
skip
Thing that we can adjust
P(zd)
P(wz)
33
n(w,d)
d
Do optimization by EM and get
w1
P(zd)
P(wz)
keep this fixed
Most likely topic for d is z1
Image topic / category discovered
34
Decide Face vs. Nonface
ROC curve
35
Slide from J. Sivic
36
Slide from J. Sivic
37
Slide from J. Sivic
38
Lets read the titles again

• ICCV05 Modeling Scenes with Local Descriptors
  and Latent Aspects
• P. Quelhas, F. Monay, J.-M. Odobez, D.
  Gatica-Perez, T. Tuytelaars , and L. Van Gool
  (IDIAP, Katholieke Univ. in Leuven)
• CVPR05 A Bayesian Hierarchical Model for
  Learning Natural Scene Categories
• Li Fei-Fei, P. Perona (UIUC, Caltech)
• ICCV05 Discovering objects and their location
  in images
• J. Sivic, B. C. Russell, A. A. Efros, A.
  Zisserman, W. T. Freeman (Oxford, CMU, MIT)

39
Unsupervised object categorization!Unsupervised
object detection?
Paradigms

• Supervised all training data labeled
• Semi-supervised some labeled, some unlabeled
• Unsupervised finds structure, no /-

40
d4
d1
d3
d2
n(w,d)
P(wz)
P(zd)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.

41
d4
d1
d3
d2
n(w,d)
P(wz)
P(zd)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.

42
d4
d1
d3
d2
n(w,d)
P(wz)
P(zd)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.
• 1. feature if feature not good (similar to
  perception), then few features from the face
  region are captured. (extreme case all features
  captured are from the background)

43
d4
d1
d3
d2
n(w,d)
P(wz)
P(zd)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.
• 1. feature if feature not good (similar to
  perception), then few features from the face
  region are captured. (extreme case all features
  captured are from the background)

44
d4
d1
d3
d2
n(w,d)
P(wz)
P(zd)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.
• 1. feature if feature not good (similar to
  perception), then few features from the face
  region are captured. (extreme case all features
  captured are from the background)
• 2. data if only one face image (only d2 d3 d4,
  no d1), then d2 is no different than d3 d4.
  Cluster will not favor d2.

45
d4
d3
d2
n(w,d)
P(wz)
P(zd)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.
• 1. feature if feature not good (similar to
  perception), then few features from the face
  region are captured. (extreme case all features
  captured are from the background)
• 2. data if only one face image (only d2 d3 d4,
  no d1), then d2 is no different than d3 d4.
  Cluster will not favor d2.

46
d4
d1
d3
d2
n(w,d)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.
• 1. feature if feature not good (similar to
  perception), then few features from the face
  region are captured. (extreme case all features
  captured are from the background)
• 2. data if only one face image (only d2 d3 d4,
  no d1), then d2 is no different than d3 d4.
  Cluster will not favor d2.
• 2. data if exist many many background images
  similar to d3 or d4, then d1 d2 will just appear
  as noise.

47
d4
d1
d3
d2
n(w,d)

• Clustering depends on 1.feature, 2. data
• 1. feature if feature not good (robust to all
  kinds of deformations), then d1 and d2 will not
  have consistent features. SIFT is designed to be
  robust.
• 1. feature if feature not good (similar to
  perception), then few features from the face
  region are captured. (extreme case all features
  captured are from the background)
• 2. data if only one face image (only d2 d3 d4,
  no d1), then d2 is no different than d3 d4.
  Cluster will not favor d2.
• 2. data if exist many many background images
  similar to d3 or d4, then d1 d2 will just appear
  as noise.

48
EM algorithm for PLSA
Data
n(w,d)
Model
Goal
param
data
49
EM algorithm for PLSA
50
EM algorithm for PLSA
51
EM algorithm for PLSA
52
EM algorithm for PLSA
Optimize with Lagrange multiplier, and get
53
EM algorithm for PLSA
Optimize with Lagrange multiplier, and get
But what is Q
54
EM algorithm for PLSA
Optimize with Lagrange multiplier, and get
M-step
But what is Q
E-step
55
Fix P, find Q increases L to L L Fix Q,
find P Lagrange Multiplier on , increases
L, but also increases L
56
EM algorithm for PLSA
update P
update Q
update P
57
(No Transcript)
58
P(zd)
P(wz)
59
Do optimization by EM and get
P(zd)
P(wz)
60
n(w,d)
P(zd)
P(wz)
P(wd)
61
n(w,d)
P(zd)
P(wz)
P(wd)
P(zd)
P(wz)
P(wd)
62
n(w,d)
P(zd)
P(wz)
In either case, d1 and d2 in one cluster, d3 in
another cluster
P(zd)
P(wz)