Scalable recognition with a Vocabulary Tree

1
Scalable recognition with a Vocabulary Tree
David Nistér and Henrik Stewénius Department of
Computer Science, University of Kentucky
Oral presentation _at_ CVPR 2006

• Selected Topics in Computer Vision

Tatiana Tommasi
2
The paper presents

• A recognition scheme that scales efficiently to a
  large number
• of objects.
• Vocabulary Tree defined using an offline
  unsupervised training stage.
• Hierarchical scoring based on Term Frequency
  Inverse Document Frequency (TF-IDF).
• Local features Maximally Stable Extremal Regions
  (MSERs), Scale Invariant Feature Transform
  (SIFT).
• Extremely efficient retrieval a query takes 25ms
  on a database
• with 50000 images.

3
Building the Vocabulary Tree
k3 L2

4
Describing an Image
5
Describing an Image
6
Definition of Scoring

• Number of the descriptor vectors of each image
  with a path along the node i (ni query, mi
  database)
• Number of images in the database with at least
  one descriptor vector path through the node i (Ni
  )

Ni2 m_Img11 m_Img21
Ni1 m_Img12 m_Img20
7
Definition of Scoring

• Weights are assigned to each node
• Query and database vectors are defined according
  to their assigned weights
• Each database image is given a relevance score
  based on the normalized difference between the
  query and the database vectors

8
Implementation of Scoring

• Every node is associated with an inverted file.
• Inverted files stored the id-numbers of the
  images in which a particular node occurs and the
  term frequency of that image.
• decrease the fraction of images in the database
  that have to be explicitely considered for a
  query.

Img1, 1
Img2, 1
Img1, 2
9
Query
10
Database

• Ground truth database 6376 images
• Groups of four images of the same object but
  under different conditions
• Each image in turn is used as query image and the
  three remaining images from its group should be
  at the top of the query results

11
Results on 1400 images

• The curves show the distribution of
• how far the wanted images drop in
• the query rankings
• A larger (hierarchical) vocabulary improves
  retrieval performance
• L1 norm gives better retrieval performance than
  L2 norm.
• Entropy weighting is important at least for
  smaller vocabularies

12
Results on 6376 images
Performance increases significantly with the
number of leaf nodes Performance increases
with the branch factor k
13
Results on 6376 images
Performance increases when the amount of training
data grows Performance increases at the
beginning when the number of training cycles
grows, then reaches a plateaux
14
Results on 1 million images
Performance with respect to increasing database
size. The vocabulary tree is defined with video
frames separate from the database.

• Entropy weighting of the vocabulary tree defined
  with video independent from the database.
• Entropy weighting defined using the ground truth
  target subset of images.

15
Applications
16
Conclusions- Take home message
This methodology provides the ability to make
fast searches on extremely large databases. If
we can get repeatable, discriminative features,
then recognition can scale to very large
databases using the vocabulary tree and indexing
approach.
17
Definition of Scoring

• Weight wi - assigned to each node of the
  vocabulary tree
• constant
• based on entropy wi ln(N/Ni)
• N number of images in the database
• Ni number of images in the database with
  at least one descriptor vector path through
• the node i
• Frequency of occurence of node i in place of Ni
• Stop lists, wi is set to zero for the most
  frequent and/or unfrequent nodes.

18
Building the Vocabulary Tree

• k-means clustering, k defining the branch factor
  of the tree
• L levels
• Determining the path of a descriptor means
  performing kL dot products
• The tree defines the visual vocabulary
• and an efficient search procedure.

k3 L1
L2
L3
L4