Video/Image Fingerprinting

1
Video/Image Fingerprinting Search

• Naren Chittar
• CS 223-B project, Winter 2008

2
Problem Definition

• Given an image, find copies on the web.
• Given a video clip, find movie that it belongs
  to.
• Two problems
• Feature Extraction
• Efficient Search

3
Feature Extraction
0 0 0 0 0 0 35 12 21 14 0 0 0 0 0 13 26 20 20
25 0 0 0 0 0 21 19 20 34 25 0 0 0 0 0 31 16 29 51
21 0 0 0 0 1 36 15 29 18 22 0 0 0 0 7 33 19 39 24
22
Partition into 10x10 blocks
Gradient magnitude
Normalized sum of gradients
Samples
4
Search Inverted Index
0 0 0 0 0 0 0 0 0 0 0 35
0 13 26 0 21 19 29 51 21 29 18 22 39 24
22
0 0 0 0 0 0 35 12 21 14 0 0 0 0 0 13 26 20 20
25 0 0 0 0 0 21 19 20 34 25 0 0 0 0 0 31 16 29 51
21 0 0 0 0 1 36 15 29 18 22 0 0 0 0 7 33 19 39 24
22
Sliding and overlapping 3x3 window
DB (spatial data structure k-d tree)?
9 dim feature vectors
0 0... --gtdoc1.jpg, doc3.jpg, doc10.jpg 0 2
... --gtdoc2.jpg doc10.jpg
5
Search (continued)?
0 0 0 0 0 0 0 0 0 0 0 35
0 13 26 0 21 19 29 51 21 29 18 22 39 24
22
hits
hits
Feature extraction
hits
query
Features
Hits 70 doc1234.jpg 20 doc2345.jpg 5
doc2222.jpg 1 doc2356.jpg .....
6
Results
Database 10,000 images. From Flickr Test
set 30 images.
7
Conclusions and Future Work

• Technique robust to scaling, resolution loss.
• Works only for small amount of cropping.
• Implement kd-tree for faster search.
• Enlarge database to 1 million images and include
  videos.
• Augment feature vector with color attributes and
  other information.
• Compute relevance score based on position of
  feature vector in image.