Contentbased Image Retrieval Term Project

1
Content-based Image Retrieval -- Term
Project
Meng Ding Dec 4, 2002
Implementing Existing Algorithms
2
Contents

• Introduction
• Implementation
• Evaluation
• Conclusion

3
Introduction

• CBIR based on Low Level Feature Extraction (Level
  1)
• CBIR based on Sample (Query) Image

4
Introduction

• Low Level Feature Extraction
• 1. Global Histogram Based
• 2. Segmentation Based (finding objects)

5
Introduction
6
Contents

• Introduction
• Implementation
• Evaluation
• Conclusion

7
Implementation Global Hist Based

• Image Preprocessing
• Feature Extraction
• Similarity Measurement

8
Global Hist Based Preprocessing

• Cutting Image Boundaries

9
Global Hist Based Color Histogram

• Convert to HSV Color Space

10
Global Hist Based Color Histogram

• Convert to HSV Color Space

where , ,
, and
is the corresponding point of
in the HSV color space.
11
Global Hist Based Color Histogram

• Quantize into 256 bins with

16 levels in H Channel 4 levels in S Channel 4
levels in V Channel
12
Global Hist Based Edge Histogram

• Convert to YCbCr Color Space

13
Global Hist Based Edge Histogram

• Compute Luminance Gradients

Vertical Horizontal 45
degree 135 degree isotropic

• Threshold 50 for Sum of Gradient Values
• Quantize Edge Histograms into 5 Bins

14
(No Transcript)
15
Global Hist Based Similarity

• Histogram Intersection for Both Color and Edge
  Features

16
Global Hist Based Similarity

• Combine Two Features
• Sort overall Distance
• Retrieve top 20 images

where s is the sample image, i is the image in
the image database, dk(I,I) is the result from
histogram intersection for all features and wk is
the weight on each feature value.
17
Implementation Segmentation Based

• Based on UC Berkelys Blobworld
• Three Steps
• 1. Extract color and texture features for
• each pixel
• 2. Grouping of pixels by Expectation
  Maximization
• Algorithm and Minimum Description Length
• Principle
• 3. Description of Region features and
• Similarity Comparison for Image Query

18
Segmentation Based Color Features

• Convert to Lab Color Space

• L Channel is used for texture extraction

19
Segmentation Based Color Features

• Smooth the image using a smooth filter

20
Segmentation Based Texture Features

• Three Local Texture Descriptors
• Contrast, Anisotropy and Polarity

• Three Local Texture Descriptors
• Contrast, Anisotropy and Polarity

• All features derived from gradients

21
Segmentation Based Texture Features

• Second Moment Matrix (Averaged Squared Gradient
  Matrix)

where is the approximation to
a Guassian smoothing kernel with
variance 2.
22
Segmentation Based Texture Features

• Eigenvalues of Second Moment Matrix
• Comparisons between and

23
Segmentation Based Texture Features

• Contrast
• Anisotropy
• Polarity
• Dominant Orientation within local region

24
Segmentation Based Combine Features

• Six Dimensional Features for each pixel
• 1. Smoothed L
• 2. Smoothed a
• 3. Smoothed b
• 4. Contrast c
• 5. Anisotropy ac
• 6. Polarity pc

25
Segmentation Based Build Models

• Selection of K mean vectors
• Two cases when K 3

26
Segmentation Based Build Models

• Evaluate Models using Expectation Maximization
  (EM Algorithm)
• Initialization
• where X is a feature vector, is
  normalized mixing weights (Initial Value 1/K) ,
  is the collection of all parameters, f is
• Guassian density with
  representing K mean vectors,
• representing K covariance
  matrices and d the number of features

27
Segmentation Based Build Models

• Updates
• Where

is the probability of fits given
parameter .
28
Segmentation Based Choose Model

• Stop Condition
• Stop when does not increase within 10 iterations.
• Choose the Number of Regions K to maximize the
  for formula (MDL)

29
Segmentation Based Group Pixels

• Compute Probability Density for Each Pixel Under
  the Selected Model
• Replace the pixels with label of the cluster for
  which it attains the highest likelihood

30
Segmentation Based Group Pixels

• Perform 8-way Connected-Components Algorithm to
  find regions
• 1 1 0 1 1 0 1 1 1 0 1 1 0
  2
• 1 1 0 1 1 0 1 1 1 0 1 1 0 2
• 0 0 1 0 0 0 1 0 0 1 0 0 0 2

31
Segmentation Based Region Description

• Lab color histogram 218 bins with width 20 in
  each direction
• Mean Contrast Value
• Mean Anisotropy Value

32
Segmentation Based Segmentation Representation
33
(No Transcript)
34
Segmentation Based Similarity Measure

• Histogram Intersection for Color features
• Euclidean distance for texture features
• Combine Color and Texture features using
  normalized weights (currently 0.5 for each
  feature)
• Maximum similarity from all region pairs between
  two images
• Sort the Maximum similarity value and pick up the
  top 20.

35
Contents

• Introduction
• Implementation
• Evaluation
• Conclusion

36
Evaluation Testing Images

• Downloaded from Stanford University Database
  Group, 10,000 test images with average size for
  each image 128X85
• Segmented images so far 1100, with 1000 images
  from the original database and the rest 100
  images (Great Walls) added by ourselves (for
  convenience of performance comparisons)

37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
Evaluation Comparison
42
(No Transcript)
43
(No Transcript)
44
Evaluation Comparison
45
Contents

• Introduction
• Implementation
• Evaluation
• Conclusion

46
Conclusion

• It works, sometimes even produce good results
• Still a VERY DIFFICULT problem, semantic
  retrieval is almost impossible at present
• Segmentation based approach is not necessarily
  better than Global histogram based approach
• Difficult to compare the performance of different
  CBIR systems
• Impractical for WWW content-based image retrieval
  given current knowledge

47
Thank You!