Content-Based Image Indexing

1
Content-Based Image Indexing

• Joel Ponianto
• Supervisor
• Dr. Sid Ray

2
Outline

• Introduction to Content-Based Image Indexing
• Images Features Extraction
• Tree Structure
• System Model
• Retrieval Approach
• Experiment Results
• Conclusion

3
Introduction to Content-Based Indexing

• Content-Based Image Indexing (CBII) is an
  interrelated issue with Content-Based Image
  Retrieval (CBIR).
• CBIR depends on CBII and vice versa.
• CBIR focus on how to retrieve image accurately
  and efficiently.
• While CBII concern with how to support retrieval
  process.

4
Introduction to Content-Based Image Indexing Cont

• CBiI as pre-process of CBIR sequences.
• Cannot ignore retrieval process to create good
  indexing structure.
• The idea of indexing is similar with a library
• Every book has a unique id
• Every book has properties

5
Introduction to Content-Based Image Indexing Cont

• Examples title, author, publisher, etc
• Those properties are used to search the book.
• People know it as keyword
• Similar idea with images, however not that
  simple.
• Cannot represent an image with simple text. (can
  but not make sense)

6
Introduction to Content-Based Image Indexing Cont

• How to represent an image?
• By using its properties such as, colour, shape,
  texture and others.
• Choose which properties need to be extracted for
  indexing purpose ( and also retrieval).
• Also choose which method to extract those
  properties / features.

7
Images Features Extraction Cont

• Colour, shape and texture have their own
  sub-features.
• Colour grey level, RGB/HUE value, grey sigma,
  local histogram and average colour value.
• Shape area, centroid, circularity and moment
  invariant.
• Texture contrast, orientation and anisotropy.

8
Images Features Extraction Cont

• The selection of features is also effected by the
  data set.
• what we want to achieve at the retrieval stage is
  effected by the data set.
• If the data set is full of houses image and a
  user want to look for a car image.
• Try to select features that can differentiate
  each class in the data set.

9
Images Features Extraction Cont

• For this project I select the following features
• Colour Sigma (Global)
• Edge density (Global)
• Colour Average (Global)
• Boolean edge Density (Global)
• Edge Direction (Global)
• Region area (Region)
• Moment invariant (Region)
• Grey level (Region)

10
Images Features Extraction Cont

• Colour Sigma
• Find the standard deviation (s) of the image, for
  each colour layer.

11
Images Features Extraction Cont

• Edge Density
• Enhance the pixels that belong to the edges and
  boundaries by using a standard edge detector.
  Pixels far from edges will drop to 0 and those
  near to an edge will increase to max. calculate
  the mean pixel value of the resultant image.
• Colour Average
• Sum all the pixel value for each colour layer and
  divide by the number of pixel.

12
Images Features Extraction Cont

• Boolean Edge Density
• From above edge density, the image is thresholded
  so that what could be called edge pixels are
  white (1) and non-edge pixels are black (0).
  Count white pixel in the image.
• Edge Direction
• With some edge detection (Sobel Operator), allow
  us to make a crude estimation of a edge direction
  for particular region.

13
Images Features Extraction Cont

• Area, Grey Value and moment invariant
• These features is calculate on regional basis.
• The region is calculated with combination of
  k-mean clustering and Connected Component
  labelling Algorithm
• Calculate a grey level value of an image and
  perform the k-mean clustering.
• Use the connectivity algorithm to group similar
  grey value by its location.

14
Images Features Extraction Cont

• http//www.cis.rit.edu/class/simg782.old/talkMomen
  ts/momentEquations.html
• I use the first four of seven invariant moment
  for this project.

15
Images Features Extraction Cont
16
Images Features Extraction Cont

• Quantisation
• To be suitable for computer processing and
  features extraction (colour), an image must be
  digitized in amplitude.
• The idea is to reduce the colour space while
  gaining the ability to localize colour
  information spatially.
• this project applies quantisation at HSV colour
  space.

17
Images Features Extraction Cont
18
Images Features Extraction Cont

• RGB to HSV
• Let RGB values ranged from 0 to 1 and MIN/MAX
  corresponds with RGB values.

19
Images Features Extraction Cont

• HSV to RGB
• H range from 0 - 360
• V and S range from 0 1
• If S 0 then RGB V
• Else use next formula

20
Images Features Extraction Cont
21
Images Features Extraction Cont
22
Tree Structure

• There are many choices of tree structures that
  can handle multi-dimensional space. Such as
  R-Tree, R-Tree and Vp-Tree
• We look at R-Tree tree structure
• This project used R-Tree to simplify the
  computation.
• Other tree structures can be use on the system.

23
Tree Structure Cont

• R-Tree (Antonin Guttman)
• A R-Tree is a height balance tree and all leaves
  are on the same level.
• Root node has at least two children unless it is
  the leaf node.
• Every non-leaf node contains between m and M
  entries unless it is the root.
• For each entries (I, childnode-pointer) in a
  non-leaf node, I is the smallest rectangle that
  spatially contains all rectangles in its child
  nodes.
• Every leaf node contains between m and M index
  records unless it is the root.
• For each index record (I, tuple-identifier) in a
  leaf node, I is the smallest rectangle that
  spatially contains the n-dimensional data object
  represented by the indicated tuple.

24
Tree Structure Cont
25
Tree Structure Cont
26
System Model

• Put into data base

27
System Model Cont

• The System input around 300 images into the data
  base.
• Those images is divided into 10 different
  classes animal, car, flower, face, fruit, house,
  lake, mountain, plane and sunset.
• Store into persistence storage.

28
System Model Cont

• In the binary threshold stage, I attempt to
  separate the background image with the object.
• Although this stage is very weak, but in some
  images. The result can be helpful (and possible
  the other way around).

29
System Model Cont

• Binary Threshold good result

30
System Model Cont

• Binary Threshold bad result

31
Retrieval Approach

• Query sequence

32
Retrieval Approach Cont

• For finding similarity, I use Euclidean distance
  measure formula
• Where
• p is the database image
• q is the query image
• Pi is the database images ith features
• Qi is the querys ith features
• n is the number of features
• W is the weight for ith feature

33
Retrieval Approach Cont

• wi is the weight of feature i
• from relevant images
• (si) is the standard deviation of
• feature i from relevant images
• wt is the total weight of feature I
• wt is the normalised weight

34
Retrieval Approach Cont

• Gaussian Normalisation (for feature
  normalization)
• d(fi,fj) is the similarity of image fi and fj,
• range in -1, 1
• sij and µij are the standard deviation and
• mean of each feature respectively.
• d(fi,fj) is to make d(fi,fj) in range 0, 1

35
Experiment Result

• Go to Excel file
• m1-m8 only use global features
• m3 uses colour avg, colour sigma and edge density
• m2 uses colour avg and colour sigma
• m8 uses colour sigma and edge density
• m9 use region features m3

36
Conclusion

• Indexing depend on retrieval and vice versa
• No universal system / method for indexing or
  retrieval.
• We can try to develop something that robust.
• Indexing base on regional features give better
  result then global features.
• With more time, more result can be produced.

37
Reference

• Kompatsiaris, I., Triantafillou, E. and
  Strintzis, M. G., Region-Based Color Image
  Indexing and Retrieval, 2001
• Parker, J. R., Behm, B., Use of Multiple
  Algorithm in Image Content Searches,
  International Conference on Information
  Technology Coding and Computing (ITCC04)
  Volume2 p.246.
• Smith, J. R., Chang, S., Single Color Extraction
  and Image Query, International Conference on
  Image Processing (ICIP-95), Washington, DC, Oct,
  1995.
• Park, J. M., Looney, C. G., Chen, H. C., Fast
  Connected Component Labeling Algorithm Using A
  Divide and Conquer Technique, Technical Report,
  2000
• Chiueh, T., "Content-Based Image Indexing," in
  Proceedings of International Very Large DataBase
  Conference, VLDB '94, Santiago, Chile, September,
  1994.
• Gonzalez, R. C. and Woods, R. E., Digital Image
  Processing, 1993, Addison-Wesley Publishing
  Company, inc, 3rd edition.