Image Databases

1
Image Databases

• Conventional relational databases, the user types
  in a query and obtains an answer in response
• It is different in image databases
• a police officer may issue a query Retrieve
  all pictures from the image database that are
  similar to this person and give the identities
  of the people.
• This query is fundamentally different from
  ordinary queries for 2 reasons
• 1. The query includes a picture as part of the
  query
• 2. The query asks about similar pictures and
  therefore uses a notion of imprecise match

2
Raw images

• the content of an image consists of all
  interesting objects in that image
• each object is characterized by
• a shape descriptor that describes the
  shape/location of the region within which the
  object is located inside the image
• a property descriptor that describes the
  properties of individual pixels (e.g. RGB values
  of the pixel, RGB values aggregated over a group
  of pixels, grayscale levels)
• a property consists of
• a property name, e.g., red, green, blue, texture
• a property domain - range of values that a
  property can assume 0, 1, ..7

3
Images

• Every image is associated with a pair of positive
  integers (m,n), called grid-resolution, which
  divides the image into (m?n) cells of equal size
  (called image grid)
• Each cell consists of a collection of pixels
• A cell property (Name, Values, Method)
• Example
• (bwcolor, b,w, bwalgo, where the possible
  values are b(black) and w(white), and bwalgo is
  an algorithm that takes a cell as an input and
  returns either black or white by somehow
  combining the black/white levels of the pixels in
  the cell
• (graylevel, 0,1, grayalgo), where the possible
  values are real numbers within the interval
  0,1.

4
Image Database

• Image Database (GI,Prop,Rec)
• GI is a set of gridded images (Image,m,n)
• Prop is a set of cell properties
• Rec is a mapping that associates with each image,
  a set of rectangles denoting objects (in fact
  this does not necessarily have to be rectangle)

5
Problems with image databases

• Images are often very large
• infeasible to explicitly store the properties on
  a pixel by pixel basis
• This led to a family of image compression
  techniques attempt to compress the image into
  one containing fewer pixels
• There is a need to determine the features of
  the image (compressed or raw)
• done by segmentation breaking up the image
  into a set of homogeneous rectangular regions
  called segments
• Need to support match operations that compare
  either a whole image or a segmented image against
  another

6
Image Compression

• Lossy Compression
• Image may contain details that human eye cannot
  recognize
• get rid of those details
• DCT(Discrete Cosine Transform)
• DFT(Discrete Fourier Transform)
• DWT(Discrete Wavelet Transform)
• convert images from time domain(Spatial) to
  frequency domain
• get rid of the frequencies which do not contain
  information.
• Transforms
• DCT and DFT are similar concepts
• From time domain to signal domain
• Given a signal of length n, these transforms
  return a sequence of n frequencies.
• Sample1, Sample2, . . . . . . . , Sample n
  transforms to
• Freq1, Freq2, . . . . . . . . . , Freq n.

7
Why do we use the transform

• Noise removal is easier in the frequency domain
• Various filters are easier to implement in
  frequency domain
• Compression (gathers similar values together)

8
Desirable Properties of Transforms

• DFT
• Invertibility It is possible to get back the
  original image I from its DFT representation.
  (useful for decompression)
• Note practical implementations of DFT often use
  DFT with other non-invertible operations thus
  sacrifice invertibility
• Distance preservation DFT preserves Euclidean
  distance.
• This is important in image matching applications
  where we often use distance measures to represent
  similarity levels
• DCT
• DCT preserves all the above
• a given signal can be represented with fewer
  frequencies
• DWT
• DFT and DCT have no temporal locality
• a change in one single part of data changes all
  frequencies
• wavelets introduce locality

9
Distance preservation
10
Distance preservation
11
Fractal Compression

• Transform-based approaches benefit from the
  difference in visual perception in different
  frequencies
• What else can we use for compression ?
• Self similarity
• We can find self similarities in a given image
  and describe the image in terms of these
  similarities.

12
Fractal Compression
13
Image Processing Segmentation

• A process of taking an image as input and cutting
  up the image into disjoint homogeneous regions
• Connected region (R)
• is a set of cells C1 .. Cn in R such that the
  Euclidean distance between Ci and Ci1 for all i
  lt n is 1
• Example
• R1,R2,R3 is connected
• R1? R2 is connected
• R2? R3 is connected
• R1?R2? R3 is connected
• R1? R3 is not connected
• Because the Euclidian
• distance between (2,3)
• and (3,4) is ?2gt1

4 3 2 1
R3
R1
R2
1 2 3 4
14
Measuring Homogeneity

• Homogeneity predicate is a function H that takes
  any connected region as input and returns either
  true or false
• Example 1
• Suppose ? is some real number between 0 and 1
• H?bw can be defined as H?bw (R) is true if over
  (100?) of cells in R have the same color

Region H0.8bw H0.89bw H0.92bw R1
true false false R2
true true false R3
true true false
Region of black of white
cells cells R1
800 200 R2
900 100 R3 100
900
15
Measuring Homogeneity

• Example 1
• Suppose each cell has a real value between 0, 1,
  this value is bw-level
• Suppose f assigns a value between 0 and 1 to each
  cell
• Assume ? is the noise factor and ? a threshold
• H?,f,?(R) is true if (x,y) bwlevel(x,y)-f(x,y)
  lt ?/(m?n) gt ?

16
Segmentation

• Given an image I with (m?n) cells, a segmentation
  of I wrt a homogeneity predicate P is a set of
  R1, .Rk such that
• Ri ? Rj ? for all 1? i ? j ? k
• I R1 ? .. ? Rk
• H(Ri) true for all i ? j ? k
• for all distinct i,j, 1? I, j ? n such that Ri ?
  Rj is a connected region, it is the case that
  H(Ri ? Rj) false

17
An Example of Segmentation

• For Hdyn,0.03(R) of the following (4?4) image
  will yield the following segmentation
• R1 (1,1),(1,2)
• R2 (1,3),(2,1),(2,2),(2,3)
• R3 (3,1),(3,2),(3,3),(4,1),(4,2)
• R4 (3,4),(4,3),(4,4)
• R5 (1,4),(2,4)

Row/Col 1 2 3 4 1
0.1 0.25 0.5 0.5 2
0.05 0.30 0.6 0.6 3
0.35 0.30 0.55 0.8 4 0.6
0.63 0.85 0.90
Row/Col 1 2 3 4 1
0.1 0.25 0.5 0.5 2
0.05 0.30 0.6 0.6 3
0.35 0.30 0.55 0.8 4 0.6
0.63 0.85 0.90
18
Segmentation Algorithm

• Split
• if the whole image is homogeneous, we are done
• otherwise, split the image into two parts and
  recursively repeat this process till we find a
  set of R1 .. Rn such that each region is
  homogeneous
• Merge
• check whether any of the Ris can be merged
  together
• at the end of this step, we obtain a valid
  segmentation R1, ..Rk

19
Similarity Based Retrieval
20
Similarity Based Retrieval
21
Similarity Based Retrieval

• The Metric Approach
• Uses a distance measure d that can compare tow
  images
• The smaller the distance, the more similar they
  are
• I.e., given an input image I, find the nearest
  neighbor of I in the image archive
• The Transformation Approach
• The metric approach assumes that the notion of
  similarity is fixed
• Whereas the transformation approach computes the
  cost of transforming one image into another based
  on user-specified cost functions that may vary
  from one query to another

22
The Metric Approach

• We define a distance function on a k dimensional
  space (kn2)
• the distance function satisfies the following
  properties
• d(x,y) d(y,x)
• d(x,z) ? d(x,z) d(z,y)
• d(x,x) 0
• Example Let the image object consists of
  (256?256) cells with 3 attributes
  (red,green,blue) each of which assumes a value
  from 0,7
• di(o1,o2) ? ? ? (diffri,jdiffgi,jdiffbi,j
  )
• where diffri,j (o1i,j.red - o2i,j.red)2
• diffgi,j (o1i,j.green - o2i,j.green)2
• diffbi,j (o1i,j.blue - o2i,j.blue)2
• Such computations can be cumbersome (65536
  expressions being computed inside the sum)

23
The Metric Approach

• How can this massive similarity computation be
  avoided?
• Through feature extraction!
• Use a good feature extraction function fe and use
  it to map objects into single points in a
  s-dimensional space where s would typically be
  pretty small compared to n2
• This leads to two reductions
• an object is originally is a set of points in an
  (n2) dimensional space. In contrast, fe(0) is a
  single point
• fe(o) is a point in s-dimensional space where s
  ltlt (n2)
• The feature extraction mapping must preserve the
  distance relationships in the original space
• (n2) dim space ? s-dim space ? indexing
  algorithm ? index ? object repository (could be
  quadtree, R-tree for s-dim data)

24
Searching

• Finding the best matches
• find the nearest neighbors of fe(o) in the tree
  using a nearest neighbor search technique.
• Finding sufficiently similar objects
• execute a range query in the tree with center
  fe(o) and radius ?

25
The Transformation Approach

• The main principle
• the level of dis-similarity between o1,o2 is
  proportional to the cost of transforming o1 into
  o2, or vice-versa
• Transformation operators
• translation
• rotation
• scaling (uniform and nonuniform)
• excision
• Transformation of o into o is a sequence of
  transformation operations (to1,to2, ..tor) such
  that
• to1(o) o1
• ...
• To(or) o
• Cost of transformation, cost(TS) ? cost(toi)

26
Example
27
Example
28
Example
29
Transformation vs. Metric

• Advantages of the transformation model
• user can setup his own notion of similarity by
  specifying certain transformation operators
• user may associate a cost function with each
  transformation operator
• Advantages of the metric model
• by forcing the user to use only one similarity
  metric, the system can facilitate the indexing of
  data so as to optimize nearest neighbor search