MMR

1
MMR

• MASK BASED
• MULTI-RESOLUTION IMAGES AND VIDEOS

2
Multi Resolution Image

• Multi-Resolution (MR) images are useful for
  compact representation of images/videos
• Quad-trees are popular data-structures for
  Multi-resolution (MR) image representation

3
Lossy vs Lossless Compression

• Lossless data compression is a class of data
  compression algorithms that allows the exact
  original data to be reconstructed from the
  compressed data
• A lossy compression method is one where
  compressing data and then decompressing it
  retrieves data that may well be different from
  the original, but is close enough to be useful in
  some way.
• Advantage of lossy method is that file size is
  reduced while maintaining the semantics.

4
Quad-tree representation of an image

• A 2nx2n image is represented as a tree of depth
  n.
• Each node at every resolution level encodes its
  own color values (RGB) the parent node color is
  the mean of the colors of its child nodes.

5
Standard Quad-Tree Pruning

• At each node a decision is made as to whether to
  decompose the corresponding block into four
  equal-size squares or to halt the decomposition
  process

6
Problems

• Image semantic is lost when standard
  color-similarity based pruning is used.
• Sub-optimal MR images.

7
Result Standard Pruning (MR)
Note that the shape of the globe is lost when
standard color-similarity based pruning is used
Standard Pruning
Original Image
8
Possible Solution

• content-driven masks to prune the trees, instead
  of simple color-similarity

9
Example Mask based Pruning (MMR)
Note that the shape of the globe is PRESERVED
when mask-based pruning is used !
Mask Based Pruning
Mask Binary Image
10
MMR Technique

• Prune nodes which are redundant
• Do not prune nodes which have a mask pixel as
  leaf.
• Define redundancy of a node by introducing a new
  attribute, REDUNDANCY, in the Quad(Tree)Node data
  structure.

struct QuadNode ENUM type NODE LEAF QuadNode
child0, 10, 1 COLOR RGBA REDUNDANCY 0 or
1 end struct
11
Algorithm Redundancy Detection
function Mask_Prune(QuadNode p) returns 0 or
1 if p. type LEAF then i, j ? pixel
corresponding to this leaf return (1 -
Mask-Bin(i, j)) end if redundancy 1 //
assume this node is redundant for each i, j 0
and 1 flag Mask_Prune(p.childij) if
flag 0 then redundancy 0 end
for p.REDUNDANCY redundancy return
redundancy end function
12
Result

• The function Mask_prune results in the best
  resolution by preserving the original pixels in
  the smallest size image blocks corresponding to
  the non zero mask values.
• Results in maximum no of low resolution blocks in
  the other regions of the image.

13
Results for Standard Pruning
Lenas face has been distorted because the
standard MR algorithm is neither aware of the
face, nor the edges
dMR 72.4
Original Image
Standard Pruning
14
Results for combination of masks
Lenas face is UNTOUCHED because the MMR
algorithm is aware of the face and the edge
regions !
dMR 78.4
Mask Based Pruning
EdgeFace Mask Combo
15
Where does the mask come from?

• Feature mask edges and boundaried
• Object mask Regions depicting semantic
  objects like face
• Motion mask Moving objects in a video

16
Edge Detection

• The goal of edge detection is to mark the points
  in a digital image at which the luminous
  intensity changes sharply.
• Sharp changes in image properties usually reflect
  important events and changes in properties of the
  world.

17
Algorithm

• Apply a Gaussian filter mask to smooth the image
  to mitigate noise effects.
• Find the magnitude and the direction of the
  gradient.
• Apply nonmaxima suppression resulting in thinned
  image
• Apply hysteresis thresholding

18
Step 1 Gaussian Smoothing

• Gaussian Function
• Array of smoothed data

19
Step 2 - Gradient Calculation

• Magnitude and direction of gradient using Sobel
  operator.
• Edge Magnitude ?(Gx2Gy2)
• Edge Direction tan-1Gx/Gy

20
Step 3 Nonmaxima Supression

• an edge point is defined to be a point whose
  strength is locally maximum in the direction of
  the gradient.
• Thinning of edges.
• 20 -
• 91-
• 92-

21
Step 4 Hysteresis Thresholding

• Avoiding false edges and missing edges.
• Apply a high threshold.
• Apply a low threshold to the pixels connected to
  the pixels with the high threshold.
• Performed multiple times.

22
Result
23
ANN

• Information processing paradigm
• Simulates a human neural network
• Learn from examples
• Configured for a specific application through a
  learning process

24
Face Detection

• Face detection is a computer technology that
  determines the locations and sizes of human faces
  in arbitrary (digital) images.
• It detects facial features and ignores anything
  else, such as buildings, trees and bodies.

25
Human neurons vs Artificial Neurons
26
ANN Models

• Feed forward Networks
• Feed back Networks
• Advantages of feed forward networks
• Easy to understand and implement
• Fairly good performance

27
Feed forward Networks
28
Closer Look at the Neuron
29
Training the Network

• Use training data to get an output
• Calculate the error
• Propagate the error down the layers
• Adjust the weights
• Back propagation Algorithm

30
Closer look at the neuron
31
Face detection using neural network

• Algorithm works in 2 stages
• Stage 1 applies a set of neural network-based
  filters to an image
• Stage 2 uses an arbitrator to combine the filter
  outputs.

32
Stage 1 Neural Filter
33
Steps

• Filter receiving 20X20 pixel region of the image
• Filter applied at every location
• Input image reduced in size by subsampling
• Preprocessing step
• Histogram equalization

34
Steps

• Preprocessed window is passed through a neural
  network.
• The network has connection to the receptive
  fields of the hidden unit
• 4 - 10X10 pixel subregions
• 16 - 5X5 pixel subregions
• 6 overlapping 20X5 pixel horizontal strips of
  pixels.

35
Result of the neural filter
36
Training data

• Collecting a representative set of
• Face and
• Non Face data
• 15 face egs are generated from the each original
  image by randomly rotating, translating, scaling
  and mirroring

37
Example face images
38
Non face training data

• Images collected during training
• Create an initial set of 1000 images with random
  pixel intensities.
• Train network using error backpropogation to
  output 1 and -1 for face and non face images.
• Run the system on an image of scenery which
  contains no face. Collect incorrect sub images.
• Select 250 of these and add as negative egs.

39
Non-face images
40
Stage 2 Merging overlapping detections

• Observation most faces are detected at multiple
  nearby positions or scales, while false
  detections occur with less consistency.
• Thresholding for each location and scale
• The centroid of the nearby detections defines the
  location of the desired result.

41
Results of Neural Network Filter
42
Motion detection

• Motion detection works on the basis of frame
  differencing - meaning comparing how pixels
  (usually blobs) change location after each frame.

43
Background Subtraction

• Background subtraction is the method of removing
  pixels that do not move, focusing only on objects
  that do.

44
Background Subtraction-The problem

• Main goal given a frame sequence from a fixed
  camera, detecting all foreground objects.
• Naïve description of the approach detecting the
  foreground object as the difference between the
  current frame and an image of the scenes static
  background
• framei-backgroundi Th

45
Problem

• How to automatically obtain the image of the
  scenes static background?

46
The problem-requirements

• The background image is not fixed but must
  adapt to
• Illumination changes
• gradual
• sudden (such as clouds)
• ??Motion changes
• camera oscillations
• high-frequencies background objects (such as tree
  branches, sea waves, and similar)

47
The basic method

• Frame difference
• frameiframei-1 Th
• The estimated background is just the previous
  frame
• It evidently works only in particular conditions
  of objects speed and frame rate
• Very sensitive to the threshold Th

48
Frame difference Example
49
Running Average

• Background as the average or median of the
  previous n frames.
• Rather fast, but memory consuming
• N size(frame)
• Background as the running average
• Bi 1 a Fi (1 -a) Bi
• a, the learning rate, is typically 0.05
• no more memory requirements

50
Running Average contd..

• Two background corrections are applied
• If a pixel is marked as foreground for more than
  m of the last M frames, then the background is
  updated as Bi 1 Fi
• designed to compensate for sudden illumination
  changes and the appearance of static new objects.
• If a pixel changes state from foreground to
  background frequently, it is masked out from
  inclusion in the foreground.
• designed to compensate for fluctuating
  illumination, such as swinging branches.

51
MMR vs MR

• Comparison involves 2 attributes
• Visual quality obtained
• Size in bytes of encoded image/video.
• Visual quality is subjective to the user, based
  on application requirement.

52
Measure of rate-distortion performance
d 100n / N n number of pixels
in the multi-resolution image, obtained by
treating each block as a single pixel N total
number of pixels in the original image
53
More Results for Standard Pruning
Lenas face has been distorted because the
standard MR algorithm is neither aware of the
face, nor the edges
dMR 72.4
Original Image
Standard Pruning
54
Results for combination of masks
Lenas face is UNTOUCHED because the MMR
algorithm is aware of the face and the edge
regions !
dMR 78.4
Mask Based Pruning
EdgeFace Mask Combo
55
More Results for Standard Pruning
Note that the moving person has been distorted,
because standard MR pruning is not aware of the
motion
dMR 72.2
Standard Pruning
Original Image
56
Results for combination of masks
Note that the resolution of the moving person is
INTACT, because MMR is aware of the motion !
dMMR 78.1
EdgeMotion Mask Combo
Mask Based Pruning
57
Comparitive analysis

• dMMR is comparable to dMR
• MMR image/video retain all the desired visually
  and semantically meaningful features of the
  image/video frames.
• MR are not subjected to same quality control
  resulting in visual distortion in critical
  regions.

58
Summary

• a mask based method for multi-resolution
  image/video representation
• Masks are created based on Edges, Objects and
  Motion (for sequence of images/video)
• The image Quad-tree is pruned using the mask for
  good quality MR representation
• MMR yields superior rate-distortion compared to
  standard color-similarity based MR images/videos

59
References

• Siddhartha Chattopadhyay and Suchendra M.
  Bhandarkar, MMR MASK BASED MULTI-RESOLUTION
  IMAGES AND VIDEOS, Proc. of IEEE International
  Conference on Image Processing, (IEEE ICIP06),
  Oct 2006, Atlanta, pp. 2149-2152.
• Canny, J., "A computational approach to edge
  detection" IEEE ransactions on Pattern Analysis
  and Machine Intelligence, vol. 8,pp. 679--698,
  1986.
• Rowley, H., Baluja, S., and Kanade, T., Neural
  Network-Based Face Detection, IEEE Transactions
  on Pattern Analysis and Machine Intelligence,
  vol. 20, number 1, pp. 23-38, Jan 1998.
• J. Heikkila and O. Silven A real-time system for
  monitoring of cyclists and pedestrians in Second
  IEEE Workshop on Visual Surveillance Fort
  Collins, Colorado (Jun.1999) pp. 74-81.
• http//www.ii.metu.edu.tr/ion528/demo/lectures/6/
  4/index.html
• http//www.doc.ic.ac.uk/nd/surprise_96/journal/vo
  l4/cs11/report.html
• www-staff.it.uts.edu.au/massimo/BackgroundSubtrac
  tionReview-Piccardi.pdf
• http//www.mcs.csuhayward.edu/tebo/Classes/6825/i
  vcnz00.pdf