PatchBased Image Classification Using Image Epitomes

1
Patch-Based Image Classification Using Image
Epitomes

• David Andrzejewski
• Computer Sciences 766
• Fall 2005

2
Problem Statement

• Given
• Positive and negative example images for a
  certain classification (contains face, is
  outdoors, etc)
• Do
• Develop classifier capable of classifying new
  images as positive or negative

3
Image Epitomes
Image epitome
Input image
A set of image patches

• Consists of patches and mappings
• Patches and mappings are learned with EM
• Applications in vision (de-noising,segmentation,ot
  hers)

www.research.microsoft.com/jojic/epitome.htm
4
Image Reconstruction

• Original image can then be reconstructed by
  mosaicing epitome patches

www.research.microsoft.com/jojic/epitome.htm
5
Recognition / Detection / Classification
The smiling point
Epitome of 295 face images
Images with the highest total posterior at the
smiling point
Images with the lowest total posterior at the
smiling point
www.research.microsoft.com/jojic/epitome.htm
6
Approach

• Construct collage of positive and negative
  examples
• Learn the image epitome of the training collage
• Find epitome patches that are preferentially
  mapped into the positive example images in the
  collage
• Calculate P(patch(i)pos/neg) for these patches
  (also use psuedo-counts)
• Use these patches to classify new images by
  calculating odds ratio

7
Preliminary Results
Training Collage
Epitome
Negative Test Images
Positive Test Images
8
Problems with Approach

• Difficult to incorporate new examples
• Would need to add to collage and re-learn epitome
  (is there a better way?)
• Bag of words ? Spatial information discarded
• Not model-based
• Pose/Illumination/Scale-variant
• Only way to handle variation is to include
  training examples for various conditions

9
Potential Modifications

• Cluster training images
• Ex Training images w/ low vs high illumination
• Discriminative patches may map exclusively to
  one subset of positive images ? take this into
  account
• Change winner take all for P calculations
• Consider relative probabilities of 'near
  matches'
• Account for multiple mappings somehow

10
References

• V. Cheung, B. J. Frey, and N. Jojic, Video
  epitomes, Proc. IEEE Conf. Computer Vision and
  Pattern Recognition, 2005
• N. Jojic, B. J. Frey, and A. Kannan, Epitomic
  analysis of appearance and shape, Proc. 9th Int.
  Conf. Computer Vision, 2003
• R. Fergus, P. Perona, A. Zisserman, Object Class
  Recognition by Unsupervised Scale-Invariant
  Learning, Proc. of the IEEE Conf on Computer
  Vision and Pattern Recognition, 2003
• Testing images from Google Images and Flickr.com
  (thanks caspa7!)