Recognition Of Textual Signs

1
Recognition Of Textual Signs

• Final Project for Probabilistic Graphics Models
• Submitted by Ezra Hoch, Golan Pundak, Yonatan
  Amit

2
Introduction

• Input 600x480 image
• Output 6X6 pixels region labeling (text /no
  text)
• Motivation narrow OCR search area for urban /
  road signs.
• Can be used by various Vehicle Vision
  applications.

3
Project Overview
Classifier
Features extractor
Loopy BP
4
Contribution of Graphical Model
This graph portraits the results of the naïve
classification vs. the results after graphic
inference. It is clear that using inference
improves the results. We also note the
importance of quality classification to our
results
Classifier
After inference
The of agreement between hand-label and graphic
inference
The of agreement between hand-label and
classifier
5
Initial Labeling Classification

• 250 pictures were taken in jerusalem
• 20 of pictures were hand labeled for three
  categories
• Text, Sign Background, General Scene
• 10 were used for training, the rest for testing.
• Feature Extraction
• Images were divided to blocks of 6x6
• Each block was considered with a 10x10
  surrounding
• 19 Features were selected, attempting to
  apprehend spatial information relevant to textual
  areas
• Using a Back-Propagation network as classifier
• Three level neural network

Classifaier output
Hand labeling
Original image
6
Learning the Potential Functions

• Our Model
• 80x100 nodes, Grid topology
• Assume movement invariance
• Horizontal VS. Vertical Psi.
• Sensitive to changes of scale and rotations.
  (Improvement requires better features)
• Our Input
• 10 Images used as input
• Local PSIs were fixed by classifier
• The Algorithm
• Using IPF as learning algorithm
• Empirical Distribution calculated from
  hand-labeled instances.
• Current Estimation calculated by approximate
  inference, namely Loopy BP
• Output
• Horizontal and Vertical Psi

7
Model Variations

• Using uniform Psi for Vertical Neighbours and
  Horizontal Neighbours.
• Results were poorer than different Psi
• A second regional grid was tested
• Creating a decent classifier for regional nodes
  has proven a difficult task
• Has shown little improvement if any
• Was not used in latest results

Regional Nodes
Regional Classifier
Local Nodes
Classifier
Features extractor
Features extractor
8
Second Layer Contribution
We attempted to integrate a 2nd layer of regional
nodes each looking at a collection of 10x10
local nodes. Uniform Local Psi was given to the
2nd layer. Intuitively selected Psi were used
between the first and second layer (no learning
was performed). On some cases this yields better
results.
Original image
Single layer output
Classifier result
2nd layer output
Arched text was blurred by single layer and
preserved with 2 layers
9
Results
10
More Results
11
Conclusions

• Using Graphical Inference allows seamless
  combination of local classification and global
  properties
• Variations of the Graphical Model improves our
  ability to perceive the underling world
• However, theres no substitute for a proper local
  classifier and feature-extractor

Future Work

• Robust feature extraction
• Complete second level regional nodes
• Improve hand labeling, enabling more accurate
  training