Distinguishing Mathematics Notation from English Text using Computational Geometry

1
Distinguishing Mathematics Notation from English
Text using Computational Geometry

• D. Drake, H.S. Baird
• Department of Computer Science and Engineering
• Lehigh University

2
Project Goal

• Differentiate isolated math and English textlines

English text or Math?
English text or Math?
How can Optical Character Recognition (OCR)
systems make this distinction?
3
Applications of Textline Classification

• Commercial OCR systems far better on text than
  on math
• e.g. OCR systems garble math
• Textline classification allows
• Processing of text/math differently
• Hand off math to special purpose recognizers
• Users can see Math textlines as image
• No OCR errors

4
Creativity New Ideas

• Current approach
• Symbol recognition

New approach Spatial analysis
5
Voronoi Diagrams
Partition of the plane into regions such that the
points in a region are closer to that point than
any other
6
Preprocessing Overview
Input Image
Sample points on boundary of black connected
components
Compute Voronoi Diagram
Compute Area Voronoi Diagram
Compute Neighbor Graph
Input to Classifier decides whether textline is
math or text
7
Preprocessing Overview
8
Features of Neighbor Graph used for Classification

• Idea detect complex spatial arrangements
  typically found in math

• Binary Edge Features
• Shadowing
• Angle (wrt horizontal)
• Area ratio
• Diameter ratio

• Binary Node Features
• Aspect ratio
• Diameter/area ratio
• Fanning

9
Classifiers

• Three classifiers were constructed
• Quadratic Bayesian Node classifier
• Quadratic Bayesian Edge classifier
• Thresholding Textline classifier
• Classifiers trained on a training image set
• Given input feature vector and correct
  classification
• Classifiers then tested on test image set
• Classification based on input feature vector and
  training
• Textline classifier used classifications from
  edge and node classifiers
• Technique of combining classifiers
• Classification accuracy improves due to
  uncorrelated errors in the component classifiers

10
Examples of Correctly Classified Textlines
11
Results

• Experiment performed on synthetically-generated
  images and scanned books

Testing Set Textline Confusion Matrix
Textline Error Rates
Example misclassified textlines
12
Future Work/Conclusions

• Future Work
• Inline math
• Detection of textlines in full document images
• Conclusions
• Spatial analysis has many advantages over symbol
  recognizers for distinguishing textlines
• Automatically trainable
• Needs no prior knowledge of font, font size, or
  spacing
• Easily extendable to other languages

13
Acknowledgements

• Mentor
• Henry S. Baird, Lehigh University
• Code to compute neighbor graphs
• Koichi Kise, Osaka Prefecture University