Features for handwriting recognition

1
Features for handwriting recognition
2
The challenge
Rappt JD 10 Feb no 175, om machtiging om af
3
Short processing pipeline
Learning
machtiging
Feature extraction
Classification
82,34,66,
machtiging
0.12
4
Processing pipeline
Preprocessing
Feature extraction
Classification
5
Input image types

• Color
• Grayscale
• Binary

6
Preprocessing

• Goal enhance the foreground while reducing other
  visual symptoms (stains, noise, pictures, ...)
• Methods
• Contrast stretching
• Highpass filtering
• Despeckling
• Change color representation (RGB, HSV, grayscale,
  black/white, )
• Remove selected connected components (?)

7
Connected components
8
Processing pipeline
Preprocessing
Segmentation
Feature extraction
Classification
9
Object of classification

• Sentences
• Words
• Characters

• (use grammar)
• (use dictionary)
• (use alphabet)

10
Object representations

• Image
• Unordered vectors (in a coco)
• Contour vectors
• On-line vectors
• Skeleton image
• Skeleton vectors

I(x, y)
(x, y)i
(x, y)k
(x, y)k
I(x, y)
(x, y)k
11
A full processing pipeline
Preprocessing
Segmentation
Normalization
Feature extraction
Classification
12
Invariance

• Luminance / contrast
• Position
• Size
• Rotation
• Shear
• Writer style
• Ink thickness

13
Invariance by normalization
Contrast stretching

• Luminance / contrast
• Position
• Size
• Rotation
• Shear
• Writer style
• Ink thickness

Center on center of gravity
Scale to standard size
14
Invariance by trying many deformations

• Luminance / contrast
• Position
• Size
• Rotation
• Shear
• Writer style
• Ink thickness

Try different scale factors
Try different rotations
Try different deformations
and use the best recognition result
15
Invariance by using invariant features

• Luminance / contrast
• Position
• Size
• Rotation
• Shear
• Writer style
• Ink thickness

Zernike invariant moments
16
A full processing pipeline
Preprocessing
Segmentation
Normalization
Feature extraction
Classification
82,34,66,
17
Feature ROI types

• Whole object
• Zones
• Windowing

18
Whole object (wholistic)
19
Zones
20
Windowing
21
Feature types

• Image itself
• Statistical
• Structural
• Abstract
• Image (off-line) features (120)
• Contour / on-line features (21 28)

22
Feature 1 3

• Connected component images
• Scaled image
• Distance transform

(on whiteboard)
23
Feature 4 density histogram
24
Feature 5 radon transform
25
Feature 6 run count pattern
3
6
26
Feature 7 run length pattern
avg
stdev
27
Feature 8 Autocorrelation
28
Feature 9 Polar zones
29
Feature 10 radial zones (tip!)
30
Feature 11 zone histograms
31
Feature 12 Hinge
(By Marius Bulacu)
32
Feature 13 Fraglets
33
Feature 14 J.C. Simon (1/2)
Singulariteiten
Regelmatigheden
34
Feature 14 J.C. Simon (2/2)
"million" gt convexconcave3(northconcave
) (northLOOP)concave(northLOOP)
concavenorth concaveHOLE
2(convexconcave)
(J.-C. Simon, 1989)
35
Feature 15 Structure of background (1/3)
36
Feature 15 Structure of background (2/3)
37
Feature 15 Structure of background (3/3)
38
Feature 16 Structure of foreground background
39
Feature 17 Fourier transform (1/2)
From http//ccp.uchicago.edu/dcbradle/pages/5.23
.06.html
40
Feature 17 Fourier transform (2/2)
Fig. 1 and 3 from http//www.csse.uwa.edu.au/won
gt/matlab.html
Fig. 2 from http//www.chemicool.com/definition/f
ourier_transform.html
41
Feature 18 Wavelet transform
From http//www.regonaudio.com/Audio20Measuremen
t20via20Wavelets.html
42
Feature 19 Hu invariant moments

• Derived from moments
• Moments describe the image distribution with
  respect to its axes
• Works on (x, y) vectors

• Invariant for scale, position and rotation

area of the object
center of mass
Slide from http//www.cedar.buffalo.edu/govind/C
SE717/lectures/CSE717_3.ppt
43
Feature 20 Zernike moments

• From Trier, O. D., Jain, A. K., and Taxt, T.
  (1996). Feature extraction methods for character
  recognition - a survey. Pattern
  Recognition,29641662.

44
Feature 21 28 Contour features

• (cos, sin) of running angle
• (cos, sin) of running angular difference
• Angular difference
• Fourier transform
• Ink density (horizontal or vertical)
• Radon transform (ink density, computed radially
  from the c.o.g.)
• Angular histogram
• Curvature scale space (?)

45
Feature 28 Curvature scale space
iteration
pos
From http//www.christine.oppe.info/blog/category
/formen-und-farben/formenvergleich/
46
End