Real Time Pattern Detection

1
Real Time Pattern Detection
Yacov Hel-Or The Interdisciplinary Center
Hagit Hel-Or Haifa University
2
Pattern Detection

• A given pattern is sought in an image.
• The pattern may appear at any location in the
  image.
• The pattern may be subject to any
  transformation
• (within a given transformation group).

3
Example

• Face detection in images

4
Why is it Expensive? The search in Spatial Domain
Searching for faces in a 1000x1000 image, is
applied 1e6 times, for each pixel location.
A very expensive search problem
5
Why is it difficult? The Search in
Transformation Domain

• A pattern under transformations draws a very
  complex manifold in pattern space
• In a very high dimensional space.
• Non convex.
• Non regular (two similarly perceived patterns may
  be distant in pattern space).

P
Q
?(Q,P)
T(?)P
6
A rotation manifold of a pattern drawn in
pattern-space The manifold was projected into
its three most significant components.
7
Suggested Approach

• Reduce complexity of search using 2 complementary
• processes
• Reduce search in Spatial Domain.
• Reduce search in Transformation Domain.

Both processes are based on a Rejection Scheme.




8
Efficient Search in the Spatial Domain

9
The Euclidean Distance
10
Complexity (2D case)
11
Norm Distance in Sub-space

• Representing an image window and the pattern as
  vectors in Rkxk

dE(p,q) p-q2 - 2

• If p and q were projected onto a vector u, it
  follows
• from the Cauchy-Schwarz Inequality

dE(p,q) ? u-2 dE(pTu, qTu)
q
p
u
12
Distance Measure in Sub-space (Cont.)

• If q and p were projected onto a set of vectors
  U

It can be shown that
13
How can we Expedite the Distance Calculations?

• Two necessary requirements
• Choose projecting kernels U having high
  probability to be parallel to the vector p-q.
• Choose projecting kernels that are fast to apply.

14
Projecting Kernels Walsh-Hadamard

• Following the above requirement we use the kxk
• Walsh-Hadamard basis vectors
• Each window in a natural image is closely spanned
  by the first few basis vectors.
• Can be applied very fast in a recursive manner.

15
The Walsh-Hadamard Kernels
16
Walsh-Hadamard v.s. Standard Basis
The lower bound for distance value in v.s.
number of standard basis projections, Averaged
over 100 pattern-image pairs of size 256x256 .
The lower bound for distance value in v.s.
number of Walsh-Hadamard projections, Averaged
over 100 pattern-image pairs of size 256x256 .
17
The Walsh-Hadamard Tree (1D case)

-

-

-
-


- -
- -
- -

-
-
-
-




- - - -
- - - -
- - - -
- - - -
- - - -
- - - -
- - - -

WH-tree for 1D pattern of size 8
18
The Walsh-Hadamard Tree - Example
15 6 10 8 8 5 10 1

-

21 16 18 16 13 15 11
9 -4 2 0 3 -5 9
-

-
-


- -
- -
- -
39 32 31 31 24
11 -4 5 -5 12

3 0 5 1 2
7 -4 -1 5 6
19
Properties

• Descending from a node to its child requires one
  addition operation per pixel.

• A projection of the entire image onto one basis
  vector is performed in a top-down traversal.

• A projection of a particular window in the image
  onto one basis vector is performed in a bottom-up
  traversal.

• All operations are performed in integers.

-

-

-
-


- -
- -
- -

20
Complexity (1D)

• Projecting all windows in the image onto a basis
  vector requires log k additions per pixel.
• Projecting all windows in the image onto lltk
  basis vectors requires m additions per pixel,
  where m is the number of nodes preceding the l
  leaf.
• Projecting all windows in the image onto k basis
  vectors requires 2k additions per pixel.
• Projecting a single window onto a single
• basis vector requires k-1 additions.

-

-

-
-


- -
- -
- -

21
Walsh-Hadamard Tree (2D)

• For the 2D case, the projection is performed in a
  similar manner where the tree depth is 2log k
• The complexity is calculated accordingly.

-

-



-
-

- -
- -
- -
Construction tree for 2x2 basis
22
Pattern Matching algorithm

• Iteratively apply Walsh-Hadamard vectors to each
  window wi in the image.
• At each iteration and for each wi calculate a
  lower-bound Lbi for p-wi2 .
• If the lower-bound Lbi is greater than a
  pre-defined threshold, reject the window wi and
  ignore it in further projections.

23
Pattern Matching algorithm - Complexity
All windows are projected onto the first kernel
2logk
ops/pixel Only a few windows are further
projected using 2k operations per active window
? ops/pixel
Total 2logk ? ops/pixel
24
Example
Sought Pattern
Initial Image 65536 candidates
25
After the 1st projection 563 candidates
26
After the 2nd projection 16 candidates
27
After the 3rd projection 1 candidate
28
Percentage of windows remaining following each
projection, averaged over 100 pattern-image
pairs. Image size 256x256, pattern size
16x16.
29
Accumulated number of additions after each
projection averaged over 100 pattern-image
pairs. Image size 256x256, pattern size
16x16. Average Number of operations per pixel
8.0154
30
Example with Noise
Original
Noise Level 40
Detected patterns.
Number of projections required to find all
patterns, as a function of noise level.
(Threshold is set to minimum).
31
Percentage of windows remaining following each
projection, at various noise levels. Image size
256x256, pattern size 16x16.
32
DC-invariant Pattern Matching
Illumination gradient added
Original
Detected patterns.
Five projections are required to find all 10
patterns (Threshold is set to minimum).
33
Complexity (2D case)
34
Advantages

• Walsh-Hadamard per window can be applied very
  fast.
• Projections are performed with additions/subtracti
  ons only (no multiplications).
• Integer operations.
• Fast rejection of windows.
• Possible to perform pattern matching at video
  rate.
• DC invariant pattern matching.
• Extensions
• Other norms.
• Multi size pattern matching.

35
Limitations

• 2n2 log k memory size.
• Pattern size must be 2m .
• Limited to normed distance metrics.

36
Efficient Search in the Transformation Domain
37
Transformation Manifold
A pattern P can be represented as a point in ?kxk
T(?)P is a transformation T(?) applied to
pattern P.
T(?)P for all ? forms an orbit in ?kxk
T(?1)P
T(?0)P
T(?)P
T(?2)P
P
38
Fast Search in Group Orbit

• Assume d(Q,P) is a distance metric.
• We would like to find

?(Q,P)min? d(Q, T(?)P)
P
Q
T(?)P
?(Q,P)
39
Fast Search in Group Orbit (Cont.)

• In the general case ?(Q,P) is not a metric.

P
Q
R

• Observation if d(Q,P) d(T(?)Q, T(?)P)

?(Q,P) is a metric
40
Fast Search in Group Orbit (Cont.)

• The metric property of ?(Q,P) implies triangular
  inequality on
• the distances.

Q
P
S
41
Orbit Decomposition

• In practice T(?) is sampled into T (?i)T?(i) ,
  i1,2,

• We can divide T?(i)P into two sub-orbits
• T2?(i)P and T2?(i)P where P T?(1) P

P
P
??(Q,P)
Q
42
Orbit Decomposition (Cont.)
Q
43
Orbit Decomposition (Cont.)
Q
Since ?2? is a metric and ?2?(P,P) can be
calculated in advance we may save calculations
using the triangle inequality constraint.
44
Orbit Decomposition (Cont.)

• The sub-group subdivision can be applied
  recursively.

45
Fast Search - Example
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
Fast Search in Group Orbit Conclusions

• Observation 1 Orbit distance is a metric when
  the point distance is transformation invariant.
• Observation 2 Fast search in orbit distance
  space can be applied using recursive orbit
  decomposition.
• Distant patterns are rejected fast.
• Important Can be applied to any metric distance
  d(Q,P).

54
Conclusion

• Pattern Detection using 2 complementary
  processes
• Reduce search in Transformation Domain.
• Reduce search in Spatial Domain.
• Processes are based on rejection schemes, and are
  
• restricted to a specific domain.
• The two processes can be combined into a single,
  highly
• efficient, search process.

--- END ---