Applications of

1
Applications of
Shape Similarity
2
ASR Applications in Computer Vision

• Robotics Shape Screening
• (Movie Robot2.avi)
• Straightforward Training Phase
• Recognition of Rough Differences
• Recognition of Differences in Detail
• Recognition of Parts

3
ASR Applications in Computer Vision
Application 2 View Invariant Human Activity
Recognition (Dr. Cen Rao and Mubarak Shah,
School of Electrical Engineering and Computer
Science, University of Central Florida)
4
Application Human Activity Recognition

• Human Action Defined by Trajectory
• Action Recognition by Comparison of Trajectories
• (Movie Trajectories)
• Rao / Shah
• Extraction of Dynamic Instants by Analysis of
  Spatiotemporal Curvature
• Comparison of Dynamic Instants (Sets of
  unconnected points !)
• ASR
• Simplification of Trajectories by Curve Evolution
• Comparison of Trajectories

5
Application Human Activity Recognition
Simplification
Trajectory
6
Activity Recognition Typical Set of Trajectories
7
Trajectories in Tangent Space
8
Trajectory Comparison by ASR Results
9
Recognition of 3D Objects by Projection
Background MPEG 7 uses fixed view
angles Improvement Automatic Detection of Key
Views
10
Automatic Detection of Key Views

• (Pairwise) Comparison of Adjacent Views
• Detects Appearance of Hidden Parts

11
Automatic Detection of Key Views
Result (work in progress)
12
Application ASR
The Database Implementation
13
The Main Application Back to ISS
Task Create Image Database Problem Response
Time Comparison of 2 Shapes 23ms on
Pentium1Ghz ISS contains 15,000
images Response Time about 6 min. Clustering
not possible ASR failed on measuring
dissimilarities !
14
Vantage Objects
Solution Full search on entire database using
a simpler comparison Vantage Objects
(Vleugels / Veltkamp, 2000) provide a simple
comparison of n- dimensional vectors (n
typically lt 100)
15
Vantage Objects
The Idea Compare the query-shape q to a
predefined subset S of the shapes in the
database D The result is an n-dimensional
Vantage Vector V, n S
s1
v1
s2
v2
q
s3
v3

sn
vn
16
Vantage Objects

• - Each shape can be represented by a single
  Vantage Vector
• - The computation of the Vantage Vector calls
  the ASR comparison only n times
• - ISS uses 54 Vantage Objects, reducing the
  comparison time (needed to create the Vantage
  Vector) to lt 1.5s
• - How to compare the query object to the database
  ?

17
Vantage Objects

• - Create the Vantage Vector vi for every shape
  di in the database D
• - Create the Vantage Vector vq for the
  query-shape q
• - compute the (euclidean) distance between vq and
  vi
• - best response is minimum distance
• Note computing the Vantage Vectors for the
  database objects is an offline process !

18
Vantage Objects

• How to define the set S of Vantage Objects ?

19
Vantage Objects

• Algorithm 1 (Vleugels / Veltkamp 2000)
• Predefine the number n of Vantage Objects
• S0
• Iteratively add shapes di ? D\Si-1 to Si-1 such
  that
• Si Si-1 ? di
• and
• ?k1..i-1 ?e(di , sk) maximal. (?e eucl. dist.)
• Stop if i n.

20
Vantage Objects

• Result
• Did not work for ISS.

21
Vantage Objects

• Algorithm 2 (Latecki / Henning / Lakaemper)
• Def.
• A(s1,s2) ASR distance of shapes s1,s2
• q query shape
• Vantage Query determining the result r by
  minimizing e(vq , vi ) vi Vantage Vector to si
• ASR Query determining the result r by
  minimizing A(q,di )
• Vantage Query has certain loss of retrieval
  quality compared to ASR query.
• Define a loss function l to model the extent of
  retrieval performance

22
Vantage Objects

• Given a Database D and a set V of Vantage
  Vectors, the loss of retrieval performance for a
  single query by shape q is given by
• lV,D (q) A(q,r),
• Where r denotes the resulting shape of the
  vantage query to D using q.
• Property
• lV,D (q) is minimal if r is the result of the
  ASR-Query.

23
Vantage Objects

• Now define retrieval error function L(S) of set
• Ss1 ,, sn ? D of Vantage Vectors of Database
  D
• L(S) 1/n ? lS,D\si (si)
• Task
• Find subset S ? D such that L(S) is minimal.

24
Vantage Objects
Algorithm V0 iteratively determine sj
in D\Sj-1 such that Sj Sj-1 ? sj and L(Vj)
minimal. Stop if improvement is low
25
Vantage Objects
Result Worked fine for ISS, though handpicked
objects still performed better.
Handpicked Algorithm 2
L(S)
Number of Vantage Objects
26
Vantage Objects
some of the Vantage Objects used in ISS
27
Vantage Objects and ISS
The Vantage Objects are used in the ASR in the
first (handdrawn) query. The query is compared
to 54 Objects, then a vector comparison is
computed with the whole database. The first
result, also called first guess, is the result
of the vantage vector search. Searching for a
grabbed a shape on the user interface leads to
direct comparison with the ASR, these results are
precomputed, since the query is a known shape !
28
Vantage Objects and ISS
A the handdrawn sketch B the result of the
Vantage search C the result of the exact match