0118 Lab meeting

1
01/18 Lab meeting
Fabio Cuzzolin

• UCLA Vision Lab
• Department of Computer Science
• University of California at Los Angeles

Los Angeles, January 18 2005
2

• past and present

• PhD student, University of Padova, Department of
  Computer
• Science (NAVLAB laboratory) with Ruggero
  Frezza
• Visiting student, ESSRL, Washington University
  in St. Louis
• Visiting student, UCLA, Los Angeles (VisionLab)
• Post-doc in Padova, Control and Systems Theory
  group
• Young researcher, Image and Sound Processing
  Group,
• Politecnico di Milano
• Post-doc, UCLA Vision Lab

3
the research

• object and body tracking
• data association
• gesture and action recognition

research
4
1

• Upper and lower probabilities

5
Past work

• Geometric approach to belief functions
  (ISIPTA01, SMC-C-05)
• Algebra of families of frames (RSS00, ISIPTA01,
  AMAI03)
• Geometry of Dempsters rule (FSKD02, SMC-B-04)
• Geometry of upper probabilities (ISIPTA03,
  SMC-B-05)
• Simplicial complexes of fuzzy sets (IPMU04)

6

• The theory of belief functions

7
Uncertainty descriptions

• A number of theories have been proposed to extend
  or replace classical probability possibilities,
  fuzzy sets, random sets, monotone capacities, etc.

• theory of evidence (A. Dempster, G. Shafer)
• belief functions
• Dempsters rule
• families of frames

8
Motivations
9
Axioms and superadditivity

• probabilities
• additivity if then

10
Example of b.f.
11
Belief functions

• belief functions s 2T -gt0,1

A
B1

• ..where m is a mass function on 2T s.t.

B2
12
Dempsters rule

• b.f. are combined through Dempsters rule

13
Example of combination
14
Bayes vs Dempster

• Belief functions generalize the Bayesian
  formalism as

• 1- discrete probabilities are a special class of
  belief functions

• 2 - Bayes rule is a special case of Dempsters
  rule

• 3 - a multi-domain representation of the evidence
  is contemplated

15
My research
Theory of evidence
16

• Algebra of frames

17
Family of frames

• refining

• Common refinement

• example a function y Î 0,1 is quantized in
  three different ways

1
18
Lattice structure
1F
maximal coarsening Q Å W
Q
W
minimal refinement Q Ä W

• order relation existence of a refining

• F is a locally Birkhoff (semimodular with finite
  length) lattice bounded below

19

• Geometric approach to upper and lower
  probabilisties

20
Belief space

• the space of all the belief functions on a given
  frame

• each subset A?? ? A-th coordinate s(A) in an
  Euclidean space

• it has the shape of a simplex

21
Geometry of Dempsters rule

• constant mass loci

• foci of conditional
• subspaces

• Dempsters rule can be studied in the geometric
  setup too

22
Geometry of upper probs
23
Belief and probabilities

• study of the geometric interplay of belief and
  probability

24
Consistent probabilities

• Each belief function is associated with a set of
  consistent probabilities, forming a simplex in
  the probabilistic subspace

• the vertices of the simplex are the probabilities
  assigning the mass of each focal element of s to
  one of its points

• the center of mass of P(s) coincides with Smets
  pignistic function

25
Possibilities in a geometric setup

• they have the geometry of a simplicial complex

26

• Combinatorial analysis

27
Total belief theorem

• generalization of the total probability theorem

• a-priori constraint

• conditional constraint

28
Existence

• candidate solution linear system n?n
• where the columns of A are the focal elements
  of stot

• problem choosing n columns among m s.t. x has
  positive components

29
Solution graphs

• all the candidate solutions form a graph

• Edges linear transformations

30
New goals...
algebraic analysis
geometric analysis
Theory of evidence
combinatorial analysis
probabilistic analysis
?
31
Approximations

• problem finding an approximation of s

• compositional criterion
• the approximation behaves like s when combined
  through Dempster

• probabilistic and fuzzy approximations

32
Indipendence and conflict

• s1,, sn are not always combinable

• any s1,, sn are combinable ? are defined on
  independent frames

33
Pseudo Gram-Schmidt

• Vector spaces and frames are both semimodular
  lattices -gt admit independence

34
Canonical decomposition

• unique decomposition of s into simple b.f.

• convex geometry can be used to find it

35
Tracking of rigid bodies

• data association of points belonging to a rigid
  body

• rigid motion constraints can be written as
  conditional belief functions ? total belief needed

36
Total belief problem and combinatorics

• general proof, number of solutions, symmetries of
  the graph

• relation with positive linear systems

• homology of solution graphs

• matroidal interpretation

37
2

• Computer vision

38
Vision problems

• HMM and size functions for gesture recognition
  (BMVC97)
• object tracking and pose estimation
  (MTNS98,SPIE99, MTNS00, PAMI04)
• composition of HMMs (ASILOMAR02)
• data association with shape info (CDC02, CDC04,
  PAMI05)
• volumetric action recognition (ICIP04,MMSP04)

39

• Size functions for gesture recognition

40
Size functions for gesture recognition

• Combination of HMMs (for dynamics) and size
  functions (for pose representation)

41
Size functions

• Topological representation of contours

42
Measuring functions

• Functions defined on the contour of the shape of
  interest

real image
family of lines
measuring function
43
Feature vectors

• a family of measuring functions is chosen

• the szfc are computed, and their means form the
  feature vector

44
Hidden Markov models

• Finite-state model of gestures as sequences of a
  small number of poses

45
Four-state HMM

• Gesture dynamics -gt transition matrix A

• Object poses -gt state-output matrix C

46
EM algorithm

• two instances of the same gesture

• feature matrices collection of feature vectors
  along time

A,C
EM

• learning the models parameters through EM

47

• Compositional behavior of Hidden Markov models

48
Composition of HMMs

• Compositional behavior of HMMS the model of the
  action of interest is embedded in the overall
  model

• Example fly gesture in clutter

49
State clustering

• Effect of clustering on HMM topology

• Cluttered model for the two overlapping motions

• Reduced model for the fly gesture extracted
  through clustering

50
Kullback-Leibler comparison

• We used the K-L distance to measure the
  similarity between models extracted from clutter
  and in absence of clutter

51

• Model-free object pose estimation

52
Model-free pose estimation

• Pose estimation inferring the configuration of a
  moving object from one or more image sequences
• Most approaches in the literature are
  model-based they assume some knowledge about the
  nature of the body (articulated, deformable, etc)
  and some sort of model

T0
tT
53
Model-free scenario

• Scenario the configuration of an uknown object
  is desired, given no a-priori information about
  the nature itself of the body
• The only info available is carried by the images
• We need to build a map from image measurements to
  body poses
• This can be done by using a learning technique,
  based on training data

54
Evidential model
1
approximate feature spaces
3
feature-pose maps (refinings)
2
training set of sample poses

• The evidential model is built during the
  training stage, when the feature-pose maps are
  learned

55
Feature extraction
2
1
3
1

• From the blurred image the region with color
  similar to the region of interest is selected,
  and the bounding box is detected.

3
2
56
Estimates from the combined model

• Ground truth versus estimates...

• ... for two components of the pose

57

• JPDA with shape information for data association

58
JPDA with shape info

• JPDA model independent targets

• Shape model rigid links

• Dempsters fusion

• robustness clutter does not meet shape
  constraints
• occlusions occluded targets can be estimated

59
Triangle simulation

• the clutter affects only the standard JPDA
  estimates

60
Body tracking

• Application tracking of feature points on a
  moving human body

61

• Volumetric action recognition

62
Volumetric action recognition

• problem recognizing the action performed by a
  person viewed by a number of cameras
• step 1 modeling the dynamics of the motion
• step 2 extracting image features

• 2D approaches features are extracted from
  single views -gt viewpoint dependence
• volumetric approach features are extracted from
  a volumetric reconstruction of the moving body

63
Multiple sequences

• synchronized views from different cameras,
  chromakeying

64
Volumetric intersection

• more views -gt more details

• silhouette extraction of the moving object from
  all views
• 3D object shape reconstruction through
  intersection of occlusion cones

65
3D feature extraction

• locations of torso, arms, and legs of the moving
  person

• k-means clustering to separate bodyparts

66
Feature matrices
X
Y
TORSO COORDINATES
Z
ABDOMEN COORDINATES
RIGHT LEG COORDINATES
LEFT LEG COORDINATES

• two instances of the action walking

67
Modeling and recognition
HMM 1
HMM 2

HMM n

• model of the walking action

• classification each new feature matrix is fed
  to all the learnt models, generating a set of
  likelihoods

68
3

• Combinatorics

69
Independence on lattices

• three distinct independence relations

• modularity ? equivalent formulations

70
Scheme of the proof
71
4

• Conclusions

72
from real to abstract

• the solution of real problems stimulates new
  theoretical issues

73
concluding

• the ToE comes from a strong critics to the
  Bayesian framework

• useful for sensor fusion problems under
  incomplete information

• real problem solutions stimulate the extension
  of the formalism

• complex objects ? mathematically rich

• Young theory ? need completion

74
In the near future..

• search for a metric on the space of dynamical
  systems stochastic models

• sistematic description of the geometric approach
  to non-additive measures

• understand the intricate relations between
  probability and combinatorics