Learning the topology of a data set

1
Learning the topology of a data set
PASCAL BOOTCAMP, Vilanova, July 2007
Michaël Aupetit Researcher Engineer
(CEA) Pierre Gaillard Ph.D. Student Gerard
Govaert Professor (University of Technology of
Compiegne)
2
Introduction
Given a set of M data in RD, the estimation of
the density allow solving various problems
classification, clustering, regression
3
A question without answer
The generative models cannot aswer this
question Which is the  shape  of this data set
?
4
An subjective answer
The expected answer is
1 point and 1 curve not connected to each other
The problem what is the topology of the
principal manifolds
5
Why learning topology (semi)-supervised
applications
Estimate the complexity of the classification
task Lallich02, Aupetit05Neurocomputing Add a
topological a priori to design a
classifier Belkin05Nips Add topological
features to statistical features Classify
through the connected components or the intrinsic
dimension. Belkin
6
Why learning topology unsupervised applications
Clusters defined by the connected
components Data exploration (e.g. shortest
path) Robotic (Optimal path, inverse
kinematic)

7
Generative manifold learning
Gaussian Mixture
MPPCA Bishop
GTM Bishop
Revisited Principal Curves Hastie,Stuetzle
Problems fixed or incomplete topology
8
Computational Topology

• All the previous work about topology learning has
  been grounded on the result of
• Edelsbrunner and Shah (1997) which proved that
  given a manifold and a set of N
• prototypes nearby M, it exists a subgraph of the
  Delaunay graph of the prototypes
• which has the same topology as M

M1
M2
more exactly a subcomplex of the Delaunay
complex
9
Computational Topology

• All the previous work about topology learning has
  been grounded on the result of
• Edelsbrunner and Shah (1997) which proved that
  given a manifold and a set of N
• prototypes nearby M, it exists a subgraph of the
  Delaunay graph of the prototypes
• which has the same topology as M

M1
M2
10
Computational Topology

• All the previous work about topology learning has
  been grounded on the result of
• Edelsbrunner and Shah (1997) which proved that
  given a manifold and a set of N
• prototypes nearby M, it exists a subgraph of the
  Delaunay graph of the prototypes
• which has the same topology as M

M1
M2
11
Computational Topology

• All the previous work about topology learning has
  been grounded on the result of
• Edelsbrunner and Shah (1997) which proved that
  given a manifold and a set of N
• prototypes nearby M, it exists a subgraph of the
  Delaunay graph of the prototypes
• which has the same topology as M

M1
M2
12
Computational Topology

• All the previous work about topology learning has
  been grounded on the result of
• Edelsbrunner and Shah (1997) which proved that
  given a manifold and a set of N
• prototypes nearby M, it exists a subgraph of the
  Delaunay graph of the prototypes
• which has the same topology as M

M1
M2
13
Computational Topology

• All the previous work about topology learning has
  been grounded on the result of
• Edelsbrunner and Shah (1997) which proved that
  given a manifold and a set of N
• prototypes nearby M, it exists a subgraph of the
  Delaunay graph of the prototypes
• which has the same topology as M

Extractible topology O(DN3)
M1
M2
14
Application known manifold
Topology of molecules Edelsbrunner1994
15
Approximation manifold known throught a data set
16
Topology Representing Network

• Topology Representing Network Martinetz,
  Schulten 1994

Connect the 1st and 2nd NN of each data
17
Topology Representing Network

• Topology Representing Network Martinetz,
  Schulten 1994

2nd
1er
18
Topology Representing Network

• Topology Representing Network Martinetz,
  Schulten 1994

2nd
1er
19
Topology Representing Network

• Topology Representing Network Martinetz,
  Schulten 1994

2nd
1er
20
Topology Representing Network

• Topology Representing Network Martinetz,
  Schulten 1994

21
Topology Representing Network

• Topology Representing Network Martinetz,
  Schulten 1994

22
Topology Representing Network
23
Topology Representing Network

• Good points
• 1- O(DNM)
• 2- If there are enough prototypes and if they
  are well located then resulting graph is  good 
  in practice.

Some drawbacks from the machine learning point of
view
24
Topology Representing Network some drawbacks

• Noise sensitivity

25
Topology Representing Network some drawbacks

• Not self-consistent Hastie

26
Topology Representing Network some drawbacks

• No quality measure
• How to measure the quality of the TRN if D gt3 ?
• How to compare two models ?

For all these reasons, we propose a generative
model
27
General assumptions on data generation
28
General assumptions on data generation
29
General assumptions on data generation
30
General assumptions on data generation
The goal is to learn from the observed data, the
principal manifolds such that their topological
features can be extracted
31
3 assumptions1 generative model
The manifold is close to the DG of some
prototypes
32
3 assumptions1 generative model
we associate to each component a weighted
uniform distribution
The manifold is close to the DG of some
prototypes
33
3 assumptions1 generative model
we associate to each component a weighted
uniform distribution
The manifold is close to the DG of some
prototypes
we convolve the components by an isotropic
Gaussian noise
34
A Gaussian-point and a Gaussian-segment
How to define a generative model based on points
and segments ?
A
B
A
can be expressed in terms of  erf 
35
Hola !
36
Proposed approach 3 steps
1. Initialization
and then building of the Delaunay
Graph Initialize the generative model
(equiprobability of the components)
Location of the prototypes with a  classical 
isotropic GM
37
Number of prototypes
min BIC - Likelihood Complexity of the model
38
Proposed approach 3 steps
2. Learning
update the variance of the Gaussian noise, the
weights of the components, and the location of
the prototypes with the EM algorithm in
order to maximize the Likelihood of the
model w.r.t the N observed data
39
EM updates
40
EM updates
41
Proposed approach 3 steps
3. After the learning
Some components have a (quasi-) nul probability
(weights) They do not explain the data and can
be prunned from the initial graph
42
Threshold setting
 Cattell Scree Test 
43
Toy Experiment
Seuillage sur le nombre de witness
44
Toy Experiment
45
Toy Experiment
46
Other applications
?
O

47
Comments

• There is  no free lunch 
• Time Complexity O(DN3) (initial Delaunay graph)
• Slow convergence (EM)
• Local optima

48
Key Points

• Statistical learning of the topology of a data
  set
• Assumption
• Initial Delaunay graph is rich enough to contain
  a sub-graph having the same topology as the
  principal manifolds
• Based on a statistical criterion (the likelihood)
  available in any dimension
•  Generalized  Gaussian Mixture
• Can be seen as a generalization of the  Gaussian
  mixture  (no edges)
• Can be seen as a finite mixture (number of
   Gaussian-segment ) of an infinite mixture
  (Gaussian-segment)
• This preliminary work is an attempt to bridge the
  gap between Statistical Learning Theory and
  Computational Topology

49
Open questions

• Validity of the assumption
•  good  penalized-likelihood   good 
  topology
• Theorem of universal approximation of manifold ?

50
Related works

• Publications NIPS 2005 (unsupervised) and ESANN
  2007 (supervised analysis of the iris and oil
  flow data sets)
• Workshop submission at NIPS on this topic
• in collaboration with F. Chazal (INRIA Futurs) ,
  D. Cohen-Steiner
• (INRIA Sophia), S. Canu and G.Gasso (INSA Rouen)

51
Thanks
52
Equations
53
Topology
intrinsic dimension
Homeomorphism topological equivalence