Topology in Manifold Learning

1
Topology in Manifold Learning

• Jonathan Huang
• Presented at misc-read, 11.22.06

2
Bibliography

• Simultaneous Inference of View and Body Pose
  Using Torus Manifolds Chan-Su Lee and Ahmed
  Elgammal The 18th International Conference on
  Pattern Recognition (ICPR), Hong Kong, August
  21-24, 2006
• Finding the Homology of Submanifolds with High
  Confidence from Random Samples. P. Niyogi, S.
  Smale, and S. Weinberger to appear, Discrete and
  Computational Geometry, 2006.
• On the Local Behavior of Spaces of Natural
  Images G. Carlsson, T. Ishkhanov, V. de Silva,
  and A. Zomorodian, /preprint/, May 31, 2006.
• Computing Persistent Homology A. Zomorodian and
  G. Carlsson, Discrete and Computational Geometry,
  33 (2), pp. 247-274, 2005.

3
Outline

• The Role of Topology in Manifold Learning
• Constrained Topology Manifold Learning
• Topology Basics
• Learning a Topology from Noisy Data
• Statistical Approach
• Multi-scale Approach

4
The ISOMAP Algorithm

• The ISOMAP algorithm
• Compute pairwise distances for some point cloud
• Choose an embedding dimension, d (d2 in this
  example)
• Run the following code in matlab
• gtgt options.dims 2
• gtgt Y, R, E Isomap(DistanceMatrix, options)
  

ISOMAP
5
Selecting the Dimensionality

• Problem How do we choose embedding dimension?
• Several solutions (maybe some non-NIPS solutions
  too)
• Brand (NIPS 2003)
• Kegl (NIPS 2003)
• Levina and Bickel (NIPS 2005)
• Raginsky and Lazebnik (NIPS 2006)

Dimensionality Estimation
d2
6
Why none of these are actually solutions
Dimensionality Estimation
d2
ISOMAP
(samples from a sphere)
7
Moral

• Manifold Learning is hard if you dont take
  topology into account!

8
Outline

• The Role of Topology in Manifold Learning
• Constrained Topology Manifold Learning
• Topology Basics
• Learning a Topology from Noisy Data
• Statistical Approaches
• Multi-scale Approaches

9
Manifold Learning with Known Topology

• Images of a periodic gait from varying viewpoints
• What is the intrinsic topology of this set of
  images?

Body Pose
View angle (0-330)
10
Learning a Mapping from a Torus

• The product of two periodic spaces is a torus!
• Use kernel methods to learn a map to/from a Torus

11
Some Results
12
Outline

• The Role of Topology in Manifold Learning
• Constrained Topology Manifold Learning
• Topology Basics
• Learning a Topology from Noisy Data
• Statistical Approaches
• Multiresolution Approaches

13
What is Topology?

• Popular answer Its the branch of math that
  cant tell the difference between a coffee cup
  and a donut

14
What is Topology?

• Topology cares about how a space is connected
• It does not care about distances
• We will define a very general class of
  topological spaces the simplicial complexes

15
Simplex

• An n-simplex is the convex hull of (n1)
  (independent) points

0-simplex
1-simplex
2-simplex
16
Simplicial Complex

• A simplicial complex is a finite collection of
  simplices S such that
• Any face of a simplex in S is also in S
• The intersection of two simplices in S is either
  empty or a face for both simplices
• The dimension of S is the maximum dimension over
  all simplices in S

17
Boundary Maps

• Graphs are one-dimensional simplicial complexes
• Define a boundary matrix M1 with rows
  corresponding to vertices and columns
  corresponding to edges
• M1(v,e) 0 if vertex v is not part of edge e
• M1(v,e) -1 if vertex v is the first vertex of
  edge e
• M1(v,e) 1 if vertex v is the second vertex of
  edge e

a
ab
ac
b
c
bc
18
Boundary Maps

• Example
• The boundary of an edge is its two vertices

19
Boundary Maps

• Example
• The boundary of the loop is empty

20
Boundary Maps

• In general, the dim(Nullspace(M)) is the number
  of different loops in the graph
• And (vertices)-Rank(M) is the number of
  connected components

21
Betti Numbers

• For a simplicial complex S, we can define a
  boundary matrix Mk at each dimension k of S
• Define the kth betti number to be
• ?k Dim(Nullspace(Mk-1))-Dim(Complement of
  column space(Mk))

22
Betti Numbers

• For an object in 3d space
• ?0 is the number of connected components
• ?1 is the number of tunnels or handles
• ?2 is the number of voids

Point
Circle
Torus
23
Outline

• The Role of Topology in Manifold Learning
• Constrained Topology Manifold Learning
• Topology Basics
• Learning a Topology from Noisy Data
• Statistical Approaches
• Multi-scale Approaches

24
Problem Formulation

• Given x1,x2,,xn, i.i.d samples from a manifold.
  What are the betti numbers of the manifold?

25
An Well Known Algorithm

• Two Steps
• First put an ?-ball around each point
• Compute the betti numbers of the union of these
  balls using the Nerve Complex

26
The Nerve Complex

• Given a collection of balls, U1,U2, in Euclidean
  space, what is the topology of their union?
• Construct the Nerve Complex
• For each ball, add a vertex
• Add a k-simplex whenever k1 balls have nonempty
  intersection

27
Nerve Complex

• The Nerve Lemma states that the original space ?
  and the Nerve Complex have the same Betti
  numbers!
• Example

A B
B
A
A
B
A C
B C
C
C
28
A PAC-bound

• With high probability and enough samples, we can
  recover the true Betti numbers!
• The number of samples depends on the volume of
  the manifold and its condition number (how close
  it gets to itself)
• (Niyogi, Smale, Weinberger, 2006)
• (but how does one choose ??)

29
Outline

• The Role of Topology in Manifold Learning
• Constrained Topology Manifold Learning
• Topology Basics
• Learning a Topology from Noisy Data
• Statistical Approaches
• Multi-scale Approaches

30
Another Example
Spiral or Torus???
31
Topological Persistence

• Topological Persistence looks at topology from
  all scales at once
• Surprisingly, its not much harder to compute the
  betti numbers at every scale!

32
Filtered Complex

• A Filtered Complex is an increasing sequence of
  simplicial complexes (Ct ? Ct1)
• t is called the filtration index

t0
t1
t3
t2
33
Barcodes - Example
34
Barcodes

• Barcodes represent betti numbers as a function of
  filtration index
• Intuitively, Barcodes measure the lifetime of
  topological features
• The Persistence Algorithm provides a way to
  compute barcodes efficiently

35
Persistence Algorithm

• Empirically, the algorithm works in linear time
• Worst case complexity bound O(m3)
• Where m is the of simplices
• Pros
• Persistence distinguishes local features from
  global features
• Applies to learning manifold topology from noisy
  data
• Cons
• No real probabilistic semantics

36
Conclusion

• Learning a topology is not hopeless. So
• The next time you decide to learn a manifold,
  take a moment to contemplate the underlying
  topology!

37
Thank You!