Seeing Through Water

1
Seeing Through Water

• Alexei Efros, Volkan Isler,
• Jianbo Shi and Mirko Visontai
• University of Allovertheplace

2
The collaboration map
Jianbo Alyosha Volkan
3
Problem Statement
4
EASY!
5
How about this?
6
Not that simple
7
The Physics
Snells law
Observation Time is of the essence!
8
Bag of Patches
Hand-tracked point
9
Finding the center
Hand-tracked point
10
If we be Gods

• We could track a point
• Compute the mean (x,y) position

11
If we be Demi-Gods

• If we knew all distances between points
• We could compute center of graph
• Node with smallest sum of distances to everyone
  else

12
Patch distances

• SSD, NCC
• They work only if the patches are visually
  close but not otherwise
• Euclidean distance in n-dimensional space
• n number of pixels in a patch
• The patches do not span the entire space but live
  on a constraint surface the bottom of the
  pool!
• Idea use geodesic distances on this surface

13
Isomap
14
In general

• Build a nearest neighbor graph G.
• Vertices patches
• (u,v) is an edge iff SSD(u,v) is small
• For any two patches, we approximate the distance
  between them with the shortest path on G
• This technique forms the basis of many manifold
  embedding techniques in Machine Learning. (e.g.
  Isomap)

15
Leakage problem
16
Shortest paths
17
Flow network
Cost ssd Capacity infinite
MIN COST FLOW SUM OF SHORTEST PATHS LOAD
18
The leakage problem
So, why not put capacities on the arcs??
19
Putting soft constraints

• If we could make the cost increasing with the
  amount of flow, this can act as a soft constraint.

Cost/capacity
Pipe with stuck hairs
f(a)
20
(No Transcript)
21
Doesnt leak any more!!
Isomap
us
22
Experiments

• What is written at the bottom of the pool?

Alyosha thanks Andreas Ferencz and Stella Yu for
the movies and support!!
23
First attempt
Pixel mean
Pixel median
24
(No Transcript)
25
(No Transcript)
26
..and the reconstruction
27
Up true mean and median patches
28
Future work

• Seeing through fire, smoke,
• Tracking patches
• Rendering novel objects underwater
• Walking on water

29
As of last morning, 755am