Computer%20Vision%20Class%20project%20proposal

1
Computer VisionClass project proposal

• Brian Clipp
• COMP 256

2
Kalman Filter Implementation of Bundle
AdjustmentBrian Clipp

• Motivation
• Generate most acurate possible estimate of 3D
  points and camera poses given multiple views of a
  scene.
• Objective

3
Kalman Filter Implementation of Bundle
AdjustmentBrian Clipp
4
Kalman Filter Implementation of Bundle
AdjustmentBrian Clipp

• Advantages
• Variable bundle adjustment window size
• Each new P matrix or point added to the bundle
  adjustment window does not force re-computation
  of the entire bundle.
• Future P matrices are influenced by P matrices no
  longer in the window, forcing continuity in the
  model.

5
Plane Histograms for Planesweeping Stereo

• David Gallup

6
Planesweeping Stereo

• Hypothesize a plane
• Project images from all cameras onto it
• Measure photoconsistency (cost) as seen from a
  reference view
• Repeat for a family of planes
• Per pixel, record plane with best cost

near
far

• Multiway Planesweep
• Sweep planes in multiple directions
• Better alignment with surface produces better
  results
• Surface normals known through heuristics or other
  information

7
Cost Function and Plane Histogram

• Cost function
• Each pixel has a cost function
• Chosen depth is plane of minimum cost
• Poorly defined minimum in some cases
• Multiple local minima in some cases

• Histogram
• Histogram 3D points against planes in each
  direction
• Helps to disambiguate poorly defined minima
• Provides temporal coherency

8
Cost Function and Plane Histogram
9
Occlusion Inference with Visual Hull

• Li Guan

10
Probabilistic Occupancy Grid

• Jean-Sebastien Franco, Fusion of Multi-View
  Silhouette Cues Using a Space Occupancy Grid,
  ICCV, 2005
• See the Video

11
(No Transcript)
12
Project Goal

• To formalize the inference of POccluder from
  POccupancy Grid
• For every view (1-POccluder) PdPOccupancy
  Grid
• To infer occlusion masks for a single view
• Similar to the occupancy grid, to have an
  occlusion grid for the environment

13
Continuous Calibration

• Tyler Johnson
• COMP 256
• Spring 06

14
Calibration in Tiled Displays

• Multiple overlapping projectors must be
  calibrated
• Large up-front calibration process
• Stereo cameras triangulate display surface and
  calibrate projectors using correspondences

15
Continuous Calibration

• Would like to be able to continuously refine
  estimate display surface and projector parameters
• Structured light vs. feature detection in
  continuous image streams
• Current structured light methods sacrifice image
  quality -gt feature detection/correlation
• Requires dynamic scene with available features

16
Proposal

• Initial calibration of projector, stereo camera
  pair and estimate of surface geometry
• Detect features in the user imagery observed by
  camera and track with optical flow
• Given calibration, find features in other views
• Use matched features to re-estimate disp.
  surface/projector calibration
• Kalman filter aids in estimation and provides
  level of uncertainty
• Predictor-Corrector

17
3D Hole Filling Using Texture Synthesis Concepts

• COMP 256
• John Mason

18
3D Hole Filling Using Texture Synthesis Concepts
John Mason

• 3D geometry
• 3D by hand
• High quality
• Takes time and people
• 3D from scans
• Tends to have holes where scanner cant reach or
  due to other surface qualities
• Humans easily distracted by unexpected holes
• Multiple scans cant always fill the holes
• How to fill the holes?

Images Davis Levoy
19
3D Hole Filling Using Texture Synthesis Concepts
John Mason

• Texture Synthesis
• Take a small piece of texture
• Make a bigger piece of texture
• Fill in gaps in texture
• 3D Hole Filling
• Look at holes neighborhood
• Determine local geometric features
• Create a reasonable representation of the missing
  geometry
• Looks like it belongs or exact match?

Image Wei Levoy
Images Efros Leung
Image Wei Levoy
Images Davis Levoy
20
3D Hole Filling Using Texture Synthesis Concepts
John Mason

• Project Target
• Explore value of various texture synthesis
  methods applied to 3D hole filling
• Develop rapid, automated hole filling
  implementation
• Make sure filled hole looks like it should belong
  there
• As time allows, compare to other hole filling
  technologies

Prior work in the area of Texture Synthesis, and
related disciplines, includes works by Efros
Leung, Efros Freeman, Turk Levoy, Wei
Levoy, Kwatra, and Lefebvre Hoppe among others.
There is additional work recreating 3D textures
and surfaces including works by Dong Chantler,
Torrez Dudek, Ricken Warren, Velho et al, and
Davis Levoy among others.
21
Paul Merrell Project Proposal

• COMP 256

22
Hole Removal

• 3D Reconstructions from stereo often have holes.
• The holes are a result of occlusions.
• Goal of my project is to fill in the holes.
• Doesnt need to be perfect, just better than the
  holes.

23
Scene with an Occlusion
Use Smoothness to fill in hole w/o texture
Texture Synthesis to Create Texture
24
2D Hole Filling Using Texture Synthesis
3D Hole Filling
Copy from Similar Parts
25
Parametric (Global) Optic Flow for Stabilization

• Brad Moore

26
Goal

• Evaluate and implement parametric (global)
  optical flow
• Speed of algorithm
• Points of failure

27
Goal (cont.)

• Apply to the problem of stabilization (removing
  jitters)
• Implement a real-time demo

28
Approach

• Development on Windows PC
• Program in Matlab
• Input Device webcam (purchase, acquire) or video
  from digital camera
• Possible Issues
• Speed
• Noise (quality of camera)

29
Ultrasound Calibration

• Tabitha Peck

30
Ultrasound Calibration
Tabitha Peck

• Problem How do you determine the relationship
  between the location of an ultrasound probe and
  the real world location of the objects in the
  image?

31
The Setup
Tabitha Peck
Ultrasound Probe
Optotrak Certus Tracker
Image
Phantom
32
Challenges
Tabitha Peck

• Distinguishing items within the phantom
• You do not know the exact location of items in
  the phantom
• Error produced by Optotrak Certus
• 0.1 mm for x, y coordinates
• 0.15 mm for z coordinate
• Ultrasound images are not 2-D

33
Image Segmentation for Display Surface
Reconstruction

• Overview (WAV)
• The goal
• - Robust automatic calibration of projectors
  displaying on complex surfaces, using commodity
  hardware (i.e. cameras, projectors, PCs)

R. Skarbez -- 20 Feb 2006
34
Image Segmentation for Display Surface
Reconstruction

• One aspect of this problem is the accurate
  modeling of the surface geometry
• WAV has used several methods
• Tessellation of the 3D point cloud
• Benefits
• Extremely general (Makes no assumptions about the
  underlying display geometry)
• Drawbacks
• Very inefficient (1000s-100000s of polys)
• Noisy (Small errors make surface look wavy)
• Does not reconstruct corners well (BIG problem)

R. Skarbez -- 20 Feb 2006
35
Image Segmentation for Display Surface
Reconstruction

• Previous attempts, contd
• Fitting planes to data (Paper in EDT 06)
• Benefits
• Many fewer polygons (1s-10s)
• Much less noise (Surface forced to be planar)
• More accurate corner detection than previously
• Drawbacks
• Recursively fit on entire point cloud can
  generate spurious or redundant planes
• Corners still not perfect dependent on quality
  of fitted planes
• Only works on piecewise-planar display surfaces
• Want to get best of all worlds Generality,
  efficiency, low noise, and better corner
  detection
• How? Segment the image, and process each segment
  independently

R. Skarbez -- 20 Feb 2006
36
Image Segmentation for Display Surface
Reconstruction

• Method
• Scan lines across the display surface (using the
  projectors)
• Segment the camera images based on
  discontinuities in the observed lines
• Process each segment individually (e.g. fit a
  plane if possible, if that fails, try fitting a
  quadric, or tessellate the data)
• Reduces to plane-fitting in the simple case
  (albeit without spurious planes, and possibly
  with more accurate corners, due to the fact that
  they are directly located)
• In the general case, allows for handling of
  arbitrary display surfaces at the cost of
  generating a more complex display model

R. Skarbez -- 20 Feb 2006
37
Avatars in VR

• Jeremy Wendt
• Avatars - representation of the user in the
  environment
• Improve Presence
• Requires tracking several joint locations

38
Camera Tracking

• Needs to be fast
• 30 fps
• latency below 100ms
• Needs to be robust
• Jerky motion or unpredictable update break
  presence
• Large tracked area, low cost would be benefits
  not necessary

39
Proposed System

• Use brightly colored markers
• Required (Real TimeUI)
• 0. Calibrate cameras
• 1. Initialize color markers for cameras
• 2. Track the color blob center images
• 3. Find 3 space coordinate for tracked position
• Bonus
• 4. Smoothing results between frames
• 5. Placing avatar at specified location
• 6. Use in HMD tracked space

40
Aerial Image registration- Changchang Wu

• Image sources
• satellites (USGS)
• helicopters
• GIS (Google Earth,)
• 3d model screenshot
• The matching problem
• Matching of different image sets
• Matching the images with 3d model
• Images with elevation information
• Retrieve the geographical location

41
Image Matching

• Local matching of SIFT features
• Scale Invariant Feature Transform (Lowe99)
• Select the best match for each feature
• Look for local constraints for generating
  reliable local matches in this particular problem
• Look for strategies of selecting more
  discriminative SIFT features
• Global matching
• Improve the efficiency and robustness of the
  current matching algorithm
• Implement the matching of images with elevation
  data, use RANSAC to fit the model.

42
A matching example

• 6032 SIFT in the left, and 5591 in the right (
  part of them are displayed)

43
Normal Recovery from Reflective Surfaces

• Talha Zaman
• COMP 256

44
The Idea

• Urban environments often have buildings with
  large reflective surfaces (plate glass windows)
• The reflections from the surfaces contain a large
  amount of information about the surface geometry
• The goal is to use this information to recover
  the normals and reconstruct the shape of the
  reflective surface

45
The Approach

• Take a picture (or several pictures) of the
  reflective surface
• Take pictures of the reflected scene directly
  (from same camera position?)
• Figure out the relationships between the surface,
  the camera, and the scene
• Relate reflected image to real image and recover
  normals
• Reconstruct the surface