Interactive Collision Detection for Free-Viewpoint Video

1
Interactive Collision Detection for
Free-Viewpoint Video

• Bert De Decker
• Tom Mertens
• Philippe Bekaert

Hasselt University - EDM - tUL
2
Interactive Collision Detection for
Free-Viewpoint Video
3
Related work (1/2)

• Interaction techniques
• Wang et al. (2005)
• Kitamura et al. (2003)
• Force feedback
• Lindeman et al. (2004)

4
Related work(2/2)

• Collision detection between real and virtual
  objects
• Intermediate geometry
• Allard and Raffin (2006).
• Stam (2000).
• Directly on images
• Lok et al. (2003).

5
Visual Hull Based

• Real object intersects virtual object
• Projection of virtual object intersects
• image of real object

6
Point based

• Object point sampling
• Many types of objects
• Meshes, implicit surfaces, point clouds,
• Per-point test is cheap, so dense sampling is
  possible
• Sampling too sparse is possible

7
Recover from collision

• Multiple Collision Response vectors possible
• Collision response vector perpendicular to
  surface
• Take speed into account

8
Silhouette extraction
9
Point inside/outside test

• project 3D point to each input image
• Fast but not cache coherent, slow memory

10
Bounding Volumes

• Bounding frustum of the visual hull
• AABB of the bounding frustum

11
Point Collision Response
12
Point Collision Response
13
Point Collision Response
14
Point Collision Response
15
Point Collision Response
16
General Collision Response
17
Camera Setup

• 7 calibrated cameras
• Green cloth

18
Network Setup
19
Visualization and Rigid Body Simulation

• Hardware-accelerated rendering of photo hulls, Li
  et al. (2004)
• Pure image based no geometric representation of
  the real object
• ODE library for rigid body simulation

20
Performance

• 10 fps
• 7 cameras
• Rendered resolution is 640x480
• Input images are 320x240
• 500K points per second
• Not optimized

21
Person topples stack of boxes
22
Person lifts real box and tips over virtual boxes
23
Person lifts up virtual box with real box
24
Person kicks a ball
25
Person versus pile of boxes
26
Future Work (1/3)

• Parallel implementation
• PCs the cameras are attached to do the collision
  detection
• Send points over network, not images
• High resolution input images possible

27
Future Work (2/3)

• Interaction Techniques
• User grabs objects
• Sticky real objects
• Tracking of body parts is not easy

28
Future Work (3/3)

• Fast objects
• Fast virtual objects
• Bounding volume of the object over time
• The position of the virtual object is known at
  every instant
• Fast real objects
• We only have information of the real object at
  discrete time steps
• Interpolation of this information is difficult

29
Conclusion

• Interactive collision detection
• Real ?? virtual objects
• Image based, no geometric representation
• Visual hull based
• Point sampling virtual objects
• Multiple simultaneous collisions possible

30
Questions