Dongyan Xu, Daniel Aliaga,

1
Dongyan Xu, Daniel Aliaga, Scott Yost, Jonathan
Deutsch
2
MRT Overview
3
Motivations

• To create a common plane for immersive
  demonstration and perception
• Allow users to focus on the tabletop instead of
  shifting their vision and attention back and
  forth between the computer screen and the
  physical objects

4
Advantages

• Cheaper than VR
• Works better than head mounting
• One focus (unlike video-conferencing)
• Can use computer-unfriendly objects

5
Conference XP

• Participants
• Venues
• RTP - Real-Time Protocol

6
DirectShow

• Insert source-filter to grab the image from the
  camera and do camera calibration
• Insert transform-filter for projector calibration

7
Raw Image
8
Camera Correction

• Camera is not perfectly aligned with deskop, has
  spherical distortion, etc.
• Create Tsai model for calibrated camera
• Sample Tsai model to warp the camera image into
  canonical space

9
Corrected Image
10
UI and Graphics
11
Raw Projector Output
12
Projector Calibration

• Projector is misaligned, projection area doesnt
  perfectly overlap with camera field of view
• Project sequential pattern onto table, scan
  output via camera to detect projector usable
  area
• Can then warp images to projector such that they
  appear squared and aligned

13
Prewarped Output
14
Image Pipeline
Camera Correction
UI graphics
Projector Correction
Projector
15
One more problem!

• Problem Camera lt-gt projector feedback loop will
  saturate image
• Solution Synchronize camera and projector

16
Synchronization!

• Immediately before the camera grabs an image,
  stop projecting onto the table
• Result Camera sees only real objects, nothing
  from projector
• Accomplished through hardware synchronization
  pins on camera, software

17
Applications

• We provide image processing, networking code
  you provide the app.
• Application 0 Digital Paper

18
Applications

• Focus collaborative teaching
• young children origami, other art
• school-age chemistry, physics experiments

19
Applications

• Your ideas?

20
Interested?
Contact Prof. Dongyan Xu (dxu_at_cs.purdue.edu) Pro
f. Daniel Aliaga (aliaga_at_cs.purdue.edu)