MultiProjector Displays

1
Multi-Projector Displays

• Forschungsseminar Virtual Reality
• SS 02

2
Overview

• Possibilities and Motivation
• Foundations
• Projection Technologies
• Scene Registration
• Geometric Transformations
• Image Transformations
• Applications and Conclusion

3
Possibilities and Motivation

• Possibilities
• Projection on large surfaces
• Non-planar surfaces and/or oblique projection
• Fast setup, reconfiguration and calibration
• Motivation
• Immersive VR environment without HMD devices or
  need of static configuration

4
Examples
PixelFlex
Princeton Display Wall
Non-planar surfaces
Dynamic shadow removal
5
Projection Technologies
6
Projection Technologies (1/5)

• CRT (Cathode Ray Tube) Projectors
• High refresh rates (100-120Hz)
• Relatively low cost
• - Large and heavy devices
• - Low brightness (250 ANSI lumen)

7
Projection Technologies (2/5)

• LCD/TFT Projectors
• Individual grayscale LCD for each color
• Pixel dimensions lt50µm
• Low cost
• - Poor contrast and black level

8
Projection Technologies (3/5)

• DLP (Digital Light Processing) Projectors
• DMD runs at ist own frequency (60Hz)
• Uses information of several frames for artifact
  compensation -gt delay
• Brightness Pulse Width Modulation

9
Projection Technologies (4/5)

• D-ILA (Digital Image Light Amplification)
• Reflective Liquid Crystal
• Write light much lower intensity than read light
• High resolution and brightness
• - Big heavy devices
• - Expensive

10
Projection Technologies (5/5)

• Projector Mosaics

11
Mounting Mechanical Calibration

• 6 DOF Projector Mounts
• Computer Controlled Pan-Tilt Units

12
Scene Registration
13
Scene Registration (1/5)

• For complete immersion we need to know
• Relationship among viewer, 3D model and display
  surface
• Relationship between Projector Pixels and surface
  points

14
Scene Registration User Location

• Head of user is tracked in World Coordinates
• sweet spot at default location

15
Scene Registration Projector Position

• Use a camera to observe projected Pixels
• Mapping Display Framebuffer -gt observed pixel

16
Scene RegistrationStructured Light

• Can be used for
• Surface Reconstruction
• Collineation estimation

17

Scene Registration Projector Position

• Mapping to Reference coordinates
• Reference Markers
• Tilt sensor

18
Geometric Transformations

19
Geometric Transformations

• Adaption of the scene to be displayed according
  to the surface where it is displayed
• Collineation
• To overcome oblique projection
• Warping using projective textures
• Rendering a perspectively correct image on
  irregular surfaces
• Surface mesh unification

20
Geometric TransformationsCollineation 1/8

• Traditional projectors are orthogonal and create
  rectangular images
• Oblique projectors create keystoned images which
  appear distorted

21
Geometric TransformationsCollineation 2/8

• Goal
• Avoiding of frequent mechanical adjustments
• Compensate for the image distortion using the
  graphics pipeline

22
Geometric TransformationsCollineation 3/8

• For planar display surfaces virtual point V must
  be displayed at M
• Simple off-axis projection matrix PT is
  sufficient for orthogonal projection

23
Geometric TransformationsCollineation 4/8

• Modified projection matrix achieves off-axis
  projection PT followed by collineation A4x4
  (mapping mT ? mP)

24
Geometric TransformationsCollineation 5/8

• Collineation is induced due to the plane of the
  screen
• A3x3 maps pixel coordinates from one image to
  another

25
Geometric TransformationsCollineation 6/8

• Computing of pixel coordinates mP directly from
  virtual point V
• PT homogenous 3D coordinates ?normalized
  homogenous 3D coordinates
• New A4x4 normalized 3D coordinates ?projector
  pixel coordinates
• Rendering and warping in a single projection
  matrix without additional cost
• Single pass ? no resampling artefacts
• Virtual Object correct on any surface coplanar
  with ?

26
Geometric TransformationsCollineation 7/8

• Sequence of steps to create immersive displays
  with an oblique projector
• Find screen locations of at least 4 projector
  pixels
• Find transformation between tracker and world
  coordinates
• Find collineation between Mi? and mPi
• For user location T compute PT and collineation
  A4x4

27
Geometric TransformationsCollineation 8/8

• Example of image displayed by highly oblique
  projector
• Its contribution to an image displayed by three
  overlapping projectors

28
Geometric TransformationsWarping using
projective textures 1/6

• Using of projective textures in a two-pass
  image-based scheme for
• Multiprojector case
• Multisurface case
• Combination of both

29
Geometric TransformationsWarping using
projective textures 2/6

• Traditional approaches
• Multiprojector systems
• Single projector for each planar plane (see CAVE)
• Multisurface systems
• Multiple viewports
• New
• Image warp using texture mapping

30
Geometric TransformationsWarping using
projective textures 3/6

• Image generation using projective texture
  rendering

è
31
Geometric TransformationsWarping using
projective textures 4/6

• 2-passRendering
• Algorithm (for each viewers eyes)
• Compute desired image for eyes viewpoint
• Project image from the eye out to the display
  surface
• For each projector render display surface from
  viewpoint of projector

32
Geometric TransformationsWarping using
projective textures 5/6

• Advantages
• Multiprojector case
• Scene is only traversed once if driven by single
  machine
• Multisurface system
• Texture coordinates for point on the display
  surface are automatically generated
• No warp function is explicitly computed

33
Geometric TransformationsWarping using
projective textures 6/6

• Example Kaiser Head-Mounted Display (KHMD)

34
Geometric TransformationsSurface mesh
unification 1/2

• Object
• Create a single representation of the display
  surface
• From the multiple meshes retrieved by the
  different cameras

35
Geometric TransformationsSurface mesh
unification 2/2

• Technique
• Weighting of associated 3D location
• Algorithm
• New display geometriccontinous displaysurface
  Di

36
Image Transformations

37
Image Transformations

• To correct distortions in color and brightness
  and to exactly align and overlap several
  projected images
• Alpha Blending
• Color and Luminance Calibration
• Dynamic Shadow Removal

38
Image TransformationsAlpha Blending (1/4)

• Blending overlapping tiles to create a seamless
  image

è
39
Image TransformationsAlpha Blending (2/4)

• Physical Shadow masks
• True black
• No processing necessary
• - No automatic calibration
• - Only straight edges

40
Image TransformationsAlpha Blending (3/4)

• Virtual Shadow Masks (alpha channel)
• arbitrary shapes
• automatic calculation
• - Viewing angle dependencies of screen reflection
• - LCD and DLP Projectors cant produce true black

41
Image TransformationsAlpha Blending (4/4)

• Calculating the mask
• Alpha weight Am for projector m at pixel (u,v)
• i ... Index of projectors
• ai(m,u,v) wi(m,u,v)di(m,u,v)
• wi(m,u,v) 1 if inside projector is hull, 0
  otherwise
• di(m,u,v) ... Distance of pixel from neares edge
  of overlap region

42
Image TransformationsColor and Luminance
Calibration

• Projectors differ in color and luminance output
• For large arrays of Projectors manual calibration
  is nearly impossible
• Closed-Loop Camera observation generates
• Color transfer function
• Color Lookup Table (CLUT)

43
Image TransformationsDynamic Shadow Removal

• 2 Projectors from different angles observing
  Camera
• 2 approaches
• Predicted image is compared to captured image -gt
  delta Pixels
• Bounding-box fitting to detect shadow regions

44
Applications
PixelFlex
Princeton Display Wall