Interactive Visualization in 4k

1
Interactive Visualization in 4k

• A Tutorial
• Jürgen P. Schulze, Ph.D.
• California Institute for Telecommunicationand
  Information Technology (Calit2)
• University of California San Diego
• PRAGMA 14 March 10, 2008

2
Bio

• M.Sc. Computer Science (1999)
• Ph.D. Computer Science (2003)
• Post-doc at Brown University (2003-05)
• Post-doc at UCSD (2005-2006)
• Project Scientist at UCSD (since 2006)
• 1 staff programmer (hiring another)
• 5 students (2 graduate, 3 undergraduate)

3
UCSD

• Founded 1960
• Number of students 26,000

Taiwan
4
California Institute for Telecommunications and
Information Technology (Calit2)

• New Laboratory Facilities
• Nanotech, Chips, Radio, Photonics, Grid, Data,
  Applications
• Virtual Reality, Digital Cinema, HDTV, Synthesis
• Over 1000 Researchers in Two Buildings
• Linked via Dedicated Optical Networks
• International Conferences and Testbeds

UC San Diego
UC Irvine
Preparing for an World in Which Distance Has
Been Eliminated
5
Calit2 Digital Cinema Theater
200 Seats, 8.2 Sound, Sony SRX-R110, SGI Prism
w/21TB, 10GE to Computers/Data
6
Stereo Wall (C-Wall)

• Dual HD resolution (1920 x 2048 pixels) with JVC
  HD2k projectors
• 2 Linux PCs w/Nvidia Quadro 5600
• Passive stereo
• Ascension Flock of Birds tracking system w/Wanda

7
Outreach
8
The StarCAVE

• Computers 15 Dell XPS PCs with Quad Core Intel
  CPUs
• OS CentOS Linux
• Graphics cards2 Nvidia Quadro 5600 per node
• Projectors 30 JVC HD2k (1920x1080 pixels), 30
  megapixelsm per eye
• Stereo passive with circular polarization
  filters
• 15 screens, 8 x 4 feet each
• Optical tracking system by ART
• Visualization softwareCOVISE, OpenSceneGraph
• Programming Language C

9
Calit2s 225 Megapixel HiPerSpaceTiled Display
10
Overview

• Interactive graphics
• Graphics frameworks
• COVISE/OpenCover
• DaVinci project

11
Computer Graphics and Virtual Reality

• Computer Graphics
• non-real-time rendering
• focus on image quality
• can be non-interactive, configuration file driven
• in interactive applications progressive
  refinement possible (simple data set during mouse
  interaction, full detail when no interaction)

• Virtual Reality
• real-time rendering (15 fps)
• focus on frame rate
• interactive, input device driven
• no progressive refinement because of head-tracked
  user

12
Low Level Graphics API

• OpenGL is most widely used
• available for Linux and Windows
• fast
• state-machine
• good support for new graphics cards
• DirectX
• requires Windows
• very good support for new features in graphics
  cards

13
Multi-Display Support Mechanisms

• Distributed scene graph
• e.g., OpenSG (not OpenSceneGraph)
• automatically updates on all cluster nodes upon
  changes
• support for very large scene graphs
• Event-synchronized
• e.g., Covise, VR Juggler/Clusterjuggler
• master node sends input events to render nodes
• each render node runs a complete copy of the code
• scene graph size limited to single node
• Pixel map based
• e.g., SAGE (EVL)
• master node sends pixel images to render nodes

14
Graphics APIs vs. Software Frameworks

• Graphics APIs
• provide interface to graphics routines
• e.g., OpenGL, SGI Performer, OpenSceneGraph,
  OpenSG, Java3D
• VR Frameworks
• integrate
• graphics API
• VR input device support
• flexible display device configurations
• audio support

15
Frameworks Differences

• Size of user base, on-line forums
• Free, open source, commercial
• Supported platforms
• Programming language
• Availability and quality of documentation
• Input device support
• Display device support
• GUI API, widget library
• Synchronization of cluster nodes
• Supported graphics APIs and scene graphs
• Data distribution paradigm on clusters
• Supported standard visualization algorithms
  (volume rendering, particle tracing, etc.)

16
Software Frameworks Examples

• COVISE (HLRS)
• SAGE (EVL)
• CGLX (Calit2)
• VTK (Kitware)
• WorldToolkit (Sense 8)
• CAVELib (VRCO)
• Ygdrasil (UIC/EVL)
• VR Juggler (Iowa State)
• DIVERSE (VA Tech)
• Avocado (GMD)
• FreeVR (Bill Sherman)
• dVS (Division)
• Lightning (Fraunhofer IAO)
• Amira (TGS/Mercury)

17
SAGE
HD Video from BIRN Trailer Macro View of Montage
Data Micro View of Montage Data Live Streaming
Video of the RTS-2000 Microscope HD Video from
the RTS Microscope Room
Source David Lee, NCMIR, UCSD Jason Leigh,
EVL, UIC
LambdaCam Used to Capture the Tiled Display on a
Web Browser
SAGE Developed Under Jason Leigh, and Luc
Renambot, EVL
18
4k Projectors at Calit2

• Sony SRXD R110
• Resolution 4096x2160 pixels
• Video input 4 x HD-SDI
• 10,000 Lumens
• Refresh rate 24 or 30 Hz
• SXRD Silicon X-tal Reflective
  Displayreflective liquid crystal microdisplay
• JVC DLA-SH4K
• Resolution 4096x2400 pixels
• 3,500 Lumens
• Video input 4 x Dual-Link DVI-D
• Refresh rate 60 Hz

19
Sony 4k Projector with HD-SDI

• Miranda DVI to HD-SDI converters
• Miranda boxes send incorrect resolution
  parameters to DVI port
• Solution
• connect JVC HD2k projector
• run 'nvidia-settings' tool
• save EDID file
• Configure EDID file in xorg.conf
• Use Twinview on each graphics card
• Use Xinerama between graphics cards

20
Quadrant Synchronization

• Synchronized buffer swap
• Hardware level NVidia G-Sync II board
• If Nvidia driver does not work try
• http//www.nvidia.com/object/linux_display_amd64_
  169.07.html

Nvidia G-Sync II
21
4k PC at Calit2

• Dell XPS 710
• Intel quad core CPU
• 4 GB RAM
• 500 GB harddisk
• Dual Nvidia Quadro 5600 graphics cards
• Nvidia G-Sync II board
• CentOS 4.6 (ROCKS compatible)

22
Operating System

• CentOS 4.6 (ROCKS compatible)
• Basic installation steps
• Download CentOS ISO image at http//isoredirect.c
  entos.org/centos/4/isos/
• Burn DVD and install OS from DVD
• Update OS (with round red icon at top of screen)
• Install latest graphics driver
• use "nvidia-settings" tool to configure display
  resolution
• Install additional libraries as needed with rpm
  or yum
• Detailed instructions for IVL are at
• http//ivl.calit2.net/wiki/index.php/Guide_to_inst
  all_Linux_on_lab_computer

23
xorg.conf for Sony 4k projector
Created with 'nvidia-settings' tool

• Section "Screen"
• Identifier "Screen0"
• Device "Videocard0"
• Monitor "Monitor0"
• DefaultDepth 24
• Option "TwinView" "1"
• Option "metamodes" "DFP-0
  1920x1080_60 19200, DFP-1 1920x1080_60 00"
• Option "CustomEDID" "DFP-0/etc/X11/edi
  d_miranda.bin DFP-1/etc/X11/edid_miranda.bin"
• SubSection "Display"
• Depth 24
• Modes "1920X1080" "1600x1200"
  "1280x1024" "1024x768" "800x600" "640x480"
• EndSubSection
• EndSection

24
COVISE

• Developed at University of Stuttgart
• Support for desktop and VR display and input
  devices
• Support for tiled displays

25
OpenCover

• COVISE's virtual reality renderer
• Can run inside or separate from COVISE
• Written in C
• Based on OpenSceneGraph
• No license required
• Closed source, but open programming interface
  (API)
• Modular application system
• Modules can run concurrently
• Download from HLRS web site

26
OpenCover Plugins

• C callbacks for initialization, rendering,
  message passing
• C module class
• Access to XML configuration file
• OpenSceneGraph based
• Support for straight OpenGL code
• Run-time linking (plugins compile separately)

27
OpenCover Plugin Structure

• C plugin interface code
• Plugin code is in C class

ModelViewer plugin NULL // global instance
of C plugin class int coVRInit(coVRPlugin m)
// initialization called once at start-up if
(pluginNULL) plugin new ModelViewer(m)
return 0 void coVRPreFrame() // called once
before rendering plugin-gtpreFrame() void
coVRDelete(coVRPlugin ) // called when plugin
is removed delete plugin
28
OpenCover Menu System

• Programming interface similar to Java/AWT
• Widgets menu, sub-menu, label, action button,
  check box, radio button, dial, slider
• Support for custom dialog windows
• All menus live in 3D space

29
Covise Configuration File

• 1 window, 1 pipe, 1 screen

ltCOVERgt ltPipeConfiggt ltPipe
display"0.0" name"0" screen"0" pipe"0" /gt
lt/PipeConfiggt ltWindowConfiggt ltWindow
width"3840" height"2160" left"0" bottom"0"
comment"FRONT" window"0" pipeIndex"0" name"0"
decoration"false" /gt lt/WindowConfiggt
ltChannelConfiggt ltChannel comment"FRONTLeft"
channel"0" left"0" right"3840" bottom"0"
top"2400" stereoMode"LEFT" windowIndex"0"
name"0" /gt lt/ChannelConfiggt
ltScreenConfiggt ltScreen width"2400"
comment"FRONTLEFT" h"0.0" originX"0"
originY"0" originZ"0" height"1350" p"0.0"
r"0.0" name"0" screen"0" /gt
lt/ScreenConfiggt ltStereo enable"off"
value"off" /gt ltStereoMode value"NONE" /gt
ltNumPipes value"1" /gt ltNumScreens value"1"
/gt ltNumWindows value"1" /gt lt/COVERgt
30
Visualization of multi-spectral images

• Goals
• Concurrent real-time visualization of multiple
  large, high-resolution images
• Research of user interface how best to interact
  with multi-spectral images for experts and
  general public
• Comparison of immersive environment (3D, large
  field of view) with high resolution displays (4k,
  tiled display wall)

31
Walking Into a DaVinci Masterpiece

• Image size400 megapixels(20,000 x 20,000
  pixels)
• Multiple layersvisible, IR, UV, Xray

The multi-spectral DaVinci painting on the C-Wall
32
Visible Light
33
Infrared Light
34
Walking into a Leonardo Masterpiece

• Co-registered modality images
• Support for translucent osgTerrain images
• Images can lock to screen
• Zoom by turning pointer
• Pan by moving pointer
• Fade between modalities by user distance from
  screen
• Single button interaction simple user interface
• Support for multiple image sets
• Co-registered marker

35
Rendering ultra-high resolution images

• Uses OpenSceneGraph's osgTerrain engine
• Supports RGB images and 2 1/2D terrain
• Clipmapping approach mipmapped tiles
• Dynamic loading manager loads only visible tiles
• Constant frame rate

36
User Interaction

• At Calit2 we use an Ascension Flock of Birds
• 6 degrees of freedom (DOF)
• Only wand is used in auditorium installation
• Distance of wand from screen determines modality
• Wand allows pan zoom
• 2D visualization with 6 DOF interaction

37
Future Work

• Master G-Sync use
• Develop lower cost solution for user interaction
  Nintendo Wii remote?
• Support gt3 modalities
• Two-handed interaction for pan zoom
• Learn from NCHC stereo 4k at Calit2!

38
Demonstration on 4k Projector
39
More Information

• IVL Wikihttp//ivl.calit2.net/wiki/
• Homepagehttp//www.calit.net/jschulze
• E-Mailjschulze_at_ucsd.edu
• Thank you!