DGL / 3DITs presentation

1
DGL / 3DITs presentation

• By
• Andrew A. Ray

2
Schedule

• 900-1000 - DGL Overview and programming
  guide
• 1000-1010 - Break
• 1010-1030 - Discussion on potential usages of
  DGL
• with your software projects
• 1030-1100 - Overview of available VR hardware
• 1100-1130 - Introduction to 3D Interaction
  techniques
• 1130-1140 - Break
• 1140-1200 - Discussion and final wrap-up
• 1200 - Lunch

3
DGL

• Capabilities
• Case studies
• Output
• OpenGL, Open SceneGraph, Coin, VTK
• Input
• Mouse, keyboard
• Communication with other processes

4
Capabilities of DGL

• Extends DIVERSEs reach into OpenGL based
  rendering and toolkits built to render to OpenGL
• Primary thrust is to provide VR capabilities to
  Open Scene Graph, while allowing more flexibility
  than DPF did with OpenGL Performer
• Improves upon some parts of the DPF API

5
Case study 1

• Road crossing experiment using VRML
• Elderly citizens crossing the street in a city
• Controlled environment with no unreasonable risk
  for the participant
• VRML animation for cars / streetlights
• Coin used to render the project
• Coinfly example was all it took to run the
  project

6
Case study 2

• Benefits of immersion study
• 3D mathematical equations
• Different levels of immersion (walls on / off)
• Sometimes head tracker used / sometimes not
• Used to help understand 3D math and what it looks
  like in its natural surroundings

7
Case study 3

• AMADEUS Underground mining visualization tool
• Designed to view statistically generated
  fractures to simulate what the rock looks like
  before it is tunneled through
• Somewhat simple novel computer graphics algorithm
  to cut different types of the tunnels out of the
  rock and be able to see them in true 3D

8
Screenshot
9
Result of graphics algorithm
10
DGL Output capabilities

• Based around a draw callback that is added to a
  dtkAugment
• Allows for OpenGL to be drawn before and after
  navigation and has an overlay capability to
  ensure that it is drawn after all the other
  OpenGL
• How does this model work with Open SceneGraph /
  Coin / VTK?

11
DGL Augment Interface

• Same exact process as making your own dtkAugment
• Derive off the draw callback
• Pass in the type of dglAugment you want it to be
  (i.e. where it is drawn)
• Load the DSO into your favorite application
• Done

12
Porting existing programs to DGL

• DGL is designed to ease porting old OpenGL
  programs.
• There are C based wrappers that offer the same
  functionality as the C interface
• dglDisplayCallback
• dglInitGLCallback
• dglPostframeCallback
• dglSet/GetData

13
Problems with porting old GL programs

• Timers arent implemented in DGL
• Input probably is going to have to be redone
• Depending on the application, it may have to be
  re-architected a bit to handle cluster
  development
• May also have to worry about multi-context issues

14
Open Scene Graph usage

• Step 1 Develop OSG application
• Step 2 Initialize DOSG (DOSGinit())
• Step 3 Insert top level OSG root node into
  DOSGgetWorld()-gtaddChild(here)
• Done

15
Open Scene Graph

• DGL handles structuring Open Scene Graphs
  internals so that all you have to do is add a
  child to a node in a scenegraph that is drawn by
  DGL
• Structure of scenegraph is
• scene
• nav ether
• world

16
Coin

• Very similar to osg, except there is no nav node
• scene
• world ether
• DCoininit Make the system
• DCoingetWorld()-gtaddChild(here)
• DCoinloadFile(model.iv/wrl)

17
VTK

• 4 step process
• Create a render window
• (DVtkRenderWindowNew())
• Create a renderer (DVtkRendererNew())
• Add the renderer to the window (AddRenderer(pointe
  r))
• Add your actors to the scene
• Flip the scene to OpenGL coordinates if desired
  (DVtkRenderer-gtRotateSceneX(90))

18
Input for DGL

• Now that we have graphical development taken care
  of, we need to get how the user develops
  applications!
• Same as DPF for wand / joystick / buttons / head
  tracker (dtkSharedMemory)
• Has a different mouse / keyboard interface due to
  a different input system (Producer)

19
Mouse input

• DGLMouse and DGLMouseButton
• DGLMouse allows for setting the mouse position,
  and reading the mouses current position or the
  queued position of the mouse (setMouse(x,y)
  readMouse(float x, float y) getMouse(float x,
  float y)

20
Mouse Input

• DGLMouseButton Tells you what happened to the
  mouse button, and the position of the mouse when
  the button action happened
• Based on top of shared memory
• Concept of Actions, state for each mouse button
• int readMouseButton(x,y, action)
• Returns 0 if it doesnt have data, otherwise it
  tells you the position it was changed at and what
  the change is
• Actions are press / release / unknown
• Allows for the ability to fake mouse events if
  there is a need

21
Keyboard Input

• It uses the same actions concept as mouse input
  did
• DGLKeyboardgetState(DGLKeyboardgetKeyCharacter
  (a))
• Pass an action pointer (to be written to) into
  readKeyCharacter and it will return the action /
  character that was pressed
• Do have to switch between char and KeyCharacter
  due to Producer syntax

22
The raw deal

• If you dont like the helper classes created then
  there is always shared memory.
• It is what the system is based off of.
• dgl/mouse dgl/keyboard dgl/mousebutton are the
  segments
• The structure is usually an integer for the data
  and an integer for the action

23
Nuts and bolts

• Building dgl programs is the same as with dpf
  programs, dgl-config is your friend
• dgl-config --include and --libs
• dgl-config --osg-include --osg-libs
• dgl-config --coin-include --coin-libs
• dgl-config --vtk-include --vtk-libs
• Example makefiles in the examples directory on
  how they are used

24
Design strategies for DGL programs

• Decide the type program that you are developing
  OSG , OpenGL, etc
• Decide the level of interactivity
• Often having a DSO that reads / displays data is
  the best approach because you can reuse DGL shell
  programs (i.e. dosg-fly)
• Full applications give the most flexibility and
  opportunity

25
DGL design strategies

• Often I break the problem into these specific
  parts
• Reading input data
• Transforming input data
• Displaying data
• Interacting with data
• Often need to be careful about structure of data
  storage / accessibility because these decisions
  often interactivity

26
DGL design strategies

• Often interactivity places the most demands on
  applications
• The earlier in the development process that you
  consider it, usually the best
• Aim for several quick iterations that can test
  different features / abilities
• Get users involved as soon as possible in the
  development cycle

27
Questions / Comments

• Capabilities?
• Design decisions?
• Application design?
• Building applications?

28
10 minute Break
29
VR Hardware

• Often what you do in VE creation is heavily
  influenced by the input devices you have
  available
• Very large range of devices, some very well
  supported, some not so much.
• In CAVE environments usually light weight one
  handed devices are preferable
• Often have to consider how equipment will affect
  others in the CAVE environment (for demos)

30
Outline for VR hardware

• Text entry
• Positional trackers
• Pointing devices (other than wands)
• Niche areas
• Haptics

31
One handed text entry
A one handed text entry method through chording
Press multiple buttons to obtain different
letters of the alphabet Advantages Only needs
16 keys to operate Can be used as a cheap input
device that has 16 different buttons Can be
combined with a velcrod tracker Disadvantages R
equires training Will not be as fast as regular
keyboard text entry
32
Frog pad
33
Software text entry
On screen keyboards are often used for text
entry. Basically make a rectangle with all of
the different keys you want. Stick a ray on
the end of the wand, put the ray into the letter
you want. Press a button on the wand to select
the letter.
34
TULIP

• Tulip menus / text entry
• Imagine wearing pinch gloves and having a
  floating keyboard in front of you in the world
• Reach out and pinch the letter in front of you
• 11 -12 WPM for expert users

35
Optical tracking
Different way of tracking Put markers (little
balls) on a set of glasses or a hat. The camera
then recognizes them and generates a 3D position
based on the Info. Advantages Cheaper than
Intersense Can easily make any object
trackable Disadvantages Requires users to wear
something Marker has to be visible to camera
36
Magnetic tracking

• Tracking through electromagnetic fields
• Doesnt work well with metal objects in the
  environment
• Higher latency than with Intersense trackers
• More portable though

37
Acoustical tracking
Acoustical tracking Low latency Not sensitive to
metal in the environment Not cheap or portable
(has to be calibrated for each environment. Very
good for fixed setups such as CAVEs
38
Computer vision tracking

• Through the use of markers and a standard webcam
  you can get 6DOF tracking
• Software process
• Useful and cheap for passive haptics

39
Gyroscopic mice
The gyroscopic mice provide 2D in air movement
and generally do not require drivers, just plug
in and the OS / USB does the rest (even on
Linux) Advantages -Allows for a cheaper (70)
more restricted wand movement (only desktop
X,Y) -Is possible to have multiple mice
influencing the VE at the same time -Scenario
(All data on front wall, multiple clickable
options). Disadvantages -Does not have
absolute tracking -Has to be used with care, a
flick of the wrist during a demo could be
problematic
40
Pinch Gloves
Pinch gloves allow for fingers to be
pinched together to produce different
combinations for input into the VE. Combine with
a 6DOF tracker to know where the users hands
are. Somewhat cheap 2,000 device Advantage
More natural style of interaction than with a
wand Disadvantage Does not track finger movement
41
Data gloves

• Allows for true finger positioning to be reported
  to the VE
• Can do sign language type gestures
• Has calibration issues
• More expensive
• Doesnt always work right
• Very expensive

42
Shape tape

• A piece of material that can be shaped and have
  this shape reported to the computer
• Useful for art, engineering (measuring / curve
  placement)
• Can be used for interfaces when combined with
  trackers
• Also can be used to imitate snakes

43
Omni Directional Treadmills
Imagine being able to walk around in an
environment naturally and not have to worry about
the running into the CAVE walls. Advantages Natu
ral movement in a VE Perfect for types of
training Disadvantages Expensive Noisy May be
harder than regular walking and may not react
well to running
44
Haptic devices

• Provide force feedback
• Sometimes in a pen form
• Sometimes a skeleton framework over a glove
• Often easier to use passive haptics (pit
  experiment)

45
Discussion on Input devices

• Would adding a different input device be useful
  for your application?
• Make for interesting demos
• Can even go so far as hang-gliders with fans /
  tilt recognition or walking in place harnesses /
  floor tiles that move around
• Questions / Comments

46
3DIT

• Interacting with VEs makes them not only more
  usable, but more useful
• Helps increase the sense of presence
• Many different types that provide very
  interesting abilities.

47
3DIT Input

• Require input devices to work
• Does not require a 3D input device to do a 3D
  task (mouse / virtual sphere)
• Input devices do enable / limit different types
  of input though

48
3DIT Realism

• Natural versus Magic techniques
• Think of having a virtual cup of water on the
  table in front of you.
• Reach out and grab it
• Now imagine it being 15 feet in front of you
• Both are possible in 3DITs
• Can be mixed to have some of both

49
3D Selection and Manipulation

• Tasks are mainly composed of selection,
  positioning, and rotation
• Selection is composed of indicating which object
  is desired, confirming that it is the object that
  is desired to be selected, and then providing
  feedback to the user.
• Can be done through a couple of different methods
  (ego / exo centric)

50
Different types of selection techniques

• Simple infinite (or finite) ray to select an item
• Two handed pointing
• Flashlight / aperture addition to raycasting
• Flexible ray casting
• Image plane techniques
• Fishing reel technique

51
Selection and Manipulation

• Once you select an object, what do you normally
  want to do with it? Manipulate it.
• Think back on the water cup example
• Imagine being able to stick your hand out 50 feet
  and grab objects
• Take a hand tracker, and use a non-linear scaling
  function on the object it is attached to

52
World in Minature

• Imagine being in a room and then having a top
  down view that is slightly bigger than your hand.
• Combine with a tracked tablet and can be used
  with a wand to pick and move objects
• Can be modified to provide an overview of the
  world on the floor of a cave and then warping
  based on that
• Can be used to replicate maps and then travel
  based on that

53
Voodoo Dolls

• Select an object in the world
• Create a miniature copy of the object in front of
  you
• Use tracked pinch gloves scale, rotate, and
  position the object

54
Travel techniques

• Exploration, search, maneuvering
• Scaling movement from slow to fast to cover long
  distances
• Velocity
• Location guides
• Reset to start usually essential
• Users get lost in VEs all the time, plan for it

55
Different types of travel techniques

• Point and fly based travel
• Gaze directed
• FPS style
• Button versus joystick
• Camera in hand technique
• Maps to place self in world
• Set of guided points

56
System Control

• There are whole hosts of options that can be
  configured with VEs
• How do you do this inside of a VE?
• 2D in 3D (Ring / XWand / VEWL type interactions /
  Pen Tablet)
• 3D menus (Tulip, cone trees, Command and control
  cube)

57
Different ways to combinetechniques

• 3DITs are often best used when combining with
  other 3DITs
• Example HOMER
• Select an object with ray casting, then
  manipulate with GoGo. Allows for selecting a far
  away object and moving it even farther.
• Flavors
• Adding small nuances to improve usability of
  technique. I.e. having the ray in ray casting
  snap to the nearest object in the world.
• Adding scaling and scrolling to the WIM.

58
Implementation strategies

• Where possible 3DITs should be created as DIVERSE
  DSOs
• Plugs into different applications
• Static interfaces are your friend
• Need to get a handle to the world (easier if the
  world is loaded as a DSO)

59
Implementation strategies

• Make sure they can be turned on and off
• Think about creating a master 3DIT DSO that turns
  sub parts on and off
• Dont be afraid to try different options
• You can only do so much with a simulator on the
  desktop, you need to use the real device in the
  CAVE to make sure that it works well

60
DGLMessage structure

• Think of having the visualization component as
  only interacting through messages sent to it
• You need a protocol for 3DITs and the Viz core to
  talk to each other
• One way is to have messages designed to be sent
  to a specific class to do something particular.

61
Structure of message usage
User
Action
3DIT
Message
Message Center
Delivers message
Vis1-3 register with the Message Center at
starting time
Output shown to user
Vis1
Vis2
Vis3
62
Message wrapup

• Advantage Works through shared memory so it can
  be used on cluster systems rather easily
• Disadvantages Not meant for large amounts of
  data transfer, mainly just directions on what to
  do. Takes planning, usage of multiple
  inheritance, and has overhead in implementation.
  Can pay off in the long term though.