3D Devices and Interaction

1
3D Devices and Interaction

• Hwang Jae-In
• 9825M30
• Computer Science and Engineering
• POSTECH

2
Introduction

• JDCAD A Highly Interactive 3D Modeling System
• 3D Direct Manipulation Interface Development of
  The ZASHIKI-WARASHI System
• FINGER-POINTERPointing Interface by Image
  Processing
• Two-Handed Spatial Interface Tools for
  Neurosurgical Planning

3
JDCAD(1)

• JDCAD A Highly Interactive 3D Modeling System
• System Overview
• Kinetic 3D display - head tracker
• 3D input using the bat - tracker
• Basic Interaction Techniques
• Menu selection daisy, ring menu
• Object selection
• Bat operated dial
• Navigation in space

4
JDCAD(2)

• Modeling Operations
• Primitive Creation
• Region-based reshaping
• Placement and alignment
• Bat-based clipping

5
3D Direct Manipulation Interface(1)

• 3D Direct Manipulation Interface Development of
  the ZASHIKI-WARASHI System
• Direct Manipulation
• Continuous representation of the object of
  interest
• Physical actions instead of complex syntax
• Rapid, incremental, reversible operations whose
  impact on the object of interest is immediately
  visible.
• 3D Direct Manipulation
• Representation of objects, environment, and space
• Object manipulation technique
• High speed processing

6
The Zashiki-Warashi System

• System Overview
• Zashiki-Warashi helps users to arrange solids and
  lighting in a virtual reality room.
• Two devices Graphical workstation, 3D
  digitizer(pen-like stylus)
• Three tool objects beam cursor, lights, camera
• Environmental factors lighting effects,
  gravity, shadows of solid

7
The Zashiki-Warashi System

• The beam cursor
• follows the movement of the stylus
• The solid that is hit by the beam can be selected
  and moved
• Lighting and view control
• allow control of lighting and viewpoint in the
  same manner as solids.
• Light source position, color, direction,
  type(parallel/spot/point)

8
The Zashiki-Warashi System

• Gravity
• zashiki-warashi system use gravity
• Drop-gt collision-gt rotate and come to rest
• Other features
• zashiki-warashi has a CAD interface, so a solid
  designed with a CAD system can be input into the
  virtual world.
• Manipulation of solids select, move,
  release(falling motion), release(coming to rest),
  rotate space

9
Finger-Pointer

• System Concept
• Pointing actions are captured by two stereoscopic
  TV cameras
• The system determines the 3D coordinates of the
  operators finger tip
• communication using combination of voice and
  gestures

10
Image Processing Methods

• Finger-tip detection for Locator
• binarize the two images
• scan each binary image and determine the pixel
  that is closest to the screen
• calculate 3D position of the candidate pixel
• Decide whether that candidate pixel represents
  the real finger tip

11
Image Processing Methods

• Thumb-click detection for Trigger
• scan the binarized wall camera image and
  determine the wrist center
• scan the image from line between finger tip and
  wrist position, determine the uppermost pixel
• Decide whether that candidate pixel represents
  the real thumb tip
• Finger-number detection for Valuator

12
VPO Virtual Projection Origin

• Pointing direction
• determined by a straight line between two points
  Tip-Point and Base-Point
• Base-Point is different for each operator
• Virtual projection origin
• Display marks on upper-right corner of the front
  screen
• Measure the operators finger tip position,
  determine pointing line
• Repeat for several other positions
• Estimate VPO as the point at which these lines
  converge

13
VPO Virtual Projection Origin

• Estimating convergence point The VPO is
  estimated as the center of a sphere that has
  minimum radius.

14
Channel Synchronization

• Multi-modal pointing
• finger is used for locator, the voice is used
  for a trigger
• Symbol mismatch delay of voice recognition
• Timing-tags

15
Applications

• Presentation system
• Video browser
• Space Writer

16
Two-Handed ...

• Two-Handed Spatial Interface Tools for
  Neurosurgical Planning
• Props-based interface
• Medical User Interface
• Head-viewing prop small dolls head
• Cross-sectioning prop plastic plate
• Trajectory-selection prop handheld stylus for
  pointing to the dolls head
• Not head-tracked, no streoscopic image, no HMD

17
Interface overview

• six degree of freedom tracking system, Fastrak
  Polhemus
• virtual head is updated 15 times per second
• for interactive rotation, use polygonal
  approximation to the patients brain.
• Cross-sectional prop as surgeon moves the
  plate, the computer shows the corresponding MRI
  volume
• trajectory selection prop

18
3D image segmentation

• Brain surface model encodes the gross brain
  surface
• voxel-based model encode typical brain shape