VE Input Devices

1
VE Input Devices

• Doug Bowman
• Virginia Tech

2
Goals and Motivation

• Provide practical introduction to the input
  devices used in VEs
• Examine common and state of the art input devices
• look for general trends
• spark creativity
• Advantages and disadvantages
• Discuss how different input devices affect
  interface design

3
Input devices

• Hardware that allows the user to communicate with
  the system
• Input device vs. interaction technique
• Single device can implement many ITs

4
Human-computer interface
User interface software
Input devices
System Software
Output devices
User
5
Human-VE interface
Env. model
Display(s)
Simulation loop -render -check for
events -respond to events -iterate
simulation -get new tracker data
Tracking system
Input device(s)
6
Input device characteristics

• Degrees of Freedom (DOFs) DOF composition
  (integral vs. separable)
• Range of reported values discrete/continuous/hybr
  id
• User action required active/passive/hybrid
• Intended use locator, valuator, choice,
• Frame of reference relative vs. absolute
• Properties sensed position, motion, force,

7
Practical classification system

• Desktop devices
• Keyboards, 2D mice and trackballs, pen-based
  tables, joysticks, 6DOF devices for the desktop
• Tracking devices
• 3D mice
• Special-purpose devices
• Direct human input

8
Desktop devices keyboards

• Chord keyboards1
• Arm-mounted keyboards2
• Soft keyboards (logical devices)

9
Desktop devices 6-DOF devices

• 6 DOFs without tracking
• Often isometric
• Exs Fig. 4.4 SpaceBall 5000, SpaceMouse Plus,
  SpaceOrb

10
Tracking devices position trackers

• Measure position and/or orientation of a sensor
• Degrees of freedom (DOFs)
• Most VEs track the head
• motion parallax
• natural viewing

11
Other uses for trackers

• Track hands, feet, etc.
• whole body interaction
• motion capture application
• Correspondence between physical/virtual objects
• Props5,6
• spatial input devices

12
Tracking physical objects (props)
13
Motion Tracking

• Critical characteristics
• Range, latency, jitter (noise or instability),
  and accuracy
• Different motion trackers
• Magnetic
• Mechanical
• Acoustic
• Inertial
• Optical
• Hybrid

14
Electromagnetic trackers

• Exs Polhemus Fastrak, Ascension Flock of Birds
• Most common
• Used with conventional monitors (for fishtank VR)
  Small workbench displays
• Transmitter
• Receiver(s)
• Noisy
• Affected by metal objects -gt distort the magnetic
  field

15
Inertial trackers

• Exs Intersense IS-300, Intertrax2
• Less noise, lag
• Only 3 DOFs (orientation)

16
Optical/vision-based trackers

• Exs Vicon, HiBall, ARToolkit
• Advantages
• accurate
• can capture a large volume
• allow for untethered tracking
• Disadvantages
• may require light emitting diodes(LEDs)
• image processing techniques
• occlusion problem

17
Hybrid tracking

• Ex IS-600 / 900
• inertial (orient.)
• acoustic (pos.)
• additional complexity, cost

18
Tracking devices eye tracking
19
Tracking devices bend-sensing gloves

• CyberGlove7, 5DT
• Reports hand posture
• Gesture
• single posture
• series of postures
• posture(s) location or motion

20
Tracking devices pinch gloves

• Conductive cloth at fingertips
• Any gesture of 2 to 10 fingers, plus combinations
  of gestures
• gt 115,000 gestures

21
Case study Pinch Gloves

• Pinch gloves are designed to be a combination
  device (add a position tracker)
• Very little has been done with Pinch Gloves in
  VEs - usually 1 or 2 gestures for
• Object selection
• Tool selection
• Travel

22
Characteristics of Pinch Gloves

• Relatively low cost
• Very light
• Users hand becomes the device
• Users hand posture can change
• Allow two-handed interaction
• Huge number of possible gestures

23
Characteristics of Pinch Gloves II

• Much more reliable than data gloves
• Support eyes-off input
• Can diminish Heisenberg effect
• Support context-sensitive gesture interpretation

24
Pinch Gloves in SmartScene13

• Lots of two-handed gestures
• Scale world
• Rotate world
• Travel by grabbing the air
• Menu selection

25
Pinch Gloves for menus

• TULIP system14
• ND hand selects menu, D hand selects item within
  menu
• Limited to comfortable gestures
• Visual feedback on virtual hands

26
Pinch Gloves for text input

• Pinch Keyboard14
• Emulate QWERTY
• Pinch finger to thumb to type letter under that
  finger
• Move/rotate hands to change active letters
• Visual feedback

27
3D mice

• Ring Mouse
• Fly Mouse
• Wand
• Cubic Mouse
• Dragonfly

28
Special-purpose devices using conductive cloth

• Virtual toolbelt
• Used to select virtual tools
• Good use of proprioceptive cues
• Interaction slippers3
• Step on displayed options
• Click heels to go home

29
Special-purpose devices Painting Table4
30
Special-purpose devices ShapeTape11
31
Human input speech

• Frees hands
• Allows multimodal input
• No special hardware
• Specialized software
• Issues recognition, ambient noise, training,
  false positives,

32
Human input Bioelectric Control
33
Human input Body Sensing Devices
34
More human input

• Breathing device - OSMOSE
• Brain-body actuated control
• muscle movements
• thoughts!

35
Locomotion devices

• Treadmills
• Stationary cycles
• VMC / magic carpet
• Walking/flying simulations (use trackers)

36
UNIPORT

• First Locomotion Device For U.S. Army (1994)
• Proof-of-concept demonstration
• Developed in six weeks
• Difficult to change direction of travel
• Small motions such as side-stepping are impossible

37
Treadport

• Developed in 1995
• Based on a standard treadmill with the user being
  monitored and constrained by mechanical
  attachment to the users waist
• User actually walks or jogs instead of pedaling
• Physical movement is constrained to one direction

38
Individual Soldier Mobility Simulator (Biport)

• Most sophisticated locomotion device
• Designed for the conduct of locomotion studies
• Hydraulic-based locomotion driven w/ force
  sensors at the feet
• Safeguards limited responsiveness
• Too awkward to operate

39
Omni-Directional Treadmill15,16

• Most recently developed locomotion device for
  U.S. Army
• Revolutionary device that enables bipedal
  locomotion in any direction of travel
• Consists of two perpendicular treadmills
• Two fundamental types of movement
• User initiated movement
• System initiated movement

40
Torus treadmill
41
ODT video
42
Virtual Motion Controller17

• Weight sensors in platform sense users position
  over platform
• Step in direction to move that direction
• Step further to go faster

43
Walking in place18,19

• Analyze tracker information from head, body, feet
• Neural network (Slater)
• GAITER project (Templeman)
• Shown to be better than purely virtual movement,
  but worse than real walking20

44
Classification of locomotion devices/techniques
Virtual turning Real turning
Virtual motion Desktop VEs Vehicle simulators CAVE wand Most HMD systems Walking in place VMC
Real motion Stationary cycles Treadport Biport Wide-area tracking UNIPORT ODT
45
Input and output with a single device

• Classic example - touch screen
• LCD tablets or PDAs with pen-based input
• Phantom haptic device
• FEELEX haptic device21

46
PDA as ideal VE device?22

• Offers both input and output
• Has on-board memory
• Wireless communication
• Portable, light, robust
• Allows text / number input
• Can be tracked to allow spatial input

47
Conclusions

• When choosing a device, consider
• Cost
• Generality
• DOFs
• Ergonomics / human factors
• Typical scenarios of use
• Output devices
• Interaction techniques

48
Acknowledgments

• Joe LaViola, Brown University, for slides and
  discussions
• Ron Spencer, presentation on locomotion devices
  used by the Army

49
References

• 1 Matias, E., MacKenzie, I., Buxton, W.
  (1993). Half-QWERTY A One-handed Keyboard
  Facilitating Skill Transfer from QWERTY.
  Proceedings of ACM INTERCHI, 88-94.
• 2 Thomas, B., Tyerman, S., Grimmer, K.
  (1998). Evaluation of Text Input Mechanisms for
  Wearable Computers. Virtual Reality Research,
  Development, and Applications, 3, 187-199.
• 3 LaViola, J., Acevedo, D., Keefe, D.,
  Zeleznik, R. (2001). Hands-Free Multi-Scale
  Navigation in Virtual Environments. Proceedings
  of ACM Symposium on Interactive 3D Graphics,
  Research Triangle Park, North Carolina, 9-15.
• 4 Keefe, D., Feliz, D., Moscovich, T., Laidlaw,
  D., LaViola, J. (2001). CavePainting A Fully
  Immersive 3D Artistic Medium and Interactive
  Experience. Proceedings of ACM Symposium on
  Interactive 3D Graphics, Research Triangle Park,
  North Carolina, 85-93.
• 5 Bowman, D., Wineman, J., Hodges, L.,
  Allison, D. (1998). Designing Animal Habitats
  Within an Immersive VE. IEEE Computer Graphics
  Applications, 18(5), 9-13.
• 6 Hinckley, K., Pausch, R., Goble, J.,
  Kassell, N. (1994). Passive Real-World Interface
  Props for Neurosurgical Visualization.
  Proceedings of CHI Human Factors in Computing
  Systems, 452-458.
• 7 Kessler, G., Hodges, L., Walker, N. (1995).
  Evaluation of the CyberGlove(TM) as a Whole Hand
  Input Device. ACM Transactions on Computer-Human
  Interaction, 2(4), 263-283.
• 8 LaViola, J., Zeleznik, R. (1999). Flex and
  Pinch A Case Study of Whole-Hand Input Design
  for Virtual Environment Interaction. Proceedings
  of the International Conference on Computer
  Graphics and Imaging, 221-225.
• 9 Ware, C., Jessome, D. (1988). Using the
  Bat a Six-Dimensional Mouse for Object
  Placement. IEEE Computer Graphics and
  Applications, 8(6), 65-70.
• 10 Zeleznik, R. C., Herndon, K. P., Robbins, D.
  C., Huang, N., Meyer, T., Parker, N., Hughes,
  J. F. (1993). An Interactive 3D Toolkit for
  Constructing 3D Widgets. Proceedings of ACM
  SIGGRAPH, Anaheim, CA, USA, 81-84.

50
References (2)

• 11 Balakrishnan, R., Fitzmaurice, G.,
  Kurtenbach, G., Singh, K. (1999). Exploring
  Interactive Curve and Surface Manipulation Using
  a Bend and Twist Sensitive Input Strip.
  Proceedings of the ACM Symposium on Interactive
  3D Graphics, 111-118.
• 12 Froehlich, B., Plate, J. (2000). The Cubic
  Mouse A New Device for Three-Dimensional Input.
  Proceedings of ACM CHI.
• 13 Mapes, D., Moshell, J. (1995). A
  Two-Handed Interface for Object Manipulation in
  Virtual Environments. Presence Teleoperators and
  Virtual Environments, 4(4), 403-416.
• 14 Bowman, D., Wingrave, C., Campbell, J.,
  Ly, V. (2001). Using Pinch Gloves for both
  Natural and Abstract Interaction Techniques in
  Virtual Environments. Proceedings of HCI
  International, New Orleans, Louisiana.
• 15 Darken, R., Cockayne, W., Carmein, D.
  (1997). The Omni-directional Treadmill A
  Locomotion Device for Virtual Worlds. Proceedings
  of ACM Symposium on User Interface Software and
  Technology, 213-221.
• 16 Iwata, H. (1999). Walking About Virtual
  Environments on an Infinite Floor. Proceedings of
  IEEE Virtual Reality, Houston, Texas, 286-293.
• 17 Wells, M., Peterson, B., Aten, J. (1996).
  The Virtual Motion Controller A
  Sufficient-Motion Walking Simulator. Proceedings
  of IEEE Virtual Reality Annual International
  Symposium, 1-8.
• 18 Slater, M., Usoh, M., Steed, A. (1995).
  Taking Steps The Influence of a Walking
  Technique on Presence in Virtual Reality. ACM
  Transactions on Computer-Human Interaction, 2(3),
  201-219.
• 19 Slater, M., Steed, A., Usoh, M. (1995).
  The Virtual Treadmill A Naturalistic Metaphor
  for Navigation in Immersive Virtual Environments,
  Virtual Environments '95 Selected Papers of the
  Eurographics Workshops (pp. 135-148). New York
  SpringerWien.
• 20 Usoh, M., Arthur, K., Whitton, M., Bastos,
  R., Steed, A., Slater, M., Brooks, F. (1999).
  Walking gt Walking-in-Place gt Flying, in Virtual
  Environments. Proceedings of ACM SIGGRAPH,
  359-364.

51
References (3)

• 21 Iwata, H., Yano, H., Nakaizumi, F.,
  Kawamura, R. (2001). Project FEELEX adding
  haptic surface to graphics. Proceedings of ACM
  SIGGRAPH, Los Angeles, 469-476.
• 22 Watsen, K., Darken, R., Capps, M. (1999).
  A Handheld Computer as an Interaction Device to a
  Virtual Environment. Proceedings of the Third
  Immersive Projection Technology Workshop.
• 23 Zhai, S. (1998). User Performance in
  Relation to 3D Input Device Design. Computer
  Graphics, 32(4), 50-54.