Output Devices

1
Output Devices
2
Output Devices

• Visual
• Auditory
• Haptic

3
Visual Devices

• Dimensions
• Stereoscopic/monoscopic
• Image resolution
• Field of view
• Display tech
• Ergonomic factors
• Cost

4
Human Eye

• 126 million photoreceptors
• Eye-gaze technology not useful yet
• Calibration
• Error
• Overlaying control
• IPD
• 53-75 mm

5
Head-tracked Displays

• Displays in which the users head is tracked and
  the image display screen is located at a fixed
  location in physical space.
• Examples?
• CRT
• Virtual Workbench or ImmersaDesk
• CAVE
• Many large screen displays

6
CRT HTD (Fishtank VR)
7
Stereoscopic Display
8
Virtual Workbench
9
CAVE
10
Any Projected Large Stereoscopic Screen
11
Characteristics

• Large
• Projection-Based (except for Fishtank VR)
• Stereoscopic
• Head Tracked
• Stationary Display Screen(s)
• Lets try to identify the pros and cons and
  application domains

12
(No Transcript)
13
Large Screen Projection

• Infrastructure
• Front Projection (user occlusion)
• Back Projection (takes up space)

14
Large Screen Projection

• Disadvantages
• One person sweet-spot
• Real objects may occlude virtual objects
• Synchronization
• Light
• Reflection
• Bleeding
• Equipment footprint
• Alignment

• Advantages
• Viewer not isolated
• Collaboration
• Minimal physical gear
• Good resolution
• Large field of view

15
Head-Mounted Display
16
Visually Coupled Systems

• Operator uses natural visual and motor skills for
  system control
• Basic Components
• An immersive visual display (HMD, large screen
  projection (CAVE), dome projection)
• Tracking system for head and/or eye motion

17
HMD Optical System

• Image Source
• CRT or Flat Panel (LCD)
• Video, SeeThrough or Video-Pass Through

18
HMD Optical System

• Image Source
• CRT or Flat Panel (LCD)
• Video, SeeThrough or Video-Pass Through

19
Head-Mounted Displays Optical System

• Mounting Apparatus
• What are some factors?
• Eyeglasses
• Weight
• Earphones
• Trackers

20
Field of View
Monocular FOV is the angular subtense (usually
expressed in degrees) of the displayed image as
measured from the pupil of one eye.
Total FOV is the total angular size of the
displayed image visible to both eyes.
Binocular(or stereoscopic) FOV refers to the part
of the displayed image visible to both eyes.
FOV may be measured horizontally, vertically or
diagonally.
21
Focal Length Diopter

• Focal Length - The distance from the surface of a
  lens at which rays of light converge.
• Diopter - The power of a lens. Equal to 1/(focal
  length of the lens measured in meters)

22
Ocularity

• Ocularity
• Monocular - HMD image goes to only one eye.
• Biocular - Same HMD image to both eyes.
• Binocular (stereoscopic) - Different but matched
  images to each eye.

23
IPD

• Interpupillary Distance (IPD)
• IPD is the horizontal distance between a user's
  eyes.
• IPD is the distance between the two optical axes
  in a binocular view system.

24
Vignetting and Eye Relief

• Vignetting
• The blocking or redirecting of light rays as they
  pass through the optical system.
• Eye Relief Distance
• Distance from the last optical surface in the HMD
  optical system to the front surface of the eye.

25
Basic Eye
26
Properties of the Eye

• Approximate Field of View
• 120 degrees vertical
• 150 degrees horizontal (one eye)
• 200 degrees horizontal (both eyes)
• Acuity
• 30 cycles per degree (20/20 Snellen acuity).

27
Simple Formulas

• Visual Resolution in Cycles per degree (Vres)
  Number of pixels /2 (FoV in degrees)
• Example (1024 pixels per line)/(240 degrees)
  Horizontal resolution of 12.8 cycles per degree
• To convert to Snellen acuity (as in 20/xx)
• Vres 600/xx (20/47)

28
Optical System

• Move image to a distance that can be easily
  accommodated by the eye.
• Magnify the image

29
Simple Magnifier HMD Design
q
p
f
Image
Eye
Eyepiece (one or more lenses)
Display (Image Source)
1/p 1/q 1/f where p object distance
(distance from image source to eyepiece) q
image distance (distance of image from the
lens) f focal length of the lens
30
Virtual Image
Virtual Image
Lens
Display
31
Resolution

• (low) 160 x 120 color pixels per eye
• (high) 1000 x 1000
• Note that resolution and FOV are independent
• Another important factor pixel density
• Pixels per degree of FOV
• How can we use the make up of the eye to better
  leverage resolution?

32
Basic Eye
33
LEEP Optics

• Large Expanse Extra Perspective
• Very wide FOV for stereoscopic images
• Higher resolution in the middle of FOV
• Lower resolution on the periphery
• Pincushion distortion

34
Fresnel Lens

• A lens consisting of a concentric series of
  simple lens sections
• Result is a thin lens with a short focal length
  and large diameter
• More even resolution distribution
• Less distortion
• from lanternroom.com

35
Distortion in LEEP Optics
A rectangle
Maps to this
How would you correct this?
36
To correct for distortion

• Predistort image
• This is a pixel-based distortion
• Graphics rendering uses linear interpolation!
• Too slow on most systems
• Pixel shaders!
• Render to Texture

37
Distorted Field of View

• Your computational model (computer graphics)
  assumes some field of view.
• Scan converter may over or underscan, not all of
  your graphics image may appear on the screen.
• Are the display screens aligned perpendicular to
  your optical axis?

38
Distorted FoV (cont.)
39
Compound Microscope HMD Design

• Relay lens produces a real image of the display
  image source (screen) at some intermediate
  location in the optical train. The eyepiece is
  then used to produce an observable virtual image
  of this intermediate image.

40
Exit Pupil

• The area in back of the optics from which the
  entire image can be seen. Important if IPD not
  adjustable.
• Compound microscope optical systems have a real
  exit pupil.
• Simple magnifier optical systems do not have an
  exit pupil.

41
Virtual Research V8 HMD

• Display
• Dual 1.3 diagonal Active Matrix LCD
• Resolution per eye 640 x 480
• focal length 1m
• Optical
• Field of view 60 diagonal
• Solve
• What is the cycles per degree?
• What is its horizontal and vertical field of
  view?
• Pros/Cons

42
Characteristics of HMDs

• Immersive
• You are inside the computer world
• Can interact with real world (mouse, keyboard,
  people)
• Mask out real world (including body)
• Ergonomics
• Headborn weight
• Length of use
• Cue conflict
• accommodation vs. parallax
• Perspective
• Resolution and field of view
• Tethered
• Avatars

43
Exercise (Part of Quiz grade)Due March 26th
(Monday)

• Fill in the following table through research on
  the Internet

Virtual Research V8 lt500 HMD 500-10000 HMD gt10000 HMD best found for each category
Resolution (RGB pixels)
FOV (d, w, h)
Distance from eye to Virtual Image
Stereo?
Price
Weight
Pros
Cons
44
Hand Mounted Displays

• Binoculars
• 1280x1024
• 19900

45
Floor Supported Displays

• Articulated mechanical arm
• Offload weight
• 0.2 ms Latency
• High accuracy
• 0.1 degree orientation error
• Pole in the way
• Limited Space (6 diam, 3 hgt)
• Good FOV
• Good resolution (1280x1024)
• Fakespace Boom3C
• WindowsVR
• Differences
• Stereo
• Tracking
• Advantages

46
Desk Supported Displays

• Fishtank VR
• Autostereoscopic Displays
• Addresses weight fatigue
• Tracking?
• DTI 2018XL Virtual Window
• Elsa Ecomo4D

47
Monitor Large Volume Displays

• Multiple people
• Monitor Based LVD
• Fishtank VR
• With limited FOV, what is a possible solution?
• Exaggerate tracking
• More monitors (synch, bezel)
• Most use active stereo (shutterglasses)
• 29-32 transmittance
• 100-800
• Can be used for prolonged periods

48
Projector LVD

• Workbench
• CAVE
• 300-500k with SGI
• 100k with PC clusters
• Issues?
• Large wall displays

49
Sound Displays

• What are good goals for VR Sound Displays
• Compare importance to
• Movies
• Video Games
• What role does sound play?
• Interactivity
• Immersion
• Perceived image quality(!)
• Dimensions
• Mono/Stereo/virtual sound
• Tracker
• Occlusion
• What does it take to uniquely place a sound
  source?

50
3D Sound

• Azimuth Cues
• Interaural Time Difference
• head radius/(speed of sound(?sin ?))
• Interaural Intensity Difference
• High frequency
• Head shadow effect
• Elevation Cues
• How are ears modeled?
• If simple holes what is the problem?
• Pinna effects sound propagation
• Frequency attenuation and amplification

51
3D Sound

• Range Cues
• Intensity
• Motion Parallax
• As the user moves his/her head, the changes in
  azimuth
• Head Response Transfer Functions
• Two microphones at the ears
• Record sounds (HR impulse responses)
• Corresponding Fourier transforms (HRTFs)
• Depends on sound frequency
• No two people are exactly the same
• Given HRTFs
• Take sound
• Apply filter
• Get two signals that should replicate the sound
  to the user
• However whats the effect of using another
  persons HRTF?

52
Sound Display Hardware

• Offload compuational load
• Convolvotron
• Crystal River
• First virtual 3D sound Generator
• NASA 1988
• Real-time DSP
• Calculate new HRTFs given
• Sound
• Head position (tracker)
• Audigy
• Soundblaster EAX/ Aureal A3D
• Simple geometry
• Occlusion, reflection
• Tracking?
• High-end SDKs - 14k

53
Speaker Layouts

• Stereo
• Quad speakers
• 5.1 surround sound (6.1/7.1)
• Multiple users?

54
Haptic Feedback

• Hapthai touch
• Greatly improves realism
• When is it needed?
• Other cues occluded/obstructed
• Required for task performance
• High bandwidth!
• Why are hands and wrist the most important?
• High density of touch receptors
• Full body are still in research and not very
  usable (back, body)
• Two kinds of feedback
• Touch Feedback information on surface geometry,
  roughness, temperature, etc. Does not resist
  user contact
• Force Feedback information on compliance,
  weight, and inertia. Actively resists contact
  motion

55
Passive Haptics

• Not controlled by system
• Pros
• Cheap
• Large scale
• Accurate
• Cons
• Not dynamic
• Limited use

56
Active Haptics

• Actively resists contact motion
• Dimensions?
• Force resistance
• Frequency Response
• Degrees of Freedom
• Latency
• Intrusiveness
• Safety
• Comfort
• Portability

57
Human Haptic System Sensors

• Meissner and Pacinian
• Quick (high frequency) sensors (50-300 Hz)
• Vibration, acceleration
• Low resolution
• Merkel and Ruffini - Slow (low frequency)
  adapting (0-10 Hz)
• Constant, surfaces, edges
• High resolution
• Thermoreceptors temperature
• Proprioceptors/Kinesthesia
• Perception of own body motion
• Page 96

58
Tactile Feedback Interfaces

• Goal Stimulate skin tactile receptors
• How?
• Air bellows
• Jets
• Actuators (commercial)
• Micropin arrays
• Electrical (research)
• Neuromuscular stimulations (research)

59
Tactile Mouse

• Logitch iFeel Mouse
• Electrical Actuator
• Shakes up and down (do not disturb XY motion)
• Mouse over buttons
• Haptic Bump
• Rumble Pack

60
CyberTouch Glove

• CyberGlove
• Immersion Corporation
• Six Vibrotactile actuators
• Back of finger
• Palm
• Off-centered actuator motor
• Rotation speedfrequency of vibration (0-125 Hz)
• When tracked virtual hand intersects with virtual
  object, send signal to glove to vibrate
• 15000

61
Force Feedback Devices

• Attempts to stop the users motion if necessary
• Larger Actuators
• Grounded
• Mechanical bandwidth - perceived frequency of a
  haptic interaction (in Hz)
• Control bandwidth system control bandwidth to
  device

62
Force Feedback Joysticks

• WingMan Force 3D
• Inexpensive (60)
• Actuators that can move the joystick given system
  commands
• Max 3.3 N of force
• Force feedback driving wheel

63
SensAble PHANToM Arm

• 3 and 6 DOF force feedback device
• Most popular (1000) in VR research
• 3 DC motors
• Little interitia/friction
• 6.4 N (1.7 continuous)
• Control loop 1000 Hz
• 1500 Omni
• 16000 standard Desktop
• 56000 6 DOF high end

64
Other Force Feedback

• Large Forces
• Haptic Master Arm
• 250 N
• Sarcos Arm
• http//www.princeton.edu/jmelli/papers/ch4-haptic
  s.pdf
• Gloves
• CyberGrasp
• 39000
• CyberForce
• 56000

65
Why is it hard?

• Control bandwidth
• Mechanical bandwidth
• Forces
• Hygiene
• Gender differences
• Portability
• Encumbrance
• Solutions?
• Specific engineering