Human Factors and Displays for VR Graphics

1
Human Factors and Displays for VR Graphics

• David Johnson

2
Goals

• Learn about
• Human vision
• Methods presenting images to users

3
Visual Bandwidth

• Bandwidth of vision is probably greater than
  other sensory modalities
• Retina bandwidth 10 Mbits/sec
• article
• Most important sense for VR?
• IMAX movie of rollercoaster can induce sensation
  of motion
• Images can induce sensations of taste/smell

4
Why Study Basis of Vision?

• If a display can match human capabilities, we are
  done
• If the display cant, nice to know where to
  concentrate resources.
• Maybe can use tricks to suggest a higher-quality
  display
• Optical illusions

5
Anatomy of Vision
6
Eye Anatomy
7
Blind Spot test

• test
• Test
• Brain fills in

8
Eye Optics
9
Accomodation

• Change in curvature of lens
• Rest focus from 6m to infinity

10
Light Sensors

• Cones
• Mostly in the fovea
• Blue, green, red/yellow cones, also called short,
  medium, long
• Rods
• Sense low levels of light

11
Visual Acuity

• Cycles per degree
• Like angular resolution
• Humans can resolve 0.93mm spacing at 1m
• Fovea is about 1 Mpixel
• 1 arc minute at fovea ( 1/60 degree)
• 20/20 vision letters are 5 arc minute letters
  (strokes 1 arc minute)
• Around 24K x 24K over field of view
• Retina is sensitive to light levels over 1013
  range
• Dynamic

12
Visual Field

• One Eye
• 120 degrees vertically and 150 degrees
  horizontally
• 60 to nose, 90 to side
• 50 up, 70 down
• Binocular 200 degrees horizontal
• Eye can rotate about 50 degrees

13
Binocular Vision

• Humans have 120 degree binocular overlap
• 2 40 degree monocular regions
• Depth disparity perception
• 0.05mm at 500mm
• 4mm at 5m

14
Eye Motion

• Vergence - the motion of the eye to maintain
  binocular vision
• Cross-eyed when focus in close
• Version eye movement in the same direction
• Duction motion of one eye
• Eye makes compensatory motions when head moving
• Shake hand vs. shake head
• Saccades eye movement to use fovea more
• Microsaccades imperceptible motions to maintain
  excitation of rods and cones

15
Frame rate

• 60Hz frame rate is generally considered important
• flicker fusion at 60Hz
• Old movies at low rates but people werent
  sensitized to it

16
Visual Displays

• How do we get computers to interact with the eye?

17
Overview

• display technologies
• head-mounted displays (HMD)
• projection-based displays

18
Overview

• Characteristics of displays
• field of view
• stereo display
• resolution issues
• brightness

19
HMD

• head-mounted displays (hmd)
• technology
• displays LCDs, CRTs, OLEDs, other
• totally immersive display experience
• tend to have small FOV (as compared to a more
  natural FOV)
• often bulky

20
HMD

• two displays provide image for left and right eye
• see-through displays used for augmented reality
• semi-transparent overlays or video see-through
• What did Brooks say was an advantage of video
  merge vs optical merge?

21
HMD characteristics

• what are important characteristics?
• image resolution, brightness, contrast
• tied to visual acuity
• field of view
• the mechanics
• ergonomics, mass, moments of inertia

22
field of view - FOV

• field of view - can mean different things
• optical field of view - field of view as
  specified by the HMD optics
• rendered field of view - field of view as
  specified in software
• Mapping between the two of them

23
field of view - FOV

• field of regard - the total area over which a
  user can view
• afford visual integration of larger space
• for HMDs this is generally 360 degrees
• provided the head is tracked
• for projection systems, generally closer to 180
  degrees
• CAVE varies

24
hmd display optics

• optics between the image plane and the users eye
  produce a virtual image farther away from the eye
• reduces accommodative effort
• ideally out a few meters to help cancel out
  convergence/accommodative rivalry
• optics magnify pixel granularity!
• other distortions?
• Leep optics radial falloff needs defocus

25
HMD displays

• CRT-based HMDs
• electron beam aimed at phosphorescent screen,
  resulting in emitted light
• generally good picture quality
• but often heavier and much more expensive
• Now rare

26
example of CRT-based HMD

• Datavisor HiRes, Datavisor 80

Datavisor 80 1280x1024, 3/pixel, accommodation
at infinity, 80 diagonal fov, 120 with 20
overlap, about 5lbs
Datavisor HiRes 1280x1024, 1.9/pixel, 42 fov
(100 overlap), about 4lbs, uses monochrome CRT
w/ color filter shutters
27
nvis nVisor SX

• liquid crystal on silicon (LCoS)
• LCD-type device by CRLOpto
• 0.9 microdisplay with 1280x1024 resolution
• horizontal fov 47, weight 2lbs
• good image quality, brightness, and contrast

28
LCD-based hmds

• in general, LCD HMDs have been less expensive and
  lighter
• however,
• usually, much lower resolution
• poorer image quality
• lower brightness, contrast
• improvements in LCD technology but VR just now
  catching up

29
LCD-based HMDs

• Virtual Research V8
• 640x480 resolution
• approx 45 fov (100 overlap)
• lightweight 2lbs
• inexpensive 12k

30
other HMDs

• Kaiser
• resolution 1024x768
• about 15k
• uses active matrix LCD screens
• 2.3/pixel
• 40 fov
• just over 2lbs
• OLED-based HMDs - better brightness than LCDs,
  800x600
• good image quality, lighter
• 5k, but with low fov (28)
• may have short life span

Kaiser ProView XL50
5DT HMD 800
31
Sensics

• First new HMD in some time
• Array of microdisplays
• OLEDs
• Panoramic field of view
• 2.9/pixel

32
setting up stereo

• monocular image viewed with one eye only
• bi-ocular both eyes see the same image
• binocular each eye sees its own image
• HMDs approximate stereo vision by showing a user
  left and right eye images
• what if the person cant fuse stereo?
• What about vergence?
• what about IPD?
• IPD inter-pupillary distance
• is this important?
• most HMDs do not provide enough control over the
  exact settings

33
perception and fov/stereo

• common explanations for problems in VR
• too small fov
• didnt use stereo
• stereo was configured improperly
• truth of the matter is that no one really knows
  in general for all situations
• requires specific experimentation with
• different setups, tasks, and applications
• FOV is likely important for speed of localizing
• stereo is likely very important for near field
  interaction

34
other pros/cons of HMDs

• definite advantages
• fully immersive
• disadvantages
• bulky, heavy, obtrusive
• poor resolution, mismatch between
  accommodation/convergence
• other comments?

35
projection displays

• setup a projector, aim it at a screen
• youre doing VR research!
• actually a little more complex than that
• goal is to surround user with the virtual
  environment
• good concept
• potentially reduces locomotion abilities
• capability to increase resolution dramatically
• tiling the display projectors

36
cave

• cave
• CAVE Automatic Virtual Environment
• developed at U Illinois-Chicago - 1992
• did you know CAVE is trademarked?

37
stereo with projection vr

• with HMDs, stereo vision is potentially easier
  (minus bad artifacts)
• two images, one for each eye
• for projection vr, you have two choices
• active stereo
• passive stereo

38
active stereo

• shutter glasses are required
• left and right eye shutters on the glasses
  synchronize with images coming from projector
• crystal eyes
• transmitter synced up with graphics/projector
  system
• turns right and left shutters on/off

39
ia state cave
40
passive stereo

• use the polarization of light to passively send
  stereo
• two projectors per screen, each with different
  polarizing filters
• left and right eyes of glasses allow the correct
  polarized light through
• potentially less expensive
• people just wearing sun glasses

41
important stereo issues

• generally, very expensive!
• requires extremely bright projection systems
• projectors must be capable of high refresh rates
  (120Hz) for active stereo
• bright projectors are required due to issue with
  light efficiency
• first off, about half of light is lost due to
  left/right switching glasses
• passive stereo - between 12 and 59 depending on
  setup
• Screen transmission

42
other painful issues with projection vr

• alignment matters
• for stereo setups, especially passive stereo,
  alignment is crucial
• for tile-able displays with high resolution
• images must overlap and blend

43
other painful issues with projection vr

• actually, should be able to project on any
  surface
• large set of research devoted to
• automatic alignment
• image blending, color matching
• projection onto arbitrary surfaces
• not just flat walls

44
Desktop VR

• Also known as fishtank VR
• Use computer monitor
• Track head
• Stereo from shutter glasses

45
vision dome and other spherics
46
Workbench
47
Autostereoscopic displays
48
Volumetric displays

• Actuality
• Rotating screen