Human Physiology and Perception in Virtual Environment

1
Human Physiology and Perception in Virtual
Environment
2
Content

• 2.1 Physiology of visual perception
• 2.2 Physiology of auditory perception
• 2.3 Physiology of haptic and kinaesthetic
  perception
• 2.4 Virtual presence
• 2.5 Summary

3
2.1 Physiology of visual perception

• Most important channel to the Virtual Environment
• human very sensitive to any anomalies of images

4
2.1.1 The Eye

• Protected by the orbit (bony cavities in skull)
  and fatty material in the surrounding
• supported by 6 extraocular muscles, allows
• movements 50o to left an right
• 40o above and 60o below

5

• Others
• eyelid provide protection from bright lights
• cornea
• iris
• pupil 2 - 8mm in diameter, dilated in accordance
  with amount of light
• cystalline lens provides variable focusing
• retina photosensitive, convert electromagnetic
  radiation into impulses

6
Rod system

• Unevenly distributed in the retina
• Fovea area - absent
• outside - 15 000 and 170 000mm-2
• Several rods linked to a single nerve cell
• React to light of lower intensity
• Contain rhodopsin maximum sensitivity,
  dark-adpted for about 35 minutes

7
Cone system

• Responsible for bright light, colour and visual
  acuity (sharpness)
• most densely distributed in the fovea
• each cone receptor is connected to a single nerve
  cell
• Contain photochemical substance called iodopsin
• the blindspots a palce where there is lack
  photoreceptor

8
Optic nerve

• Connection to the brain (visual cortex of the
  brain)
• the nerves form the left and the right eyes
  intersect at chiasma

9
Adaption

• Human sensitive to extremely small variation in
  light over a wide range 1013 (from levels of
  just about perceptible to level of max. energy -
  can burn retina)
• light sensing device must be in the above range
• So far no man-made sensor within this range

10
Accomodation

• Resting eyes depth 6m - infinity
• pupil diameter will effect the depth fill
• the process of altering the curvature of the
  crystalline lens by means of the ciliary muscles
  is call Accomodation
• expressed in dioptres (the unit for specifying
  the refractive power of a lens)

11
2.1.2 Visual field

• The ability to see an object
• the object in the image of the retina
• where on the retina it appears
• charts constructed to show the visual field

12
Visual angle

• Dimensions of objects expressed in terms of
  visual angle - angle substended by the eye

13
2.1.3 Stereopsis (binocular vision)

• Both eyes share large portion of visual field
• relates to neural and physiological interaction
  of the two eyes in the overlap region
• if 2 very different images presented, the visual
  system often suppresses one of the images,
  occasionally alternates - binocular rivalry (br)
• br effect depends on size, brightness and hue
• users performance will be effected by br
• stereopsis in not always necessary for depth
  perception

14
Partial binocular overlap

• Alternative to binocular display
• display monocular image inwards or outwards to
  create partial binocular overlap
• can produce smaller and lighter optical system
  also an improvement in resolution
• Average human has a binocular overlap (between
  two eyes) of 120o with 35o monocular vision
  either side of the overlap region
• also consider the optics of the eye and the
  bridge of the nose

15

• binocular overlap can be convergent and divergent
• depending on the optical system being mounted
  inward or outward
• outward gtgt divergent system, inward convergent
  system

16
Binocular rivalry effects

• All observer have a dominant eye, brain
  suppresses the disparate image on one eye
• dominant eye, eye whose image is perceived for a
  greater period of time
• occurs in helmet-mounted display (HMD)
• one channel is brighter than the other
• display scene representation and complexity
• easier to control rivalry effects in closed
  helmet display (no direct view of the real world)
• harder to control rivalry in normal
  helmet-mounted displays that overlay outside
  world
• substantial amount of information can be ignored

17
Stereopsis Limit of depth cue

• 4 cues are responsible for giving depth
  perception
• lateral retinal image disparity
• motion parallax
• differential image size
• texture gradients

18
Lateral retinal image disparity

• The difference in relative position of the image
  of an object on the observers retinas as a
  function of the IPD(interpupillary distance) and
  vergence position
• depth perception
• lateral retinal image disparity in the range of
  0o-10o
• gt 0o-10o results in double images (diplopia) and
  the sensation of depth is lost

19
Vertical retinal image disparity

• Occurs when vertical displacement of the retinal
  image in one of the eyes
• limits the horizontal movement of the eye
• vertical disparities do not convey any depth
  information to the observer
• small amount can lead to diplopia (most users can
  adapt to this after 15-20 mins)
• the user need to readapt to the real world
• accounts for some criticisms of the VE system

20
Image rotation misalignment

• A gradual increase of rotation misalignment
  between left and right eye channels quickly
  reaches the point where double images occur
• Can cause
• visual fatigue
• image blurring
• tears
• queasiness

21
Image magnification induced retinal disparity

• Slightly magnified image in one eye can cause
  several disparities depending on the nature of
  the magnificatin
• known as aniseikonia

22
Stereoacuity

• The ability to resolve small differences in depth
  between two object and is expressed in terms of
  visual angle
• must eliminate other cues such as monocular depth
  cues, inter-position and perspective

23
2.1.4 Visual motion perception

• Not properly understood
• can be convincing if the observer is static

24
Motion parallax

• Refers to the relationship between objects in an
  observers field of view as the observer moves in
  relation to the objects
• does not matter if the observer is moving
  relative to the scene or vice versa the visual
  effects do not alter

25
2.1.5 Temporal resolution

• The eye respond to changing or varying light
  levels in order to give dynamic representations
  of a scene

26
Perception of flicker in display device

• Majority of displays produce images that are
  derived sequentially in raster-like fashion
• so observer may perceive the display to be
  flickering on and off

27
2.1.6 Spatial resolution

• Display structure can effect the perceived eye
  level
• Spatial frequency response of a display system
• an optical system can increase the amount of
  blurring produced by a display system
• blurring can be caused by a small point source in
  the image plane being reproduced as a spot of
  finite size
• Spatial frequencies relationship to frame rate
  and bandwidth
• the video bandwidth of a CRT(Cathode Ray Tube) or
  matrix display is proportional to the frame rate
  and the total number of pixels.
• Slow frame rates can increase the perception of
  flicker

28
2.1.7 Visual Space perception

• Surroundings provides a framework for the
  perception of spatial relationships between
  object
• best define axis set more centred on the observer

29
Constancy scaling

• Misleading effect on distance perception
• examples Delboeuf, Judd, Lipps, Muller-Lyer,
  ponzo...

30
2.1.8 Colour perception

• 3 attributes applied to colour perception
• Hue the correct term to describe a colour by
  name
• Saturation describe the amount of purity or
  proportion of pure chromatic colour in the
  perception
• Brightness/Intensity the degree of a luminous
  light source

31
Intensity

• The appearance of a coloured object depends
  greatly upon a few conditions
• a complex subject

32
Test for colour perception

• Test includes
• simple matching of lightness or hue of one test
  objects to another
• comparisons involving coloured objects on
  coloured background
• others
• achromatic response- exhibits a certain behaviour
  for a wide range of wavelength usually from blue
  to red (white light response)

33

• monochromatic - a narrower spectral response over
  a few tens of wavelength (result in a single
  colour display - incorrect to describe a black
  and white display)
• a coloured object when viewed under incandescent
  light will appear to have different hue,
  lightness and chrome, compared to natural
  daylight

34
Colour specification

• Definition subjective
• more accurate to plot the radiance of the light
  source as a function of wavelength
• for transparent objects such as coloured filters,
  the spectral transmittance distribution is
  usually specified

35
2.2 Physiology of auditory perception

• Extremely impressive sensory system
• responsive to a wide range with a sensitivity of
  0.2
• the ears of an average young person are sensitive
  to all sounds from about 15 Hz to 20,000 Hz
• the frequencies to which the ear is most
  sensitive (about 1,000 to 2,000 Hz)
• upper capability of around 110dB(unit to measure
  the intensity/loudness of sound)
• a sensitivity of a fraction of decibel at all
  intensity levels is something that no current
  electronic system can easily emulate

36
TO synthesize realistic auditory environment

• take into account
• acoustic environ and deal with aspects such as
  echo or reverberation
• position and orientation of the listeners head,
  the source position (head related transfer
  function, HRTF)
• phase differences
• overtones
• 2 pathways for the reception of sound normal air
  conduction route via ears, the bone conduction
  route in the head
• total auditory perception the routes of sound
  the masking effects caused by the head
  inter-aural time delay between two ears

37
2.2.1 Hearing

• Divided into 3
• the external
• middle
• internal
• external collects sound waves with pinna
  (auricle) gtgt into external auditory canal gtgt
  tympanic membrane (eardrum)

38
External ear

• The pinna elastic-like cartilage coverd by a
  thick layer of skin
• acts as a linear filter whose transfer function
  is dependent upon the direction and distance of
  the sound source
• code the spatial characteristics of the sound
  field into temporal and spectral attributes
• experiences properties such as diffraction,
  dispersion, interference, masking, reflection and
  resonance
• the external auditory canal 2.5 cm long, wall
  composed of bone lined with the same cartilage
  material of pinna, surface covered with thin,
  highly sensitive skin
• the tympanic membrane semi transparent layer of
  connective tissue that separates the external
  auditory canal and the middle ear
• vibrates according to sound, external ear
  teminates here

39
Middle ear

• Small air-filled cavity (epithelial lined)
• one end is the tympanic membrane and the other
  end are two openings - oval and round windows
• also contain auditory tube that connects with the
  throat
• also contain small auditory assicles known as the
  malleus, incus and stapes
• malleus connected to tympanic membrane and gtgt
  vibrations to incus gtgt stapes gtgt oval window

40
Inner ear

• Complicated series of canals
• an outer bony labyrinth
• consists of series of cavities, the vestibule,
  the cochlea and semicircular canals
• an inner membranous labyrinth
• tympanic membrane as an amplifier, increasing the
  auditory signals to the oval window gtgt into the
  perilymph fluid of inner ear
• vibration in perilymph cause the oval window to
  vibrate accordingly with the auditory signal
  gtgtcauses pressure to vary in the endolymph of
  cochlea gtgt causing hairs of the basilar membrane
  to move against the tectorial membrane gtgt causes
  the generation of nerve impulses gtgt travel to
  midbrain, the thalamus gtgt finally to auditory
  region of the temporal lobe of the cerebral cortex

41
2.2.2 Auditory localization

• The perception of where the sound is coming from
  - the pinnae palys an important role
• 2 mechanism involve in our localization of sound
• Intensity differences
• operate for audio tones whose frequency is above
  1.5kHz
• the higher the frequency, the greater the effect
  of head masking
• relative intensity can approach 20 dB attenuation
  in one ear compared to the other
• Temporal differences
• an interaural delay as small as 10 µs can be
  detected by some individual
• an interaural delay in excess of 650 µs can be
  sufficient to localize a sound

42
Cont.

• Audio localization is a function of
• The difference of sound reaching the two ears
• the differences in sound relative to the
  directions and range relative to the observer
• monaural cues derived from pinna
• Head movement
• localization understood in the horizontal plane
  (left to right) but very little is known for
  median plane and front to back
• possible for sound to arrive 700 µs earlier in
  one ear, can be attenuated as much as 40 dB, the
  interaural differences is negligible at distances
  grater than 1m

43
Cont.

• Interaural differences 0, if sound source
  exists directly in front of (or behind) the
  listener
• even if vertical position is raised or lowere
  from the central position
• if head is moved the sound is put outside th
  emedian plane, resulting in interaural
  differences
• familiar sound distinguished because of the
  differential effects of the pinna
• A sound is generally softer when originating from
  the back compared to the front
• timbre is the name given to distinguish 2
  auditory signals that have the same pitch and
  intensity
• spectral composition of the sound source
• brightness, mellowness and richness
• depends on the envolope of the sound signal and
  the rate of amplitude modulation

44
2.2.3 Frequency analysis

• Signals mixed into complex audio signal
• in order to hear the individual components, the
  frequency must be sufficiently separated from one
  another - frequency selectivity
• When the component frquencies are too close, the
  signal is perceived as a single component the
  Ohms acoustic law
• the ear can be said to act as a series of
  narrowly tuned filters (oversimplification)

45
2.2.4 Pitch discrimination

• Human sensitive to sound pitch variation in the
  range of 1kHz to 3 kHz
• perceptible frequency difference by average human
  is 0.3 (change fromm 1000Hz to 1003Hz)
• this figure can be improved gtgt musicians
• At lower frequency on range of 32 Hz to 64 Hz,
  the perceptible frequency difference is in the
  order of 1
• At higher frequencies 16kHz to 20 kHz pitch
  discrimination is extremely poor
• average human can resolve about 2000 pitch
  variations
• musiciangtgt pitch depends on intensity and
  frquency
• pitch is not a simple function of frequency

46
2.3 Physiology of haptic and kinaesthetic
perception

• Haptic- concerned with the sense of touch or
  force on the body)
• Kinaesthetic - awareness of where body parts an
  limbs are in space, both statically and
  dynamically

47
2.3.1 Physiology of touch (cutaneous sensitivity)

• Mechanical contact with skin
• contact, vibration, sharp, pressure
• touch more acurately calledcutaneous sensitivity
• complex
• must consider other stimuli such as heat
• skin 1-2mm below the eperdermis (outer layer)
• local disturbance distributed, thereby
  attenuating the sensation of touch
• certain region of skin more sensitive to touch
  than others

48
Anatomy of the skin

• Three mechanical stimuli that produce the
  sensation of touch
• step function a displacement of the skin for an
  extended period of time, for example a small
  point resting on the surface of the skin
• Impulse function a transitory displacement of
  the skin that lasts for a few milliseconds
• periodic functions transitory displacement of
  the skin that is repeated regularly at constant
  or variable frequency
• skin has certain thresholds for touch
  (vibrotactile thresholds)gtgt depends on
• the type and position of the stimuli (where on
  the body)
• temporal summation (the frequency of stimuli)
• Rapidly adapting receptor (phasic) pressure,
  touch and smell
• Slowly adapting receptor (tonic) pain and body
  position

49
Cont.

• Some sensation have after-images, which persist
  when the stimulus has been removed
• Some sensation may disappear, even when the
  stimulus is still being applied
• Cutaneous receptors consists of dendrites of
  sensory neurons that can be enclosed in a capsule
  of epithelial or connective tissues
• The nerve impulses generated by the cutaneous
  receptors gtgt somatic afferent neurons in the
  spinal and cranial nerves gtgt thalamus gtgt the
  general sensory area of the parietal lobe of the
  cortex
• neural pathway for touch, light and pressure
  anterior (ventral) spinothalamic pathway gtgt
  ventral posterolateral nucleus of the thalamus
  (some sensation of light touch and pressure) gtgt
  cerebral cortex (fully localized)

50
Cont.

• Light touch ability to perceive that something
  has touched the skin
• discriminative touch ability to localize exactly
  the point of touch
• skin
• tactile receptors
• hair root plexuses detect movement on the
  surface of body, hair as simple lever
• free nerve endingsdetection of pain, object
  continuous contact
• tactile discs assist in discriminative touch
• corpuscles of touch(numerous in the fingertips
  and palms of the hand) perception of
  discriminative touch
• type II cutaneous mechanoreceptors heavy and
  continous touch sensation
• Cutaneous sensitive nerve fibres
• SA1, SA2 slowly adapting nerve fibres -
  responsive to an object that comes into contact
  with the skin
• RA1, RA2 fast adapting nerve fibres - respond on
  the movement of the skin

51
Cont.

• To construct device to communicate the sensation
  of touch to a user
• must be aware of the dynamic range of touch
  receptors and adaption to certain stimuli
• too easy to disregard the fundamental
  characteristics of the human body

52
Proprioception/kinaesthesia

• Awareness of the movements and relativepositions
  of various part of the body
• takes into account
• the rate and movement of our limbs
• the static position when movement ceases
• so can estimate the weight supported by the limbs
  / the force exerted by the contaction of our
  muscles
• human can remember a particular joint position
  even after 24 hours of an event
• visual cues help in kinaesthetic judgement
• muscles receptors awareness of static limb
  position
• muscles receptors vitaneous receptors
  awareness of limb dynamics

53
Interstingly

• lack of sense of static position of the finger
  joints -
• this comes from cues of movement signals from the
  muscles and the skin receptors and
• non-kinaesthetic cues
• joints closer to body insensitive to small angles
  compared to distal joints
• also consider the velocity of individual joints
• small rate movement might be too small for
  perception
• subject not fully understood for example tnesing
  ones muscles improve movement sense

54
Cont.

• Human can gauge or measure the force produced by
  the contraction of certain muscles by 2
  mechanisms
• sensory inputs from tension receptors provide an
  awareness of mucscle tension
• command signals generated in the brain are
  monitored and provide a sense of effort
• term light or heavy perception of the weight
  of an object depends on weight of object and the
  condition of the muscles
• effect known as weight expectancy illusion
  when heavy object is lifted prior to lifting a
  lighter object gtgt underestimation of the weight
  of the lighter object

55
Cont.

• Human detection of joint angle in range of 0.20o
  - 6.10o
• order of decreasing sensitivity of the joints
• hip, shoulder, knee, ankles, elbow, knuckles,
  wrist, the last toe metacarpophalangeal joints,
  toes
• human also possess an internal image of the
  positions of our limbs/joints that does not
  depend on sensing information memory capability
  for limb position
• mental capability can be fooled by the effect of
  surroundings

56
Mechanoreceptors

• Cutaneous mechanoreceptors partly responsible
  kinaesthesia gtgt
• skin stretches and contracts as we move
• differentiation in cutaneous mechanoreceptors
  over different regions of body hands, feet and
  face more sensitive than skin around elbows
• does not convey a sense of joint position
• joint mechanoreceptors found in ligaments and
  capsules of joint and adapt slowly to stretching
  of ligaments and compression of capsules
• exception when hyperextension of a joint occurs
• Muscles mechanoreceptors most dominant mechanism
  for kinaesthesia

57
Muscles spindles and Golgi tendon

• Muscles spindles Sensitive to limb position and
  movement
• lie in parallel with the contractile elements of
  muscle tissue
• can determine muscles stretch and rate of
  increase in muscle length
• Golgi tendon organs detecting tension
• lie in series with the tension producing muscle
  elements
• produce an indication of tension developed in the
  muscles

58
2.4 Virtual presence

• Two types of feeling in virtual presence
• Feeling of being part of a synthetic experience
• feels immersed to some extend but also aware of
  the outside world
• not a complete immersion
• such as some of the virtual environment games
• other factors also contribute
• poor graphics, cartoon type environment,
  hand-held button ...
• Feeling of being immersed in the synthetic
  experience
• felt part of the actual environment
• can take palce slowly
• visual cue important to gain sense of presence

59
Breakdown of virtual presence

• When
• person feels tired
• head-mounted display becomes uncomfortable
• unnatural movements or lags in VE
• visual or auditory information not realistic
• this subject not fully understood
• need to be aware of the way human being interacts
  with a real world and the adaptation that takes
  place when things change in the real or external
  environment. Ellis (1991) states that
• knowledge constantly being updated by behavioral
  plasticity of visual-motor coordination and
  vestibular reflexes
• large part of our sense of physical reality gtgt
  internal processing NOT from immediate sensory of
  info we receive

60
Cont.

• Important to
• Match our virtual environment peripherals to the
  operator perceptual system
• how we present information to the operator
• Sheridan (1992) states that
• presence is subjective sensation like mental
  workload and mental model
• likely to be multidimensional
• Presence - subjective or objective measures? Have
  not been determined!

61
Seeing parts of ones own body

• Seeing parts of ones body reinforce the feeling
  of presence
• strong feeling of presence if visual system
  allows actual parts of operator to come into
  field of view at the appropriate time.

62
High resolution and large field of view

• Must no have the framework of display visible gtgt
  can cause conflict in the sense of presence
• operator/user should be able to move / see things
  in the environment as in the real world
• restrictive view can reduce the feeling of
  presence

63
Familiarity of virtual environment of scene

• time taken to adapt to VE shorter if VE relates
  to real world
• depends on the VE system as a whole
• Zeltzer (1991 and 1992) states that, VE has 3
  components
• a set of models/objects or processes
• a means of modyfying the states of these model
• a range of sensory modalities to allow the
  participant to experience the VE

64
Zeltzers cube

• The cube (scaling 0 - 1)
• Autonomy refers to qualitative measure of the
  virtual objects ability to react to events and
  stimuli.
• Interaction refers to the degree of access to
  the parameters or variables of an object.
• 0 non real time control of variables
• 1 variables that can be manipulated in real time
  during program execution
• Presence crude measure of the fidelity of the
  sensory input and output channels.
• Dependen upon the task requirements application
  has a bearing

65
Determinants of presence

• Sheridan (1992) proposed 3 determinants another
  one
• Extent of sensory information
• Ability of the observer to modify theor viewpoint
  for visual parallax or visual field. Includes the
  ability to reposition the head to maintain
  binaural hearing
• Ability to modify the spatial relationships of
  objects in the virtual environment
• The closed loop performance due to an
  opertor-induced motor movement. Also includes the
  dynamic behaviour of movable objects in the
  virtual environment
• task to be performed important not just the
  visual displayed
• matching devices to performance of human sensory
  gtgt difficult

66
Summary

• Human factors principles in VE
• Key factor is to develop a series of metrics to
  measure human performance
• subject of human perception is vast