Auditory Displays

1
Auditory Displays

• Vause Grantham (1999)
• Wang et al (2002)
• Haas Casali (1995)
• Burt et al (1995)
• Belz et al (1999)

2
Theme of Research Articles

• Auditory displays provide benefits over visual
  displays
• Care should be taken in design of auditory
  warnings to combine qualities of acoustic urgency
  and situational urgency
• 3 D audio can be effectively implemented with
  server based COTS items but will require operator
  visual feedback training to compensate for lack
  of individualized HRTFs
• Auditory Icons can be employed
• Military and Industrial workers should carefully
  weigh the protective benefits of HPDs with helmet
  wear against the loss of localization
  capabilities and the likelihood/danger of
  front/back errors

3
Vause Grantham (1999)

• Background
• Sound localization critical to survival
• Essential to orientation/surveillance of
  environment
• Critical in military and industrial settings
• Helmets worn to protect head from injury
• Hearing Protective Devices (HPDs) worn to protect
  from hearing damage
• Previous Studies
• Early military helmet designs (Hayes-Stewart and
  M-1) did not disrupt localization performance
  Randall Holland (1972)
• Increased localization errors found when tanker
  helmets worn with HPDs Howse Elfer (1982)
• HPDs wear resulted in significant localization
  difficulties Abel Hay (1996), Atherly Noble
  (1970), Noble et al (1990) and Noble Russell
  (1972)
• Not known if helmet design (amount of ear pinnae
  occlusion), overall attenuation or
  frequency-specific characteristics of HPDs were
  factors in above results

4
Vause Grantham (1999)

• Purpose of Study
• What is the effect of wearing Kevlar helmet with
  no or one of two types of earplugs on sound
  localization ability?
• - Can any sound degradation be accounted for
  simply by the attenuation provided?
• What were the combined effects of the Kevlar
  helmet and earplugs on horizontal plane
  localization performance
• - Does HPD affect performance for a person
  wearing a helmet

5
Vause Grantham (1999)

• Participants
• 6 adults with normal bilateral hearing
• 5 active duty, Reserve, or ROTC cadet
• 1 university student
• Methods
• Subject seated in center of anechoic chamber
• Surrounded from 80 to 80 degree hemifield of
  43 speakers with 4 deg arc separation

6
Vause Grantham (1999)

• Methods
• 8 experimental conditions
• BH bare head/open ears
• EAR bare head/EAR ear plugs
• ER25 bare head/ER25 plugs
• K Kevlar helmet only
• KEAR Kevlar with EAR plugs
• KER25 Kevlar with ER25 plugs
• BL bare head/open ears at lower SPL of 35 dB
• BH2 2nd bare head/open ears
• DV Sound localization performance
• IVs protection condition and orientation to
  sound (front/lateral)

7
Vause Grantham (1999)

• Methods
• Stimulus digital recording of cocking of M-16
  rifle at 60 dB for all except barehead at 35 dB
• Participant facing front to center of speaker
  array or lateral with center of speaker array at
  left ear
• Stimulus emitted in random from one of 20
  speakers, 4 x per speaker
• 3 runs per orientation, 6 runs per earplug/helmet
  condition
• Called out speaker and was allowed to visually
  locate

8
Vause Grantham (1999)

• Results
• K no sig diff to BH no sig diff in front/back
  confusions
• EAR large sig diff in lateral
• ER25 sig diff in both frontal and lateral to BH
  sig diff in lateral to BL
• KEAR sig diff in lateral large increase in
  front/back confusions
• KER25 small but sig in frontal compared to K
  sig diff in lateral large increase in front/back
  confusions

9
Vause Grantham (1999)

• Conclusions
• Kevlar helmet by itself doesnt significantly
  affect sound localization performance compared to
  no helmet
• Earplugs by themselves decreased sound
  localization performance in the frontal
  orientation but was only significant for ER25
• In the lateral orientation, both types of
  earplugs decreased performance significantly
• Moreso for the EAR presumably due to disruption
  of spectral cues caused by differential
  attenuation
• Data couldnt answer conclusively whether the
  ER25 lateral disruption was accounted for by the
  flat attenuation
• Combining earplugs with a Kevlar helmet
  substantially increased localization errors in
  the lateral orientation through large increases
  in front/back confusions
• Small but not significant decrease in lateral
  orientation errors found in second bare head/open
  ears condition

10
Vause Grantham (1999)

• Further research needed
• Look at effect of long-term wear of HPDs with
  head protection and reduction of location errors
• Development and use of protective devices that
  offer both adequate hearing protection without
  sacrificing localization ability
• In the meantime, military and industrial workers
  may have to choose between hearing protection and
  localization performance

11
Wang et al (2002)

• Purpose
• To address limitations in use of Commercial
• off-the-shelf (COTS) 3 D sound cards in 3D
• auditory displays
• Development of a client/server systems utilizing
  COTS 3D sound cards
• Utilizing a method of visual-feedback training
  for 3D sound localization

12
Wang et al (2002)

• Background/Problem Set
• 3D audio beneficial for
• Spatial Localization outside of users field of
  view
• Reduce visual display clutter and cue
  interpretation time
• Differentiate multiple concurrent sound sources
• Custom-built 3 D audio systems tuned to listener
  and sound position and orientation (Head Related
  Transfer Functions, HRTF) are prohibitive due to
  time and cost

13
Wang et al (2002)

• Solution offered by COTS 3D sound cards
• Limitations
• Platform limited (Windows/Intel PC only)
• Resource limited
• No standardized application programming interface
  (API) that supports all features
• Vendor instability
• Lack of support for individualized HRTFs

14
Wang et al (2002)

• Rockwell Scientific Company (RSC) 3D audio server
• Integrates 3D audio into applications through
  TCP/IP
• Supports pre-generated or live sound sources
  through line input or streamed over IP
• Utilizes a common API between the client and
  server independent of the 3D audio system
• Eliminates need to program in multiple APIs
• Support added for new APIs without client code
  changes
• Computing resources not limited to users PC

15
Wang et al (2002)

• Proposal
• Method of visual feedback training to help
  compensate for lack of individualized HRTFs
• Without HRTFs, sound source localization accuracy
  is often degraded
• Listeners are provided with paired auditory and
  visual feedback to sound source location
• May help perceptually re-map to modified spatial
  cues

16
Wang et al (2002)

• Subjects wore Head Mounted Diplay with 6 dof head
  tracker
• Presented sound source via headphones using a
  COTS 3D audio card via the 3D audio server
• 144 spatial positions tested, 18 each from 8
  spatial regions
• Pre-test, training phase, post-test (4 days after
  training)

17
Wang et al (2002)

• Pre-test recorded subjects ability to judge
  position of sound source with generalized HRTF
  no feedback provided
• Training same as pre-test but with visual
  feedback to subject who corrected via cross-hair
  on HMD
• Post-test same as pre-test but 4 days post
  training

18
Wang et al (2002)

• Results
• Largest improvements in accuracy were seen in a
  reduction in front-back confusions
• Improvements lasted at least several days
• Conclusions
• COTS 3D audio use with server can open up
  opportunity for low cost availability
• Users can be trained via visual feedback to
  compensate for lack of individualized and costly
  HRTFs

19
Haas Casali (1995)

• Background
• Auditory warning signals can improve operator
  performance and reduce accidents
• Problems
• Mismatch between perceived and situational
  urgency
• Masking background noise
• Signal annoyance
• Other task-driven attentional demands
• Signal characteristics can improve urgency rating
• Lack of research on specific signal parameters on
  both subjective and objective performance

20
Haas Casali (1995)

• Purpose
• To investigate effect of pulse format, pulse
  level, and interpulse interval on the perceived
  urgency of warning signals
• To investigate the effect of these variables on
  the response time to warning signals
• Examine the relationship between perceived
  urgency and response time tow warning signals

21
Haas Casali (1995)

• Method
• 36 subjects (18 male, 18 female) with unimpaired
  hearing in 500 3000 Hz range
• Subject sat in center of diffuse sound field
  facing away from loudspeaker
• DV
• 1) magnitude estimation of signal urgency
• 2) response time in ms to warning signals
• 3) paired comparison ranking of signal urgency

22
Haas Casali (1995)

• IVs
• 1. Pulse Format
• Simultaneous 500, 1K, 2K, 3K Hz
• Sequential 500, 1K, 2K, 3K Hz
• Sawtooth frequency-modulated 500 to 3K
• 2. Interpulse interval
• 0 ms
• 150 ms
• 300 ms
• 3. Pulse level
• 65 dB
• 79 dB

23
Haas Casali (1995)

• Procedure
• Each participant underwent 4 sessions
• Audiogram
• Magnitude estimation (perceived urgency rating)
• Response time and probability monitoring task
• Paired comparison task
• Signals presented over 68 dB masking noise

24
Haas Casali (1995)

• As length of interpulse interval increases,
  perceived urgency decreases
• As pulse level increases, perceived urgency
  increases
• As pulse level increases, pulse interval
  perceived urgency increases
• Sequential pulse rated less urgent in all
  conditions
• Sawtooth pulse more urgent under all conditions
• 0ms interval rated more urgent at both pulse
  levels

25
Haas Casali (1995)

• Paired Comparion Data Pulse Format x Pulse Level
  Interaction
• Perceived urgency increases as pulse level
  increases
• Sawtooth and Simultaneous formats rated
  significantly more urgent than sequential at both
  pulse levels
• Sequential rated significantly less urgent

26
Haas Casali (1995)

• Response Times Results
• Pulse format and pulse level were significant
  factors in response time
• Response time was longer with sequential pure
  tones than with sawtooth and simultaneous
• No difference between sawtooth and simultaneous
• Decreased response times at high pulse levels

27
Haas Casali (1995)

• Higher pulse levels elicit higher perceived
  urgency and decreased response times
• Signals with shorter interpulse intervals
  elicited higher perceived urgency but did not
  significantly decrease response times
• Both sawtooth frequency-modulated and
  simultaneous pulse formats elicited both higher
  perceived urgency as well as better response times

28
Haas Casali (1995)

• Conclusion
• Future experiments should tease out how much of
  an increase in pulse is ideal (too much or too
  little) given background noise and attentional
  load
• Should look at even longer interpulse intervals
  that would allow speech commands/warnings in
  between
• Should look at the relationship between perceived
  and situational urgency to response rates

29
Burt et al (1995)

• Background
• Cockpit caution and warning signals designed to
  attract pilot attention/provide information about
  current or impending hazards
• Auditory warnings can have nuisance
  characteristics that make them less effective
• Most significantly is they lack a sense of
  situational priority particularly when elicited
  among other warnings

30
Burt et al (1995)

• 4 Hypotheses
• Subjects will be able to rank the perceived
  urgency of warnings
• Faster reaction times will occur during a manual
  tracking task and in response to the most urgent
  warning (perceived)
• EEG alpha and theta will increase during
  automated tracking and that an increase in alpha
  and theta will be seen in interaction of
  automated tracking to least urgent warning
• An increase in attentional engagement will be
  seen in an EEG and Event Related Potential (ERP)
  in response to the most urgent warning (perceived)

31
Burt et al (1995)

• Method
• Subjects wore elastic electrode head cap for EEG
  and ERP
• Session 1 rated urgency of warnings from least
  urgent to most urgent
• Session 2 rated urgency of warnings from zero
  (not urgent) to 100 (very urgent)
• Session 3 tracking and reaction time task with
  warning associated with system failure
  probability
• Tracking was either automatic or manual
• Session 4 rated urgency of warnings but
  instructed to ignore meanings given to warnings
  during tracking session

32
Burt et al (1995)

• Results Subjective Assessments
• Ratings of urgency level
• Significant difference between high and moderate
  urgency warnings and high and low urgency
  warnings
• Ranking of urgency level
• Subject rankings were in agreement with perceived
  urgency scales
• Post ratings changed

33
Burt et al (1995)

• Results Response Times
• Reaction times were significantly faster in
  response to both perceived and situational
  urgency during the automated tracking task
  compared to manual.
• Results EEG and ERP
• Automated tracking saw increase in alpha and in
  low and moderate perceived urgency signals
• Highest alpha in low siutational urgency and
  manual tracking
• Most alert response (beta) to high situational
  urgency during automated tracking
• Largest early ERP amplitude seen in response to
  moderal urgency warning
• Larget later ERP amplitude seen in response to
  high urgency warning during manual tracking and
  in low urgency during automated tracking

34
Burt et al (1995)

• Conclusions Hypothesis 1
• Subjects could rank perceived urgency based only
  on the frequency and harmonic components of
  signal
• Rankings were altered by task that reassigned
  rankings randomly not based on acoustic
  characteristics

35
Burt et al (1995)

• Conclusions Hypothesis 2
• Slower tracking times seen for both peceived and
  situational urgencies in manual task
• Conclusions Hypothesis 3
• Increased alpha EEG recording in automated
  tracking for both perceived and situational
  urgency signals
• Only situational urgency independent of tracking
  condition produced differences in alpha

36
Burt et al (1995)

• Conclusions Hypothesis 4
• Significant urgency level effects were found for
  EEG and ERP in the situational urgency
  manipulation while only a single interaction
  effect was found in the perceived urgency
• Situational manipulation of urgency produced
  significantly different physiological responses
  to the warnings

37
Burt et al (1995)

• Discussion
• In operational settings, sense of urgency of an
  auditory warning will be based upon its inherent
  acoustic properties and its situational context
• EEG and ERP are sensitive measures along both
  dimensions but moreso for situational urgency
  manipulations
• Need to investigate auditory warnings with a
  larger range of sound parameters
• Examine the value of direct urgency mapping
  compare group with correct mapping to group with
  inverse urgency mapping
• Use knowledge gained to design auditory warnings
  that promote optimal states of attention and
  awareness

38
Belz et al (1999)

• Looked at using auditory icons as effective
  warning signals
• Representational sounds that have specific
  stereotypical meanings defined by the objects or
  actions that created the sound
• Used impending front-to-rear and side collisions
  in driving simulator and identification of signal
  meaning
• Used male drivers with CDLs

39
Belz et al (1999)

• Front-to-rear collision scenario
• DV was response time initial and braking
• IVs
• Display presentation mode
• No display
• Dash-mounted visual only
• Auditory icon only
• Conventional auditory only
• Mixed auditory icon and visual
• Mixed conventional auditory and visual
• Vehicle speed and headway
• 35 mph
• 55 mph
• 2.5 s or 3.5 s headway

40
Belz et al (1999)

• Side Collision Scenario
• DV collision avoidance did or didnt
• IVs
• Dash mounted iconic display (yes or no)
• Mirrors (yes or no)
• Auditory Display (iconic or conventional)
• Speed (35 or 55 mph)
• Workload (low or high)

41
Belz et al (1999)

• Results
• Front-to-rear collision
• Auditory icons elicited significantly faster
  brake response times than conventional or no
  display conditions
• No significant difference between conventional
  and the no display condition
• Both multi-modal displays elicited faster
  response times than only the dash-mounted visual
  or the conventional audio alone
• No differences between auditory icon alone and
  multi-modal displays (with auditory icon)

42
Belz et al (1999)

• Results
• Side Collision Avoidance
• Significantly fewer collisions with auditory icon
  than conventional auditory warning
• Mirror used resulted in fewer collisions
• Significant effect of using visual display with
  auditory icon reduced collisions

43
Belz et al (1999)

• Results
• Correct Identification of signal meaning
• Approx half of participants could recognize
  conventional auditory signal meaning
• 96 could correctly identify auditory icon
  meaning
• Display preference
• Front to rear combined auditory and
  dash-mounted visual display
• Side Collison mirrors, auditory display and
  dash-mounted visual

44
Belz et al (1999)

• Research needed on workload effects of
  multi-modal displays
• Research in more realistic environment
  background noise effects, traffic congestion,
  time-pressure, etc
• Auditory icons as representational sounds show
  great promise for use as warning signals

45
Conclusions

• More research into situational and acoustical
  construct of warning signals needed
• Utilization of multi-sensory modes of
  warning/communication is the present
• Best fit should incorporate limitations imposed
  by hearing and head protection