Audio Concepts in Plain English: 3D and I3DL2

1
Audio Conceptsin Plain English3D and I3DL2

• Scott Selfon
• Xbox Advanced Technology Group

2
Overview

• Whats so great about 3D?
• Simulating aspects of 3D sound
• Audio engine implementations
• I3DL2
• Advanced 3D Challenges for Games
• Q A

3
Why bother with 3D audio?

• Screen 45-90 degree visual window
• Whats happening in the other 270-315 degrees?
• Sound informs player of distances, obstacles,
  environment
• Competing with TV movie experience

4
Typical 3D Sound Presentation Priorities

• Location
• Distance
• Speed
• Geometry
• Sound emitter orientation
• World obstacles

5
Locating a Sound in 3D Space

• Speaker panning
• Cheap, can be effective
• Doesnt cover surround for 2-speaker scenarios
• HRTF Head Related Transfer Function
• Simulates 3D spatialization based on shapes of
  generic head and ears
• Filtering (shape of ears, sound waves not
  directly reaching ears)
• Delays (sound reaches one ear slightly before the
  other

6
Distance in the Real World

• Sounds attenuate (get quieter) with distance
• Approximately half as loud (-6 dB) for every
  doubling of distance
• Sounds muffled with distance
• Air is a low-pass filter
• Sound travels slowly vs. light
• 340 m/sec (vs. 300,000 km/sec for light)
• All values vary with humidity, altitude,
  temperature, etc.

7
Distance Simulation Volume

• Typical distance model (DSound, OpenAL)
• Minimum distance (where to start attenuating)
• Maximum distance (where to stop attenuating)
• Rolloff (how quickly to attenuate with distance)
• Others simplify to
• Minimum distance (where to start attenuating)
• Distance to silence (where sound is silent
  rolloff implied from here linear or log)

8
Distance Attenuation Example
9
Distance and Rolloff

• Rolloff factor adjust attenuation curve
  relative to Real World
• .5 Less rolloff (sound louder at given distance)
• 2.0 More rolloff (sound quieter at given
  distance)
• Rolloff also influenced by minimum distance
• Remember, half volume at doubled distance

10
Rolloff Factor and Attenuation
11
Minimum Distance and Attenuation
12
Rolloff Confusion

• Maximum distance isnt silence
• Sound not further attenuated
• Might abruptly stop voice (PC)
• Distance factor unrelated to rolloff
• Sets units (meters 1.0, feet 0.3048, etc.)
• Used for doppler shifting

13
Rolloff Model Challenges

• Unintuitive calculations to determine distance to
  silence
• spreadsheet to help
• Hard for sound designer to experiment without
  game-exposed controls
• Simplification options
• Replace max dist/rolloff with distance to
  silence
• Arbitrary rolloff curves

14
Simplifying Rolloff

• Distance to silence
• Trivial to calculate
• Arbitrary user-defined point curves
• Custom curvaturecontrol

15
Simulating Other Distance Cues

• Filtering with distance
• EAX supports
• Programmable LPFs in other engines can be used
  for this
• Delay with distance (speed of sound)
• Rarely simulated by engines
• Can programmatically delay sound triggers
• Perceived in games as lag?

16
Speed Perception in the Real World

• Doppler Effect
• Sound/listener moving closer pitched up
• Sound/listener moving apart pitched down

17
Doppler Effect Simulation

• Programmer updates listener/source velocities
• DSound Relative velocity used
• OpenAL Separate velocities used
• Distance factor
• Units do matter here (m/sec)
• Doppler factor
• Exaggerate (gt1.0) or lessen (lt1.0) effect

18
Sound Geometryin the Real World

• Few sounds are truly omnidirectional
• Most volume projected in one direction
• Few sounds are truly point sources
• Sounds interact with the environment
• Environment interaction (reverb)
• Filtered by obstacles

19
Sound Geometry Simulation

• Directional sound emitters
• Inside/outside sound cone angles
• Outer volume attenuation
• 3D spatial orientation for emitter
• Non-point emitter solutions
• Simulate with multiple points
• Simulate with relative positioning
• Pre-rendered multichannel
• More sophisticated engines

20
Environmental Simulation

• Simulation of sound propagation paths
• Direct-path signal reaches listener directly
  from emitter
• Reflected paths echoes off walls, obstacles,
  etc.
• Some delay versus direct-path

21
Basic Reverb Simulation

• Direct-path sound presented intact
• Distance attenuation
• Low pass filtering based on obstacles/distance
• Reflected path delays and filtering
• Often less noticeable distance attenuation
• Early reflections discrete echoes
• Late reflections reverberation (more dense,
  complex, decaying echoes)

22
Obstacle Simulation Obstruction

• Objects may block the sounds path to the
  listener
• Direct-path diffracted by or transmitted through
  obstacle
• Reflected pathgenerally notaffected

23
Obstacle Simulation Occlusion

• Sound is being played in another environment
• Direct, reflectedpaths bothfiltered

24
An Introduction to I3DL2

• Designed by Interactive Audio Special Interest
  Group (IASIG www.iasig.org)
• Interactive 3D Audio Rendering Guidelines, Level
  2
• Provides standard for simulating
• 3D positioning (attenuation, doppler)
• Environmental reverberation
• Occlusion/obstruction materials

25
I3DL2 Concepts

• Listener describes environment
• Timings for early/late reflections (Room)
• Differences between high and low frequency
  behavior
• Source per-emitter settings
• Occlusion/obstruction
• Exaggerate/reduce environments effect on this
  emitter

26
I3DL2 Environmental Reverb(Listener)

• Decay time time for reverb to fadeto -60 dB

27
I3DL2 Reverb Settings

• 30 pre-defined high level presets
• Generic environments (cave, bathroom, etc.)
• music reverbs
• or 12 low-level properties
• Adjust timings Reflections_delay, Reverb_delay,
  Decay_time
• Set volumes Room, Reflections, Reverb,
  Room_rolloff_factor
• Adjust behavior of high vs. low frequencies
  HF_reference, Room_HF, Decay_HF_ratio
• Reverb timbre Diffusion (graininess), Density
  (hollowness)

28
I3DL2 Sound Emitters(Source)

• Volume adjustment
• Direct-path level Direct, Direct_HF
• Room (reverb) rolloff factor and level
  Room_rolloff_factor, Room, Room_HF
• Occlusion/obstruction control
• 8 presets supplied (doors, walls, etc.)
• Volume attenuation Obstruction,
  Obstruction_LF_ratio, Occlusion,
  Occlusion_LF_ratio

29
I3DL2 Reverb and Occlusion/Obstruction Demos
30
I3DL2 Limitations

• Reverberation flexibility restrictions
• I3DL2 doesnt handle multiple simultaneous
  environments (listener only)
• Can be expensive (CPU, memory)
• Morphable reverb settings are hard
• No sense of game geometry
• Game must test for and set occlusion/obstruction
• Early reverb reflections are static,
  non-directional
• Reverb transitions

31
Advanced 3D Challenges

• Multiple players, one set of speakers
• Present sound relative to closest listener
• Reproduce sound for all listeners
• Doppler issues
• Overlayed soundscapes
• Speaker biasing? (P1left, P2right)
• Audio for complex geometry
• Rivers
• Non-cubical environments

32
Advanced 3D Challenges

• 3D integration with multichannel audio
• 4- vs. 5- channel 3D?
• Morphing 4.0/5.1 sounds to point sources?
• Multiple 3D sources presenting same audio (PA
  system?)
• Sync and Doppler challenges
• More sophisticated sound propagation simulation
• Complex, distant environments (pathfinding?)
• AI response to sound propogation (enemies can
  hear you)

33
Q A

• Resources
• OpenAL http//www.openal.org/
• DirectSound http//msdn.microsoft.com/
• I3DL2, IA-SIG http//www.iasig.org/
• http//www.midi.org/
• Audio physics resources
• http//hyperphysics.phy-astr.gsu.edu/hbase/sound/s
  oucon.html
• http//www.gmi.edu/drussell/Demos/doppler/doppler
  .html
• Questions, comments?