3D Audio and Environmental Effects in SMIL

1
3D Audio and Environmental Effects in SMIL

• Jean-Michel Trivi jmt_at_atc.creative.com
• Jean-Marc Jot jmj_at_atc.creative.com
• Creative ATC, Scotts Valley, CA, U.S.A.

2
Outline

• Why ?
• What ?
• How ?

3
Existing SMIL audio features

• Temporal presentation properties
• Sound Volume
• AudioLayout module
• Region element
• soundLevel property
• Useful
• (example)

4
Existing SMIL audio features

• Temporal presentation properties
• Sound Volume
• Useful
• but not what we are used to

5
Limitations why do we care ?

• Poor audio experience compared to other common
  multimedia experiences
• Movies
• Games
• Music
• TV
• Restrict
• application fields
• perceived quality of content

6
YASM !?!

• Yet Another SMIL Module ?
• Goal extend the sound rendering capabilities of
  SMIL
• 3D sound / environmental effects
• Audio processing effects

7
3D sound and Environmental Effects

• Positional sound
• Optimization of rendering to playback device
• Artistic freedom
• Spatial sound source discrimination
• Environmental effects
• Reverberations
• Filtering
• More realistic Integration (SMIL)

8
3D sound

• Sound spatialization techniques
• ITD
• ILD
• HRTF
• Cross-talk cancellation

9
Environmental effects

• Reverberation model (I3DL2)

10
Environmental effects

• Reverberation example

11
Environmental effects

• Reverberation model
• Source rendering models
• Distance attenuation (air absorption)
• Frequency dependent directivity

12
Environmental effects

• High-level muffling effects
• Obstruction
• Exclusion
• Occlusion

13
Environmental effects

• Obstruction

14
Environmental effects

• Exclusion

15
Environmental effects

• Occlusion

Occlusion
16
Environmental effects

• High-level muffling effects
• Virtual worlds effect
• but also useful to convey sense of
  presence/absence.

17
Audio processing effects

• Used commonly in music
• Flanger
• Chorus
• Pitch shifting
• Distortion
• Adds to the artistic palette
• Music playlists

18
Module proposal concepts

• Egocentric positional audio
• Mixing console paradigm

19
Egocentric Positional audio

• SMIL is presentation oriented
• Doesnt target virtual world simulation
• One static listener
• (azimuth, elevation, distance)
• Easy to animate/author
• Constant distant motions

20
Mixing Console paradigm
21
Mixing Console paradigm
Sends to effects
Sources
Effect 1 Reverb
Effect 2 Chorus
3D
3D
3D
3D
mixer
22
Implementation

• 3D sound
• OpenAL
• MS DirectSound
• I3DL2 reverberator
• Creatives EAX
• MS DMOs
• Conformance TBD

23
Conclusion

• Proposal to enhance SMILs audio rendering
  capabilities
• Uses common audio concepts
• Effects
• Architecture
• Based on open I3DL2 specification

24
(No Transcript)