Dynamic Range

1
Dynamic Range

• A Study of Software DSP and Runtime Mixing
• Rob Bridgett

2
Overview

• Why have games suffered from lack of dynamic
  range up until now? Technical, Production and
  Aesthetic
• Technical Console Overview
• Interactive Mix Architecture (snaphot mixer
  systems and DSP routing)
• Production Sound Alpha and Sound Beta
  Production Getting quality and polish out of
  mixing features
• Aesthetic Dynamics
• Case Study Scarface The World Is Yours (2006)
• Compressing the final output of a video game
  does the type of game make a difference?
• Beyond the Technical Some thoughts on the Future
  of Game Audio (Randy Thom Interview)
• Questions

3
Why have games suffered lack dynamic range?

• Those who have worked on PS2 titles have
  encountered situations where sounds need to
  constantly be made louder in order for them to be
  intelligible. (Dialogue)
• Due to a lack of any kind of runtime DSP
  (compression). Means destructive processing on
  the files themselves.
• Limiting (e.g. Waves L1)
• Compression (e.g. Reducing the dynamic range)
• Reasons Consoles such as PS2 (most developers
  target platform), etc suffered from severe
  hardware limitations...

4
Platform Comparison

• The ps2 outputs audio at true consumer line
  level, barely -10db max
• most other consoles are 0Db (semi-pro) or even
  4Db (pro).
• On PS2 Sony pads all of its ADPCM voices (used
  for everything except movies) by half amplitude,
  or -3 db, so that brings the maximum output of an
  in game sound to -13db.  This means in-game
  music and non-3d sfx that are cranked in the game
  will only hit -13db.
• Depending on who implemented surround sound,
  usually the calculations impose constant volume
  curves as the sound pans between the speakers. 
  This means that when the sound is directly in
  front of the listener each speaker is at half
  volume.  This adds another -6db for a whopping
  maximum reduction of -19db max
• This wasnt taking into consideration roll-off
  over distance, so at this point our roll off is
  from -19Db to infinity (or noise floor).
• Also we tend to down sample more on the ps2,
  which makes it sound quieter because were
  missing half the spectrum.
• The Xbox or PS3 has none of these problems
  because they have high quality compression
  algorithms, output at pro audio level, and we do
  software mixing so we could clip the crap out of
  the system if we wanted to
• If you do the calculations, compared to a console
  outputting 4db (pro line level), the ps2 will
  sound about 3 times quieter which is why
  destructive limiting on the files was so
  important to the overall output.

5
Lack ofRuntime Mixing

• In-Game mixing has not had chance to develop due
  to the production process of game development.
• Traditional Alpha and Beta production methods
  mean that audio is cut off at exactly the same
  time as art and design. Thus no polish time.
• A proliferation of pre-rendered FMV in games has
  meant that pre-rendered movies have been mixed
  professionally by film-sound studios, but in-game
  dynamics and mixing have not been considered.

6
The Results?

• Individual sound assets are over-compressed and
  limited.
• Has the effect of crushing the overall dynamic
  range of game-audio.
• FMVs sound great but make in-game sound weak by
  comparison.
• Game sound is crunched MUCH louder than movies.
• This is starkly revealed when compared to a
  movie-mix

7
Recent Motion-Picture DVD MIX5.1 Dolby Digital
8
PS2 Game Waveform(Pro Logic II)
9
X360 Cinematic In-game(5.1 Dolby Digital)
10
Interactive Mix Architecture

• Ability to gracefully adjust volumes in real-time
  is clearly missing from the old model.
• Interactive Mixing requires that volumes for all
  sounds can be set for different events in the
  game.
• Snapshot Mixes allow groups of busses to be moved
  at runtime to coincide with in-game events.

11
How Do Mixer Buses Work?

• Every single sound is sent to a particular bus.
  (e.g. M16 to weapons bus)
• Busses are arranged on a mixing panel.
• Different snapshots of the mixing panel are
  used to define the volumes throughout the game

12
Other Parameters

• Other Parameters for the Mixer Snapshots to
  contain (other than Volume) include
• Amount of LFE
• Pitch
• Fade-in and Fade-out times of snapshot
• Sends to Reverbs or other DSP
• Hi and Low Pass Filters

13
Mixer Snapshot (Default)
14
Mixer Snapshot (Duck Event)
15
Mixer Snapshot (Return to Default)
16
Fade In and Out
17
Project Management

• Traditional Game Development models follow Alpha,
  Beta and GMC dates.
• Sound is actually expected to check-in final
  changes at the same time as art, design tuning
  and scripting.
• The problem with this is that brand new art
  assets tuning that changes the feel of the game
  can completely change the way sound has been
  implemented.
• This leaves no time for audio to be addressed
  (with the exception of critical bugs) therefore
  QUALITY will always suffer.

18
Post-Production Model

• If audio follows a post-production movie model
  where sound is allowed time to work AFTER the art
  and gameplay are locked quality can be greatly
  improved.
• Sound Alpha This allows offline work such as
  Foley for cinematic cut-scenes to be done in good
  time.
• Sound Beta Allows two or three weeks for
  Post-Production Sound Design and a full in-game
  mix to be carried out to balance all elements of
  the game.

19
Example Schedule
20
Post-Production Mix

• Advantages of a final mix are immeasurable to
  overall quality.
• Allows balance of overall output of the game
  ensure it is mixed at reference levels.
• Allows for moments of quiet and loudness to be
  implemented correctly (narrative dynamics)
• Allows for MUSIC, SFX and DIALOGUE to be
  considered, balanced and emphasized at various
  moments throughout the game. (fresh ears
  critical) Strengthens direction.

21
Aesthetic Dynamics

• Previously talked only about technical reasons
  for lack of dynamic range. Look at Aesthetic
  challenges.
• Story and game design must incorporate dynamics
  i.e. quiet moments preceding louder moments.
• Most games have constant combat or game play
  input. E.g. shooting. Game Play Narrative
  Dynamics are very often an afterthought.
• Working with designers early on a project means
  dynamics can be considered. In the story, in the
  level design and in the overall flow. This
  provides opportunities to make game play moments
  more powerful.
• Special modes or features such as bullet-time
  can be used as dynamic events
• Think at the snapshot mixer level about all the
  game events and draw up a timeline.
• Horror movies provides an excellent model of how
  dynamics of sound (quiet) and image (darkness)
  can draw the audience into a vulnerable situation.

22
Case Study Scarface

• PS2, Xbox and PC (2006)
• Post-Production Sound Design done by Randy Thom
  real-time sound effects replacement.
• Game was mixed over a three-week period at
  Skywalker Ranch
• 2 weeks mixing PS2 (main platform) in pro logic
  2, a further week of Xbox Mixing (Dolby Digital)
  (Also cloned for PC)

23
Mixer Snapshot Editing System

• A fast and reliable system of mixing was devised
  for the game to be mixed by a motion picture film
  mixer (Juan Peralta)
• Mackie Control surface was hooked up to the
  software mixing system in turn hooked up to the
  game running in real-time.

24
Mackie Control
Demo Mix Movie (Strong Language
25
Mackie Control Mixing Demo Movie
26
Scarface Dynamic Range (Xbox)
27
DSP Routing

• Since Scarface was previous-gen we had no
  software DSP running.
• On Next Gen we have added
• High and Low pass filters to each bus.
• DSP reverb sends to each bus
• Master Bus effects, such as compression.

28
A Word of Caution on Master Bus Compression

• Master Bus compression seems like a good idea at
  first.
• It allows quiet sounds to be made louder and also
  prevents extremely loud sounds from peaking.
• Caution It should be added and tuned at the very
  end of a mix, and should not be introduced prior
  to a mix.
• TYPE OF GAME makes all the difference (solitary
  single-player cinematic experience vs. group
  participatory)
• Master compression should be considered as an
  option in the SOUND MENU for the player to
  determine their listening environment
• Quiet movie-like immersion No Master
  Compression
• Noisy environ, Master Compression On

29
Dynamic Range Summary

• Technical limitations of consumer line-level
  consoles, run-time DSP and snapshot mixer systems
  are now overcome.
• Project management issues Sound Alpha and Sound
  Beta Production process still needs education.
  E.g. notion of signing off sounds
• Aesthetic issues Working closely with designers
  on features and dynamic flow early in production
  still needs to be championed
• Aesthetic area is the most important

30
Randy Thom Interview

• Randy Thom is a well-known evangelist for movie
  sound aesthetics and production.
• Scarface was Randy Thoms first video game
• Randy Began movie sound career on Apocalypse Now
  (1979). 2 Academy Awards, The Right Stuff (1983),
  The Incredibles (2004)
• A good way to get him into games as he didnt
  have to get into the technical aspects. I was
  there as implementer and director.
• Interview has interesting things to say about the
  Future of Video Game Audio

31
Randy Thom Movie
32
Randy Thom Interview

• Most of the technical limitations Randy cites
  were limited to our PS2 production
• Sample Rates
• Iteration times of Sound Effects
• Aesthetic problems are the same for Randy as his
  work for film sound.

33
Questions