Use of Sound in Games

1
Use of Sound in Games

• CIS 487/587
• Bruce R. Maxim
• UM-Dearborn

2
Speech Technology

• Discrete word recognition
• Continuous speech recognition
• Speech store and forward
• Speech generation

3
Discrete Word Recognition

• 90 to 98 reliability for small vocabulary
• Usually requires speaker dependent training
• Most people would rather type than dictate

4
When should you use it?

• Speakers hands are busy
• Mobility required
• Speakers eyes are occupied
• Harsh or cramped conditions prevent use of key
  board

5
Speech Store and Forward

• Voice mail type technology
• Video games
• Low cost
• Resource intensive

6
When to use computer generated speech?

• Message is simple
• Message is short
• Message will not be referred to later
• Message deals with events in time
• Message requires immediate response
• Visual communications channels are overloaded
• Environment lighting is bad
• User must move around
• User subjected to high G forces or lack of oxygen

7
Sound

• Sound travels more slowly than light (this makes
  coordination tricky)
• Sound wave travel at constant speed and can be
  specified with two parameters
• Amplitude (wave height, volume of air moved)
• Frequency (number of complete cycles per second)

8
The next 4 slides are from Rabins book
9
Digital Representationof a Sound Wave

• Most common technique known as sampling
• Sampling involves measuring the amplitude of the
  analog wave file at discrete intervals
• The frequency of sampling is known as sampling
  rate
• Each sample is typically stored in a value
  ranging from 4 to 24 bits in size
• The size of the sample value in bits is known as
  the bit depth
• Music CDs have a sample rate and bit depth of
  44.1 kHz (samples/sec) and 16 bits (sample size)

10
Bit Depth and Signal Noise

• Bit depth of sample data affects signal noise
• Signal to noise ratio number of available bits
  / 1
• For example, 8-bit samples have a 2561 SNR (48
  dB), and 16-bit samples have a 65,5361 SNR (96
  dB)
• Decibel ratio is calculated using 10 x log10
  (ratio) or 8.685890 x log e (ratio)

11
Sampling Frequency and Frequency Reproduction

• Sampling frequency affects range and quality of
  high-frequency reproduction
• Nyquist Limit
• Frequencies up to one-half the sampling rate can
  be reproduced
• Audio quality degrades as frequency approaches
  this limit

12
Modern Audio Hardware

• Samples are piped into sound channels
• Often a hardware pipeline from this point
• Various operations, such as volume, pan, and
  pitch may be applied
• 3D sounds may apply HRTF algorithms and/or mix
  the sound into final output buffers.

13
Wave Shapes

• Sine wave (pure sound)
• Square wave
• Saw tooth wave
• Half-rectified sine wave
• Most sounds are mixtures of several waves, their
  spectrums (frequency distributions) look quite
  ragged
• To make realistic sounds we need to replicate
  this spectrum

14
Computer Sound

• Two types of computer generated sounds
• Digital (recordings of sound)
• Used for sound effects and people talking
• Synthesized (programmed reproductions of sounds)
• Might only be used for music

15
Digital Sound

• Analog-to-digital
• Created by converting the analog sound vibrating
  a microphone to bit string that can be written to
  a disk
• Sample rate (frequency) should be two times the
  frequency of original sound (e.g 400 Hz for male
  voice)
• Amplitude resolution (8 bits for games and 16
  bits for professional sounds and music)

16
Synthesized Sound

• Not as good as digital
• People are used to hearing 16 to 32 different
  tones in a sounds spectrum (not just one pure
  note)
• FM synthesizers use feedback to synthesize
  additional background noise in a sounds
  spectrum

17
MIDI

• Musical Instrument Digital Interface
• Example (using English)
• Turn on channel 1 using an A
• Turn on channel 2 using a C
• Turn off channel 1
• Turn off all channels
• Good for music, bad for explosions

18
Wave Table Synthesis

• Mix between synthesis and digital recording
• Real sampled sounds are stored in a wave table to
  be played back by the DSP (digital sound
  processor)

19
Wave-Guide Synthesis

• Uses DSP chips and special hardware
• Sound synthesizer can generate a mathematical
  model of a virtual instrument and play it
• Most game companies buy sound libraries for sound
  effects

20
Getting Sample Sounds

• Sample from real world using a microphone
• Buy a sample sound library (e.g. LaMothe CD)
• Use synthesizer (e.g. Sound Forge) to create them
  either 22 KHz or 11 KHz using either 8-bit or
  16-bit

21
Recording You Own Sounds

• Use 16 bit samples, 22 KHz, mono (two microphones
  need wide separation to notice stereo effect)
• Cleanup sounds with Sound Forge
• Apply effects (frequency shift, echoes,
  distortions)
• Write your processed sounds using the same
  settings as recording

22
The next 10 slides are from Rabins book
23
Sound Playback Techniques

• Two basic playback methods
• 1. Play sample entirely from memory buffer
• 2. Stream data in real-time from storage medium
• Streaming is more memory efficient for very large
  audio files, such as music tracks, dialogue, etc
• Streaming systems use either a circular buffer
  with read-write pointers, or a double-buffering
  algorithm

24
Sample Playback and Manipulation

• Three basic operations you should know
• Panning is the attenuation of left and right
  channels of a mixed sound
• Results in spatial positioning within the aural
  stereo field
• Pitch allows the adjustment of a samples
  playback frequency in real-time
• Volume control typically attenuates the volume of
  a sound
• Amplification is generally never supported

25
Compressed Audio Format

• Compressed audio formats allow sound and music to
  be stored more compactly
• Bit reduction codecs generally are lightweight
• ADPCM compression is implemented in hardware on
  all the major current video game console systems
• Psycho-acoustic codecs often have better
  compression
• Require substantially more computational
  horsepower to decode

26
MP3, Ogg Vorbis,Licensing Patent Issues

• The MP3 format is patented
• Any commercial game is subject to licensing terms
  as determined by Fraunhofer Thompson
  Multimedia, the holders of the patents
• Ogg Vorbis is similar to MP3 in many ways
• Open source and patent-free (royalty-free)
• Be aware of patent and license restrictions when
  using 3rd party software

27
Programming Music Systems

• Two common music systems
• MIDI-based systems
• (Musical Instrument Digital Interface)
• Digital audio streaming systems
• (CD audio, MP3 playback, etc)

28
Advantages and Disadvantages of MIDI

• Actual music data size is negligible
• Easy to control, alter, and even generate in
  real-time
• High quality music is more difficult to compose
  and program
• Only effective if you can guarantee playback of a
  common instrument set

29
Other MIDI-based technologies to be aware of

• DLS (DownLoadable Sound) Format
• A standardized format for instrument definition
  files
• iXMF (Interactive eXtensible Music Format)
• New proposed standard for a container format for
  interactive music

30
Advantages / Disadvantages of Digital Audio
Streams

• Superb musical reproduction is guaranteed
• Allows composers to work with any compositional
  techniques
• Some potential interactivity is sacrificed for
  expediency and musical quality
• Generally high storage requirements

31
A Conceptual Interactive Music Playback System

• Divide music into small two to eight-bar chunks
  that well call segments.
• A network of transitions from segment to segment
  (including loops and branches) is called a theme.
• Playing music is now as simple as choosing a
  theme to play. The transition map tracks the
  details.

32
API Choices

• DirectSound (part of DirectX API)
• Only available on Windows platforms
• OpenAL
• Newer API
• Available on multiple platforms
• Proprietary APIs
• Typically available on consoles
• 3rd Party Licensable APIs
• Can offer broad cross-platform solutions

33
Direct Sound

• IDirectSound object need one for each installed
  sound card (emulation possible if sound card is
  missing)
• IDirectBuffer use primary and secondary buffers
  like DirectDraw
• IDirectSoundCapture used to record sounds and
  speech recognition
• IDirectSoundNotify used to send messages in
  complex systems

34
LaMothe Examples
35
Game_Init( )

• // create directsound object and test for error
• if (DirectSoundCreate(NULL,lpds,NULL)!DD_OK)
• return(0)
•  
• // set cooperation level to normal priority
• if (lpds-gtSetCooperativeLevel(main_window_handle,D
  SSCL_NORMAL)!DS_OK)
• return(0)

36
Game_Shutdown( )

• // release the directsoundobject
• if (lpds!NULL)
• lpds-gtRelease()

37
Generating a Sound

• // this example does everything it sets up
  directsound
• // creates a secondary buffer, loads it with a
  synthesizer
• // sine wave and plays it
•  
• void audio_ptr_1 NULL, // used to lock
  memory
• audio_ptr_2 NULL
•  
• DWORD dsbstatus // status of sound
  buffer
•  
• DWORD audio_length_1 0, // length of locked
  memory
• audio_length_2 0,
• snd_buffer_length 64000 // working buffer
•  
• // allocate memory for buffer
• UCHAR snd_buffer_ptr (UCHAR )malloc(snd_buffer
  _length)

38
Generating a Sound

• // we need some data for the buffer, you could
  load a .VOC or .WAV
• // but as an example, lets synthesize the data
•  
• // fill buffer with a synthesized 100hz sine wave
• for (int index0 index lt (int)snd_buffer_length
  index)
• snd_buffer_ptrindex 127sin(6.28((float)(ind
  ex110))/(float)110)
•  
• // note the math, 127 is the scale or amplitude
• // 6.28 is to convert to radians
• // (index 110) read below
• // we are playing at 11025 hz or 11025 cycles/sec
  therefore, in 1 sec
• // we want 100 cycles of our synthesized sound,
  thus 11025/100 is approx.
• // 110, thus we want the waveform to repeat each
  110 clicks of index, so
• // normalize to 110

39
Generating a Sound

• // set cooperation level
• if (lpds-gtSetCooperativeLevel(main_window_handle,D
  SSCL_NORMAL)!DS_OK)
• return(0)
•  
• // set up the format data structure
• memset(pcmwf, 0, sizeof(WAVEFORMATEX))
•  
• pcmwf.wFormatTag WAVE_FORMAT_PCM
• pcmwf.nChannels 1
• pcmwf.nSamplesPerSec 11025
• pcmwf.nBlockAlign 1
• pcmwf.nAvgBytesPerSec pcmwf.nSamplesPerSec
  pcmwf.nBlockAlign
• pcmwf.wBitsPerSample 8
• pcmwf.cbSize 0

40
Generating a Sound

• // create the secondary buffer (no need for a
  primary)
• memset(dsbd,0,sizeof(DSBUFFERDESC))
• dsbd.dwSize sizeof(DSBUFFERDESC)
• dsbd.dwFlags DSBCAPS_CTRLDEFAULT
  DSBCAPS_STATIC DSBCAPS_LOCSOFTWARE
• dsbd.dwBufferBytes snd_buffer_length1
• dsbd.lpwfxFormat pcmwf
•  
• if (lpds-gtCreateSoundBuffer(dsbd,lpdsbsecondary,
  NULL)!DS_OK)
• return(0)
• // copy data into sound buffer
• if (lpdsbsecondary-gtLock(0, snd_buffer_length,
• audio_ptr_1, audio_length_1, audio_ptr_2,
• audio_length_2, DSBLOCK_FROMWRITECURSOR)!DS_OK
  )
• return(0)

41
Generating a Sound

• // copy first section of circular buffer
• CopyMemory(audio_ptr_1, snd_buffer_ptr,
  audio_length_1)
•  
• // copy last section of circular buffer
• CopyMemory(audio_ptr_2,(snd_buffer_ptraudio_lengt
  h_1),audio_length_2)
•  
• // unlock the buffer
• if (lpdsbsecondary-gtUnlock(audio_ptr_1,
  audio_length_1,
• audio_ptr_2, audio_length_2)!DS_OK)
• return(0)
•  
• // play the sound in looping mode
• if (lpdsbsecondary-gtPlay(0,0,DSBPLAY_LOOPING
  )!DS_OK)
• return(0)
• // release the memory since DirectSound has made
  a copy of it
• free(snd_buffer_ptr)

42
Reading from Files

• DirectSound has no support for reading .WAV files
  from disk (involves 2 steps)
• Reading .WAV files, involves reading a header to
  get format info (e.g. channels, bits/channel,
  playback rate, length of sample sound
• Loading the sound

43
Dsound_Load_WAV

• Open .Wav file and extract header info
• Create and fill DirectSound buffer
• Stores info in open slot in sound.fx and
  returns SounfID as an index
• Sound card can be played at any time using
• sound_fxsound_id.dsbuffer-gtPlay(0,0,DSBPLAY_LOOP
  ING)

44
Globals

• // this holds a single sound
• typedef struct pcm_sound_typ
• LPDIRECTSOUNDBUFFER dsbuffer // the ds buffer
  containing the sound
• int state // state of the
  sound
• int rate // playback rate
• int size // size of sound
• int id // id number of
  the sound
• pcm_sound, pcm_sound_ptr

45
Globals

• LPDIRECTSOUND lpds // directsound
  interface pointer
• DSBUFFERDESC dsbd // directsound
  description
• DSCAPS dscaps // directsound caps
• HRESULT dsresult // general directsound
  result
• DSBCAPS dsbcaps // directsound
  buffer caps
•  
• LPDIRECTSOUNDBUFFER lpdsbprimary, // you won't
  need this normally
• lpdsbsecondary // the sound
  buffers
•  
• WAVEFORMATEX pcmwf // generic
  waveformat structure
•  
• pcm_sound sound_fxMAX_SOUNDS // array of
  secondary sound buffers
•  
• HWND freq_hwnd, // window handles for
  controls
• volume_hwnd,
• pan_hwnd
• int sound_id -1 // id of sound we load for
  demo

46
Game_Init( )

• // create a directsound object
• if (DirectSoundCreate(NULL, lpds, NULL)!DS_OK )
• return(0)
•  
• // set cooperation level
• if (lpds-gtSetCooperativeLevel(main_window_handle,D
  SSCL_NORMAL)!DS_OK)
• return(0)
•  
• // clear array out
• memset(sound_fx,0,sizeof(pcm_sound)MAX_SOUNDS)

47
Game_Init( )

• // initialize the sound fx array
• for (int index0 indexltMAX_SOUNDS index)
• // test if this sound has been loaded
• if (sound_fxindex.dsbuffer)
• // stop the sound
• sound_fxindex.dsbuffer-gtStop()
• // release the buffer
• sound_fxindex.dsbuffer-gtRelease()
• // end if
• // clear the record out
• memset(sound_fxindex,0,sizeof(pcm_sound))
• // now set up the fields
• sound_fxindex.state SOUND_NULL
• sound_fxindex.id index
• // end for index

48
Game_Init( )

• // load a wav file in
• if ((sound_id DSound_Load_WAV("FLIGHT.WAV"))!-1
  )
• // start the voc playing in looping mode
• sound_fxsound_id.dsbuffer-gtPlay(0,0,DSBPLAY_LO
  OPING)
• // end if

49
Game_Shutdown( )

• // release the sound buffer
• if (sound_fxsound_id.dsbuffer)
• sound_fxsound_id.dsbuffer-gtRelease()
•  
• // release the directsoundobject
• if (lpds!NULL)
• lpds-gtRelease()

50
DSound_Load_Wav( )

• HMMIO hwav // handle to wave file
• MMCKINFO parent, // parent chunk
• child // child chunk
• WAVEFORMATEX wfmtx // wave format structure
•  
• int sound_id -1, // id of sound to be
  loaded
• index // looping variable
•  
• UCHAR snd_buffer, // temporary sound
  buffer to hold voc data
• audio_ptr_1NULL, // data ptr to first
  write buffer
• audio_ptr_2NULL // data ptr to second
  write buffer
•  
• DWORD audio_length_10, // length of first write
  buffer
• audio_length_20 // length of second write
  buffer

51
DSound_Load_Wav( )

• // step one are there any open id's ?
• for (index0 index lt MAX_SOUNDS index)
• // make sure this sound is unused
• if (sound_fxindex.stateSOUND_NULL)
• sound_id index
• break
• // end if
•  
• // end for index
•  
• // did we get a free id?
• if (sound_id-1)
• return(-1)

52
DSound_Load_Wav( )

• // set up chunk info structure
• parent.ckid (FOURCC)0
• parent.cksize 0
• parent.fccType (FOURCC)0
• parent.dwDataOffset 0
• parent.dwFlags 0
•  
• // copy data
• child parent
•  
• // open the WAV file
• if ((hwav mmioOpen(filename, NULL, MMIO_READ
  MMIO_ALLOCBUF))NULL)
• return(-1)
•  
• // descend into the RIFF
• parent.fccType mmioFOURCC('W', 'A', 'V', 'E')

53
DSound_Load_Wav( )

• if (mmioDescend(hwav, parent, NULL,
  MMIO_FINDRIFF))
• // close the file
• mmioClose(hwav, 0)
•  
• // return error, no wave section
• return(-1)
• // end if
•  
• // descend to the WAVEfmt
• child.ckid mmioFOURCC('f', 'm', 't', ' ')

54
DSound_Load_Wav( )

• if (mmioDescend(hwav, child, parent, 0))
• // close the file
• mmioClose(hwav, 0)
• // return error, no format section
• return(-1)
• // end if
•  
• // now read the wave format information from file
• if (mmioRead(hwav, (char )wfmtx, sizeof(wfmtx))
  ! sizeof(wfmtx))
• // close file
• mmioClose(hwav, 0)
• // return error, no wave format data
• return(-1)
• // end if

55
DSound_Load_Wav( )

• // make sure that the data format is PCM
• if (wfmtx.wFormatTag ! WAVE_FORMAT_PCM)
• // close the file
• mmioClose(hwav, 0)
• // return error, not the right data format
• return(-1)
• // end if
•  
• // now ascend up one level, so we can access data
  chunk
• if (mmioAscend(hwav, child, 0))
• // close file
• mmioClose(hwav, 0)
• // return error, couldn't ascend
• return(-1)
• // end if

56
DSound_Load_Wav( )

• // descend to the data chunk
• child.ckid mmioFOURCC('d', 'a', 't', 'a')
•  
• if (mmioDescend(hwav, child, parent,
  MMIO_FINDCHUNK))
• // close file
• mmioClose(hwav, 0)
•  
• // return error, no data
• return(-1)
• // end if
•  
• // finally!!!! now all we have to do is read the
  data in and
• // set up the directsound buffer

57
DSound_Load_Wav( )

• // allocate the memory to load sound data
• snd_buffer (UCHAR )malloc(child.cksize)
•  
• // read the wave data
• mmioRead(hwav, (char )snd_buffer, child.cksize)
•  
• // close the file
• mmioClose(hwav, 0)
•  
• // set rate and size in data structure
• sound_fxsound_id.rate wfmtx.nSamplesPerSec
• sound_fxsound_id.size child.cksize
• sound_fxsound_id.state SOUND_LOADED

58
DSound_Load_Wav( )

• // set up the format data structure
• memset(pcmwf, 0, sizeof(WAVEFORMATEX))
•  
• pcmwf.wFormatTag WAVE_FORMAT_PCM // pulse
  code modulation
• pcmwf.nChannels 1 // mono
• pcmwf.nSamplesPerSec 11025 // always
  this rate
• pcmwf.nBlockAlign 1
• pcmwf.nAvgBytesPerSec pcmwf.nSamplesPerSec
  pcmwf.nBlockAlign
• pcmwf.wBitsPerSample 8
• pcmwf.cbSize 0
•  
• // prepare to create sounds buffer
• dsbd.dwSize sizeof(DSBUFFERDESC)
• dsbd.dwFlags control_flags DSBCAPS_STATIC
  DSBCAPS_LOCSOFTWARE
• dsbd.dwBufferBytes child.cksize
• dsbd.lpwfxFormat pcmwf

59
DSound_Load_Wav( )

• // create the sound buffer
• if (FAILED(lpd-gtCreateSoundBuffer(dsbd,sound_fx
  sound_id.dsbuffer,NULL)))
• // release memory
• free(snd_buffer) 
• // return error
• return(-1)
• // end if 
• // copy data into sound buffer
• if (FAILED(sound_fxsound_id.dsbuffer-gtLock(0,
  child.cksize,
• (void )audio_ptr_1, audio_length_1, (void
  )audio_ptr_2, audio_length_2,
  DSBLOCK_FROMWRITECURSOR)))
• return(0)

60
DSound_Load_Wav( )

• // copy first section of circular buffer
• memcpy(audio_ptr_1, snd_buffer, audio_length_1)
• // copy last section of circular buffer
• memcpy(audio_ptr_2, (snd_bufferaudio_length_1),au
  dio_length_2)
• // unlock the buffer
• if (FAILED(sound_fxsound_id.dsbuffer-gtUnlock(aud
  io_ptr_1,
• audio_length_1, audio_ptr_2,
  audio_length_2)))
• return(0)
•  
• // release the temp buffer
• free(snd_buffer)
•  
• // return id
• return(sound_id)

61
Direct Music

• Supports downloadable sounds (e.g. MIDI files)
• Supports on the fly music composition
• Supports unlimited MIDI channels (not just the
  standard 16)
• Uses hardware acceleration if available

62
Direct Music Interfaces

• IDirectMusic main interface
• IDirectMusicPerformance controls and
  manipulates playback
• IDirectMusicLoader loads MIDI files
• IDirectMusicSegment chunk of music data
• IDirectMusicSegmentState checks current state
  of segment
• IDirectMusicPort destination of streamed output
  data

63
Globals

• include ltdsound.hgt // include dsound, dmusic
• include ltdmksctrl.hgt
• include ltdmusici.hgt
• include ltdmusicc.hgt
• include ltdmusicf.hgt
• // direct music globals
• IDirectMusicPerformance dm_perf NULL
• IDirectMusicLoader dm_loader NULL
• IDirectMusicSegment dm_segment NULL
• IDirectMusicSegmentState dm_segstate NULL

64
CreatePerformance( )

• IDirectMusicPerformance CreatePerformance(void)
• // this function creates the performance
•  
• IDirectMusicPerformance pPerf
•  
• if (FAILED(CoCreateInstance(CLSID_DirectMusicPerfo
  rmance,
• NULL, CLSCTX_INPROC,
  IID_IDirectMusicPerformance,
• (void)pPerf)))
• // return null
• pPerf NULL
• // end if
•  
• return pPerf
• // end CreatePerformance

65
CreateLoader( )

• IDirectMusicLoader CreateLoader(void)
• // this function creates the loader
• IDirectMusicLoader pLoader
• if (FAILED(CoCreateInstance(CLSID_DirectMusicLoa
  der,
• NULL,CLSCTX_INPROC,IID_IDirectMusicLoa
  der,
• (void)pLoader)))
• pLoader NULL
• // end if
• return pLoader
• // end CreateLoader

66
FreeDirectMusic( )

• // If there is any music playing, stop it.
• dm_perf-gtStop( NULL, NULL, 0, 0 )
•  
• // Unload instruments this will cause silence.
• // CloseDown unloads all instruments, so this
  call is also not
• // strictly necessary.
• dm_segment-gtSetParam(GUID_Unload, -1, 0, 0,
  (void)dm_perf)
• // Release the segment.
• dm_segment-gtRelease()
• // CloseDown and Release the performance object.
  
• dm_perf-gtCloseDown()
• dm_perf-gtRelease()
• // Release the loader object.
• dm_loader-gtRelease()
• // Release COM.
• CoUninitialize()
• return S_OK

67
LoadMidiSegment( )

• IDirectMusicSegment LoadMIDISegment(IDirectMusicL
  oader pLoader,
• WCHAR
  wszMidiFileName )
• // this function loads a midi segment off disk
• DMUS_OBJECTDESC ObjDesc
• HRESULT hr
• IDirectMusicSegment pSegment NULL
• // get current working directory
• char szDir_MAX_PATH
• WCHAR wszDir_MAX_PATH
• if(_getcwd( szDir, _MAX_PATH ) NULL)
• return NULL
• // end if
• // convert to wide characters
• MULTI_TO_WIDE(wszDir, szDir)

68
LoadMidiSegment( )

• // set the search directory
• hr pLoader-gtSetSearchDirectory(GUID_DirectMusicA
  llTypes,wszDir, FALSE)
•  
• if (FAILED(hr))
• return NULL
• // end if
• // setup object description
• ZeroMemory(ObjDesc, sizeof(DMUS_OBJECTDESC))
• ObjDesc.dwSize sizeof(DMUS_OBJECTDESC)
• ObjDesc.guidClass CLSID_DirectMusicSegment
• wcscpy(ObjDesc.wszFileName, wszMidiFileName )
•  
• ObjDesc.dwValidData DMUS_OBJ_CLASS
  DMUS_OBJ_FILENAME

69
LoadMidiSegment( )

• // load the object and query it for the
  IDirectMusicSegment interface
• // This is done in a single call to
  IDirectMusicLoaderGetObject
• // note that loading the object also initializes
  the tracks and does
• // everything else necessary to get the MIDI data
  ready for playback.
• hr pLoader-gtGetObject(ObjDesc,IID_IDirectMusicS
  egment,(void)pSegment)
• if (FAILED(hr)) return(0)
• // ensure that the segment plays as a standard
  MIDI file
• // you now need to set a parameter on the band
  track
• // Use the IDirectMusicSegmentSetParam method
  and let
• // DirectMusic find the trackby passing -1 (or
  0xFFFFFFFF)
• // in the dwGroupBits method parameter.
• hr pSegment-gtSetParam(GUID_StandardMIDIFile,-1,
  0, 0, (void)dm_perf)
•  
• if (FAILED(hr)) return(0)

70
LoadMidiSegment( )

• // The next step is to download the instruments.
• // This is necessary even for playing a simple
  MIDI file
• // because the default software synthesizer needs
  the DLS data
• // for the General MIDI instrument set
• // If you skip this step, the MIDI file will play
  silently.
• // Again, you call SetParam on the segment, this
  time specifying the GUID_Download parameter
•  
• hr pSegment-gtSetParam(GUID_Download, -1, 0, 0,
  (void)dm_perf)
•  
• if (FAILED(hr))
• return(0)
•  
• // return the pointer
• return pSegment
• // end LoadSegment

71
Game_Init( )

• // set up directmusic
• if (FAILED(CoInitialize(NULL)))
• // Terminate the application.
• return(0)
• // end if 
• // create the performance
• dm_perf CreatePerformance()
• if (dm_perf NULL)
• return(0)// Failure -- performance not created
• // end if 
• // initialize the performance
• if (FAILED(dm_perf-gtInit(NULL, NULL,
  main_window_handle)))
• return(0)// Failure -- performance not
  initialized
• // end if

72
Game_Init( )

• // add the port to the performance
• if (FAILED(dm_perf-gtAddPort(NULL)))
• return(0)// Failure -- port not initialized
• // end if
•  
• // create the loader to load object(s) such as
  midi file
• dm_loader CreateLoader()
•  
• if (dm_loader NULL)
• return(0)// Failure -- loader not created
• // end if

73
Game_Init( )

• // release the old segment
• if (dm_segment)
• dm_segment-gtRelease()
• dm_segment NULL
• // end if 
• // load the segment
• if (dm_loader)
• dm_segment LoadMIDISegment(dm_loader,L"BATTLE.
  MID")
• // end if
• // start the song
• if (dm_segment)
• dm_perf-gtPlaySegment(dm_segment, 0, 0,
  dm_segstate)
• // end if

74
Game_Shutdown( )

• int Game_Shutdown(void parms NULL, int
  num_parms 0)
• // this is called after the game is exited and
  the main event
• // loop while is exited, do all you cleanup and
  shutdown here
•  
• FreeDirectMusic()
•  
• // return success or failure or your own return
  code here
• return(1)
•  
• // end Game_Shutdown

75
Using Both

• You can use DirectSound and DirectMusic in the
  same application
• You must initialize DirectSound before DirectMusic

76
The next 4 slides are from Rabins book
77
Audio Scripting and Engine Integration

• Very little audio programming should be done by
  general game programmers
• Game Engine should offer robust support for audio
  triggers and scripts
• Engine should deal with audio scripts, not sound
  files
• Why is this so important?

78
Audio Scripting

• Many situations require much more information
  than can be embedded in a linear audio file
• Sound Variation
• Sound Repetition
• Complex Sound Looping
• Background Ambience

79
Lip-sync Technology

• Lip-sync technology is a blending of audio and
  visual techniques to create realistic-looking
  speech by in-game actors.
• Simple techniques such as waveform amplitude
  measurement has worked previously, but
• In future titles, it will be considered
  inadequate.
• Much work can still be done in this field.

80
Advanced Voice Playback

• Real-time spoken feedback is especially important
  in sports titles (simulated announcers)
• Game are reaching the limits of what current
  techniques (canned, prerecorded phrases combined
  in series) can provide.
• Again, this is an opportunity for future
  groundbreaking audio work.