CG for Vision Research

1
CG for Vision Research

• Lecture 9
• Winter Term 05-06
• Quoc Vuong, Franck Caniard, MPI

2
OpenGL Projections Transformations Graphics
Primitives Textures Lighting Material Properties
Vision Research Viewpoint Effect Structure from X
(Motion, Shading, Stereo) Active
Exploration Navigation
GLUT Window Management Input Devices Event
Loop Timing
Fun/Advanced Modeling Animations Mapping (bumps,
terrains) Physics Simulation Sound (OpenAL)
3
Audition
4
Properties of Sound
Low Frequency Low Pitch
Amplitude Loudness
High Frequency High Pitch
5
Loudness

• Measured in decibels (log units)
• Leaves rustling 20
• Avg. speaking voice 60
• Traffic 80
• Rock band 120
• Jet engine at takeoff 140
• Depends on Frequency!

http//www.medicine.mcgill.ca/physio/cullenlab/img
209/aud-cuopt.gif
6
Auditory LocalizationITD

• Max Time Difference ? 600 microseconds (6/10,000
  sec)
• Threshold detections ? 10 microseconds
• From Goldstein, Sensation Perception

Interaural Time Difference
7
Auditory LocalizationIID

• Depends on frequency
• E.g., for 6,000 Hz tone, intensity difference ?
  20 dB less in far ear
• From Goldstein, Sensation Perception

Interaural Intensity Difference
8
True 3D Sound

• ITD and IID only localizes horizontally
• Bias for sound to come from in front
• Head, pinna torso help localize sound in 3D
• Head-related Transfer Function (HRTF) describes
  how frequency of sound source is reflected and
  diffracted by body before sound reaches auditory
  sensors

9
Auditory Time-To-Contact TTC

• Two cues to TTC
• Intensity change
• Intensity of sound I Energy / (Time Area)
• Decreases as a function of distance following
  inverse square law
• Doppler shift
• Change in pitch (frequency)

10
Doppler Effect

• Apparent shift in frequency (pitch change) due to
  relative motion of sound sources and listeners
• Increase in frequency for source approaching
  listener
• Decrease in frequency for source moving away from
  listener

• Depends on velocities of sources and listeners
  relative to the medium and propagation speed of
  sound in that medium

http//en.wikipedia.org/wiki/Doppler_shift
11
Visual-AuditoryIntegration

• McGurk Effect (demo http//www.media.uio.no/perso
  ner/arntm/McGurk_english.html)
• Auditory-Induced Visual Illusions, e.g., Shams et
  al. 2002
• (demo http//shamslab.psych.ucla.edu/demos/)

12
OpenAL 1.1
13
What is OpenAL?

• Audio version of OpenGL
• Cross-platform 3D audio Application Programming
  Interface (API)
• Allows application to position audio sources in
  3D space around a listener
• Released in 2000 Hardware acceleration in 2001
• Development
• www.openal.org
• Creative Labs developer.creative.com
• EAX 5.0 Audio Extensions
• Xbox Xbox 360
• OpenAL Utility Toolbox (ALUT) 1.0.6
• Equivalent to glut

14
Fundamental Objects of OpenAL
Device
Listener
Context (3D Scene)
Source
Source
Source
Buffer
Buffer
Buffer
15
Steps

• Initialize a device (e.g., speakers)
• Generate BUFFERs to hold audio files
• Generate SOURCEs and set their properties
• Set properties of LISTENER (only 1 per context)
• Attach BUFFERs to SOURCEs
• Dynamically update SOURCEs and LISTENER properties

16
Buffer
// Creating Buffers ALuint Buffer alGenBuffers(1
, Buffer) // Creating multiple buffers ALuint
Buffers5 alGenBuffers(5, Buffers) // Getting
buffer properties ALint Freq alGetBufferi(Buffer
, AL_FREQUENCY, Freq) // Deleting buffer from
memory alDeleteBuffers(1, Buffer)
17
Listener
// Only 1 listener per context // created //
Setting Listener properties ALfloat
lPos0.0,0.0,0.0 alListenerfv(AL_POSITION,
lPos) // Getting Listener properties ALfloat
lGain alGetListenerf(AL_GAIN, lGain) printf("Li
stener Gain .1f\n", lGain)
18
Source
// Creating Sources in a context ALuint
Sources2 alGenSources(2, Sources) // Setting
Source properties ALfloat Source0Pos0.0,0.0,0
.0 alSourcef(Sources0, AL_PITCH,
1.0f) alSourcefv(Sources0, AL_POSITION,
Source0Pos) // Attaching Buffer to
Source alSourcei(Sources0, AL_BUFFER,
Buffers0) // Playing Source alSourcePlay(Sour
ces0)
19
Distance Model

• The drop-off in sound intensity as a function of
  distance implemented in OpenAL (see Programming
  Guide for formula)
• alDistanceModel(ALEnum DistanceModel)
• No model (AL_NONE)
• Inverse Distance Model (AL_INVERSE_DISTANCE)
• Default Clamped Inverse Distance Model
  (AL_INVERSE_DISTANCE_CLAMPED)
• Changes gain of the sound source (AL_GAIN)

20
Doppler Shift
PS min( PS, Vsound / DopplerFactor )
PL min( PL, Vsound / DopplerFactor )
freqshifted freq ( Vsound DopplerFactor
PL ) / ( Vsound DopplerFactor PS )
alDopplerFactor(ALfloat, DopplerFactor) //
Default 1.0
alSpeedOfSound(ALfloat, Speed) // Default
343.3 m/s (air)
21
Issues and Future Outlook

• Synchronization of vision and audition
• OpenML Open Media Library (2001)
• Syncrhonization of video and audio streaming, 2D
  and 3D graphics, I/O, and networking
• http//www.khronos.org/openml/

22
File Issues

• Files for running programs
• OpenAL32.dll, alut.dll (Dynamic Link Libraries)
• Two locations
• C\WINDOWS\systems32 folder accessible to all
  programs need Administrator rights though.
• Locally in the same folder as the executable
  (.exe) file note that any local copy overrides
  the system copy!
• Files for compiling programs
• Header files al.h, alc.h, alut.h
• MinGW\include\AL
• Compiled libraries (.lib) OpenAL32.lib, alut.lib
• MinGW\lib
• There are also extensions that makes use of sound
  card (e.g., hardware acceleration)
• For OpenAL, this is EAX 5.0 being developed by
  Creative

23
Template Code 6
24
This slide left intentionally blank