Acoustics of Music

1
Acoustics of Music

• Dr Ian Drumm

2
Perception and Realism

• Aims
• To review perception of musical sound by
  correlating objective parameters and subjective
  percepts
• Learning Outcomes
• Synthesis influencing perception of loudness,
  pitch and timbre.
• How varying synthesis parameters overtime may be
  perceived
• The notion of Timbre Space and the application of
  Multidimensional Scaling
• Develop ideas on perceptually guided data
  reduction for synthesis

3
Available objective parameters and corresponding
percepts

• Consider additive synthesis
• Fundamental frequency is associated with pitch
• Amplitude is associated with loudness
• Spectrum (harmonic series) is associated with
  timbre
• Phase is associated with?

4
Perception of phase

• Using Fourier build 100Hz square wave using the
  first ten non-zero harmonics.
• In one line the harmonics all have the correct
  phase and in the other the phases are random.
• Waveforms look very different
• It is hard to hear the difference.
• So can we neglect phase in additive synthesis?

5
Perception of parameter variations

• How do we perceive changes in the objective
  quantities of frequency, amplitude and spectrum?
• Frequency variation
• Think of frequency being modulated by another
  wave (perhaps a sine)
• Small change (6Hz, small amplitude) gives
  vibrato
• Large change completely alters spectrum (see FM
  later on).

6
Perception

• Amplitude variation
• Amplitude envelope defines individual notes
• Attack is important for timbre perception
• Low-level modulation of amplitude envelope heard
  as tremolo
• Spectrum
• Gives realism (can be complicated and subtle)
• Or effects (e.g. wah-wah pedal is gross change of
  centre freq of a band-pass filter)
• Or comparatively new sounds (see morphing, later
  on)
• Phase variation
• Seems to add life and realism to notes with
  many partials like a piano

7
What is needed for realism

• TVPS implies a high computational load so we need
  to make omissions and optimisations. Hence we
  should ask
• What features of a real instrument are the most
  important to synthesis?
• How do we recognise instruments to be guitars,
  trumpets, wood winds?
• Can we say an aspect of an instruments timbre is
  its signature?
• Can we find the minimum components needed to
  convey and instruments signature?

8
Are signatures in the spectral balance?

• If we analyse a clarinet we see the harmonics
  are odd corresponding to a square wave
• Conversely if we build a square wave it is
  recognisably clarinet-like
• However if we analyse a trumpet and then
  re-synthesise the sound is more oboe-like than
  brass-like
• What components do we need for brass-like?

9

• What does this sound like?
• What does this sound like

10
Rissets Trumpet

• Jean-Claude Rissets at Bell Telephone
  Laboratories
• Analyses and re-synthesised a variety of
  instrument sounds
• Series of subjective tests removing key
  components

11
Rissets Trumpet

• The relative behaviour of harmonics in the rapid
  attack component (i.e. first 20ms) of the timbre
  was crucial for creating the recognisable
  signature of a trumpet sound
• As the overall amplitude increases so does the
  relative contribution of the 3rd and 4th
  harmonics
• i.e. centroid of the spectrum shifts up in
  frequency with the increase of overall amplitude
  of the tone.

12
Subjective Distance

• A still greater understanding of how we perceive
  timbre can be achieved by statistical methods
  such as multidimensional scaling as applied by
  Grey (JASA 1972).
• If a given pair of timbres (e.g. trumpet and
  guitar) sound more different than another pair
  (e.g. mandolin and guitar) then the first pair
  has a greater subjective distance.

13
Multidimensional Scaling (MDS)

• Consider the relative distances between a
  selection of cities
• Construct a table
• Multidimensional scaling lets you extract the
  dimensions X and Y to hence represent these
  cities on a 2D map.

14
The steps of MDS

• The technique employs a computer to try different
  positions for objects in a space until it
  eventually finds to position that fit best with
  the original data.
• The steps are
• Define a space (in this case X,Y and Z is 3D).
• Arbitrarily position our objects (cities or
  timbres) within this space.
• Form a matrix of distances between points called
  a Dhat matrix
• Compare this Dhat with the original input matrix
  of distances (D matrix) using a stress function.
• Adjust coordinates of each object in the
  direction that minimises stress
• Repeat until stress values wont get lower

Reproduced distances
Original data
15
Greys timbre space

• Grey asked trained musicians to rate the
  similarity of pairs of synthesised emulations of
  real instrumental sounds
• The timbres were synthesised emulations to make
  it easy for the researchers to present all
  timbres with the same loudness, pitch and
  duration so that these factors dont become
  part of the analysis

16

• The sixteen timbres included two oboes, English
  horn, bassoon, clarinet, bass clarinet, flute,
  two alto saxophones, soprano saxophone, trumpet,
  French horn, muted trombone, three celli (normal
  playing, muted, bowed at bridge).
• Each listener listened to 24016(16-1) possible
  pairs of timbres given in both directions and
  presented in random order. Hence each subject
  gave a similarity rating on a scale of 1 to 30,
  where 1-10 represented very dissimilar, 11-29
  average level of similarity and 21-30 very
  similar.
• Given subjective distance is clearly the inverse
  of the similarity rating a suitable 16x16 input
  matrix of subject distances for MDS analysis was
  created.

17
Greys Timbre Space
BN - Bassoon C1 - E flat Clarinet C2 - B flat
Bass Clarinet EH - English Horn FH - French
Horn FL - Flute O1 - Oboe O2 - Oboe S1 -
Cello, muted S2 - Cello S3 - Cello, muted TM -
Muted Trombone TP - B flat Trumpet X1
Saxophone X2 Saxophone X3 - Soprano Saxophone
18

• As we move up I axis, can see spectral bandwidth
  narrows (hence axis relates to spectral energy
  distribution)

19
Axis I

• French Horn has a narrow bandwidth of
  predominantly low frequency energy compared to
  the other extreme of the Trombone with a much
  broader distribution of energy.

20
Axis II

• This relates to the synchronicity in attacks and
  decays of upper harmonics. For example a
  Saxophone timbre at one extreme of the axis has
  its upper harmonics appear, reach maximum and
  decay at the same time. Compare that with the
  other extreme of Strings whose upper harmonics
  rise or taper off a different times.

21
Axis III

• Relates to an-harmonic content in the attack
• Clarinets, Flutes and Strings tend to have a lot
  of high frequency an-harmonicty in the attack.
  Where as on the other extreme of the axis the
  French Horn, Trumpet and Bassoon either have only
  low frequency or no an-harmonic transients in the
  attack.

22

• Hence three significant dimensions resulted
• I Spectral energy distribution
• II Synchronicity of partials
• III An-harmonicity of attack
• Interesting to note the importance of attack in
  characterising timbre

23

• Similar studies concur (almost) Krumhansl,
  Lakatos, etc
• Timbre characterised by
• Frequency Distribution
• Spectral Flux
• Attack Quality
• Higher dimensions?
• Some dimensions unique to certain instruments for
  example a clarinet has hollow tone due to even
  harmonics.
• Timbres can be perceptually grouped according to
  shared characteristics e.g. type of excitation
• Neural Correlates of Timbre Perception
• Apparently right brain processes timbre
  perception left ear has a timbre perception
  advantage