Psychoacoustics and Music Perception

1
Psychoacoustics and Music Perception

• 509.211 VO, 2st.
• S06, Mi 1730-1915
• HS 06.03
• Richard Parncutt
• Email ((my last name))_at_uni-graz.at
• Office hours Thursday 10 am

2
This file is

• available in the internet and updated regularly
• only a PART of the course material. Missing
• verbal explanations in lectures
• figures drawn on board and displayed with OHP
  (transparencies)
• contents of folder in reading room of department
  library
• sound examples (including those linked to this
  document but many of these are on the CD in the
  folder)
• written in point form but exam answers must be
  complete sentences! (see Schriftliche
  Prüfungen)
• Questions and suggestions? ((familyname))_at_uni-graz
  .at

3
Lecture 1, 8.3.06

• Adminstrative details
• aims
• dates
• examination
• Introduction Musical relevance of
  psychoacoustics
• Course outline, literature
• Philosophy of perception the 3 worlds of
  Popper Eccles (1977)
• Literature
• Parncutt, R. (in press). Psychoacoustics and
  music perception
• Terhardt, E. (1998). Akustische Kommunikation.
  Berlin Springer. (1. Kapitel)

4
Musical relevance

• Consider some everyday musical examples
• J. S. Bach O Haupt voll Blut und Wunden
  (Baroque choral)
• Frank Sinatra White Christmas (pop)
• Miles Davis So what (modal jazz)
• Igor Stravinsky Sacré du printemps (modern
  orchestral)
• Consider some psychological issues
• What do you hear or experience in this music?
• Chain physics perception structure
  associations
• Direct perception ecological psychology
• Indirect perception cognitive psychology

5
Relevance for music analysis

• Perception of pitch structures
• harmony, voice-leading, phrasing, tonality,
  modulation
• Quality of sound
• consonance/dissonance, timbre
• Cognitive organisation
• foreground, background
• Emotional character
• associations
• Not considered
• Accents dynamic, grouping, metrical, melodic,
  harmonic
• Expressive timing and dynamics

6
Some course aims

• Overview and understand
• musically relevant fundamentals of
  psychoacoustics
• perceptual correlates of music-theoretical
  concepts (cons./diss., root/tonic)
• Understand technical primary literature
• extract relevant information from it
• Show relevance for music theory and analysis
• Contribute to understanding of musical meaning
• perceptual/cognitive processes
• personal/cultural associations
• Raise awareness of applications
• music theoretical, analytical, and practical
• Prepare for the SE "Cognition of Musical
  Structure"

7
Tentative semester plan (1)
8
Tentative semester plan (2)
9

• Preparation for lectures
• Read the literature in advance!
• Making up for lost time
• Students at the first lecture on 8.3.06 preferred
  to extend each lecture by 15 minutes (i.e.
  1730-1915) than to schedule two additional
  lectures.

10
Central literature sources

• Houtsma et al.(1987). Auditory demonstrations on
  compact disc.
• Articles in Semester Plan above
• Both are in folder Psychoacoustics
• Handapparat, reading room, musicology
• To copy articles
• take folder to secretarys office

11
Auxiliary literature sources

• Bregman (1994). Auditory scene analysis
• De la Motte-haber (2005). Musikpsychologie.
• Deutsch (Ed., 1999). Psychology of music (2. ed.)
  
• Hall (1997). Musikalische Akustik
• Handel (1993). Listening
• Harwood Dowling (1995). Music cognition
• Howard James (1996). Acoustics and
  psychoacoustics.
• McAdams Bigand (Eds., 1993). Thinking in sound
• Pierce (1985). Klang
• Roederer (1993). Physikal. und psychoakust.
  Grundlagen der Musik
• Rosen Howell (1991). Signals systems for
  speech hearing
• Terhardt (1998). Akustische Kommunikation
• Zwicker (1982). Psychoakustik

12
The process of sound perceptionWhy do we
experience a complex tone as one thing?
13
Philosophy of reality Karl Poppers three
worlds (1)

• World 1 physical matter, energy
• World 2 experiential sensations, emotions
• World 3 abstract information, knowledge,
  culture
• Example A visit to an art gallery
• physical walls, floor, canvas, paint, light
  waves, retina
• experiential colors, shapes, emotions (feeling,
  mood), sound or silence, smell, taste, touch
• abstract program, thoughts, content of
  conversation, historical knowledge, digital
  representations, theory of art
• Group exercise repeat this analysis for a visit
  to a concert

14
Philosophy of reality Karl Poppers three
worlds (2)

• Aim clarity of terminology and thinking
• Example A visit to concert
• physical walls, floor, violins, human bodies,
  sound waves, frequencies, amplitudes, spectra
• experiential what it sounds like, melodic shape,
  tension-relaxation, sense of time, speed,
  emotion (mood, feeling)
• abstract music notation, program, thoughts,
  historical knowledge, digital representations
• Especially relevant for this course
• physical freq. amplitude spectrum duration
• experiential pitch loudness timbre perc.
  duration
• abstract note dynamic instrument note value

15
Lecture 2, 15.3.06

• Intro to psychoacoustics
• Sound examples
• Frequency perception
• object perception survival
• freq. analysis, physiol., masking, CBW, loudness
• Literature
• Houtsma, A. J. M. et al. ((1987) Auditory
  demonstrations. New York Acoustical Society of
  America.
• Howard, D. M., Angus, J. (1996). Acoustics and
  psychoacoustics. Oxford Focal. Chapter 2 (pp.
  65-91) Introduction to hearing.
• Rasch, R. A., Plomp, R. (1999). The perception
  of musical tones. In D. Deutsch (Ed.), Psychology
  of music (2nd ed., pp. 89-111).
• Terhardt, E. (1988). Psychophysikalische
  Grundlagen der Beurteilung musikalischer Klänge.
  In J. Meyer (Hg.), Qualitätsaspekte bei
  Musikinstrumenten (S.1-15) Celle Moeck.

16
Psychophysics Worlds 1 and 2

• Each experiential parameter depends on each
  physical parameter!
• Sound examples ASA-CD
• Pitch depends on spectrum (missing fundamental)
  (Track 37)
• Timbre depends on temporal envelope backward
  piano (Track 56)
• Loudness does not double when intensity doubles
  (Track 9)
• More examples
• Pitch depends on intensity (Tracks 27-28)
• Pitch salience depends on tone duration (Track
  29)
• Loudness depends on frequency (Tracks 17-18)
• Loudness depends on spectrum (Track 7)

17
Frequency perception Intro

• as opposed to pitch perception
• object perception and survival
• frequency analysis
• physiology
• masking
• critical band
• loudness

18
Survival value of frequency perception

• Darwins theory of evolution
• individual differences (mutation)
• environment danger limited resources
• survival successful reproduction
• Relevance for hearing and music
• aim survival by identifying and describing
  objects
• input to ear superposition of direct and
  reflected sound
• unaffected frequency randomized phase
• frequency is reliable phase is unreliable
• sensitivity to frequency insensitivity to phase

19
Musical implications

• Timbre (identifies sound sources)
• strongly dependent on spectrum (esp. frequencies)
• not directly dependent on waveform (phase)
• Music notation and theory
• primary pitch, time
• secondary loudness, timbre
• irrelevant phase

20
Aural frequency analysis

• Aim identify environmental objects (sound
  sources)
• Approach monitor frequency-time patterns
  (contours)
• Method frequency analysis (separate
  frequencies)

21
Physiology of frequency analysis

• Basilar membrane changes along length
• heavy, floppy end sensitive to low frequencies
• light, tight end sensitive to high frequencies
• Each hair cell on basilar membrane
• responds to limited range of frequencies
• is an auditory filter
• filter bandwidth critical bandwidth

22
Cut-off frequency of a filter
Arbitrary cut-off point 3 dB down from maximum
23
Bandpass filter
bandwidth
A (dB)
f (Hz)
Center frequency
24
Frequency analysis by a filter bank
Harmonic complex tone
25
Critical bandwidth

• Auditory filters have no sharp cut-off
• gt exact value of critical bandwidth is arbitrary
• depending on experimental method
• above about 500 Hz 2...3 semitones
• below about 500 Hz 60...100 Hz (e.g. 80-160 Hz
  1 oct.!)
• Implications for tonal music
• If aim is Separately audible voices in harmony
  and counterpoint
• Then need Separately audible partials in
  sonorities
• Physiology Excite different hair cells with
  different partials
• Result Closer spacing of higher tones in chords

26
Critical bandwidth Bark vs ERB
Bark Eberhard Zwicker et al. (München) ERB
Brian Moore et al. (Cambridge)
27
Auditory masking

• drowning out
• everyday example piano accompanist
• simple example two pure tones
• masked threshold of a pure tone (Mithörschwelle)
• number of audible partials of a complex tone

28
Auditory threshold
29
Masked threshold of a complex tone
30
Loudness

• Depends on
• number of excited hair cells (hence bandwidth of
  sound)
• excitation of each cell (energy in each auditory
  filter)
• Repeat sound demonstration (ASA Track 7)

Critical band
SPL (dB)
Frequency (Hz)
1000 Hz
31
Loudess of a steady-state complex sound

• after Stevens and Zwicker
• within critical bands
• add energy (physical)
• across critical bands
• add loudness (experiential)

32
Revision until Easter

• Read the literature
• Reread the lecture notes
• Ask questions (e.g. email)

33
Lecture 3, 26.4.06

• Pitch of complex tones
• psychoacoustics (explained in lecture)
• neuroscience (read Laden and Zatorre)
• Literature
• Parncutt, R. (1989). Harmony A psychoacoustical
  approach. Berlin Springer. (Chapter 2,
  Psychoacoustics).
• Laden, B. (1994). A parallel learning model of
  musical pitch perception. Journal of New Music
  Research, 23, 133-144.
• Zatorre, R. J. (1988). Pitch perception of
  complex tones and human temporal-lobe function.
  Journal of the Acoustical Society of America, 84,
  566-572
• Handout
• Parncutt, R. (2005). Perception of musical
  patterns Ambiguity, emotion, culture. Nova Acta
  Leopoldina NF 92 (341), 33-47

34
Pitch Introduction

• Abbreviations
• PT pure tone CT complex tone HCT harmonic CT
• SP spectral pitch VP virtual pitch
• Pitch perception according to Terhardt
• SP (analytic) pitch of an audible partial
• VP (holistic) pitch of a complex tone
• Examples
• most consciously noticed pitches are VPs
• pitch at missing fundamental of HCT is a VP (e.g.
  telephone)
• pitch of a heard-out harmonic is a SP
• strike tone of church bell is VP as sound dies,
  hear SPs

35
Harmonic series

• To typical western ears, harmonics no. 7 and 11
• sound noticeably out of tune
• 7 is 1/3 semitone flatter than a m7 above 4
• 11 is about midway between P4 and TT above 8
• The harmonics are
• equally spaced on a linear frequency scale (e.g
  in Hz)
• unequally spaced on a log frequency scale (e.g.
  in semitones)

36
Pitch at the missing fundamentalASA track 37

• Conclusions
• Pitch does not necessarily correspond to a
  partial
• Pitch is multiple/ambiguous
• VP at missing fundamental
• SP at lowest partial

1
2
5
4
3
37
Sound demo Masking SP and VP
ASA-CD tracks 40
41 42
38
Sound demo Masking SP and VPConclusion

• Westminster chimes example demonstrates that
  pitch at missing fundamental is virtual,
  because
• when PT masked by low-pass noise,
• missing fundamentals is audible inside the noise
• If it were physical it would be masked!
• when HCT masked by high-pass noise,
• missing fundamental is inaudible outside the
  noise
• If it were physical it would be audible

39
Sound demo Shift of VPASA-CD Track 39

• Conclusion
• VP corresponds to
• best-fit subharmonic (or approx. fundamental) of
  all partials
• NOT to difference in frequencies

40
Sound demo VP with random harmonicsACA-CD
tracks 43 44 45

• HCTs of 3 random successive harmonics
• harmonic numbers 2 to 6
• (3 possibilities 234, 345, 456)
• harmonic numbers 5 to 9
• harmonic numbers 8 to 12
• Conclusion
• salience of VP depends on effective harmonic
  number of SPs above it
• lower harmonic numbers ? more salient VP

41
Sound demo Strike note of a chimeASA-CD Track
46 47

• 1. hearing out partials
• pure reference tone, then complex test tone
• Can you hear the PT inside the CT?
• Procedure encourages analytic listening
• 2. matching a virtual pitch
• reverse order first complex test tone, then pure
  reference tone
• Do the two tones have the same pitch?
• Procedure encourages holistic listening
• Conclusions
• partials are audible (as SPs)
• the pitch (VP) is ambiguous

42
Experimental determination of pitch

• Question
• Pitch is an experience. How can it be measured?
• Answer
• Compare pitch of two successive sounds
• Assume pitch of one sound is known
• If two sounds have same pitch, pitch of second
  sound is known
• The pitch of a pure tone is assumed
• to be unambiguous
• to correspond to its frequency (provided SPL
  constant)
• Standard experimental method
• Test sound, pause, reference tone (each about
  200-400 ms)
• Listener adjusts frequency of reference until
  same pitch
• A pitch exists when intra- and inter-listener
  agreement

43
Pitch properties of complex tones

• A CT generally evokes several pitches.
• If only one is perceived at a time, the pitch is
  ambiguous.
• If more than one can be perceived at a time, the
  pitch is multiple.
• The pitches of a CT vary in salience, i.e.
  either
• the probability of noticing the pitch, or
• the subjective importance of the pitch

44
Perception of complex tones

• Stage 1 Auditory spectral analysis (Ohm, 1843
  Helmholtz, 1863)
• E.g. A HCT in speech or music typically has 10
  5 audible harmonics.
• Stage 2 Holistic perception of CTs (Stumpf,
  1883 Terhardt, 1976)
• A HCT is normally experienced as one thing
• a complex tone sensation with pitch (VP),
  timbre, and loudness.
• But when partials are heard out, the CT is
  experienced as many things
• pure tone sensations, each with pitch (SP),
  timbre, and salience.

45
Examples of physical spectra (YL) and
experiential spectra (salience) 1. pure tone
(PT on C4)2. harmonic complex tone (HCT on
C4)3. octave-complex tone (OCT on C)Pitch
category 48 C4, 60 C5 etc.(Parncutt, 1989)
46
Terhardts model of pitch perception Input-output

• Input
• physical spectrum of a steady-state sound
• (frequency and amplitude of each partial)
• Output
• experiential spectrum
• (pitch and salience of each tone sensation)
• Aim
• predict experiential spectrum from physical
  spectrum

47
Terhardts model of pitch perception Detail

• 1. masking ? SPs and their saliences
• Nearby partials mask each other more strongly
• Inner partials are masked more than outer
  partials
• 2. recognition of harmonic pitch patterns ? VPs
  and saliences
• Salience depends on
• fit between harmonic template and spectrum
• number and accuracy of matches
• salience of matching SPs
• more salient SPs ? more salient VP
• harmonic number of matching SPs
• lower harmonic nos. ? higher VP-salience
• 3. combination of SPs and VPs ? all pitches and
  saliences
• experiential spectrum contains both
• relative weighting depends on analyic/holistic
  perception

48
Hearing out harmonics (1)Terhardt CD track 17

• HCT, 200 Hz, 10 harmonics
• harmonic numbers 4,3,4,5,6 3 dB
• Conclusion
• SPs exist independently of VP

49
Further sound examples

• See CD in back of Terhardt (1998)

50
Hearing out harmonics (2) Terhardt CD track 18

• HCT, 200 Hz, 10 harmonics
• Pure tone 600 Hz
• Harmonic not heard out
• Same HCT twice, once with missing harmonic
• Attention attracted to replaced harmonic
• Conclusions
• Attention is attracted to changes and differences
• Again SPs exist independently of VP

51
Virtual pitch (2) Terhardt CD track 21

• HCT, 200 Hz, harmonics 6-12 (residue tone RT)
• Pure tone 200 Hz
• Conclusions
• SPs can be heard out if tone is long and constant
• It is possible to attend directly to VP

52
Der Dominanzbereich der spektralen Tonhöhe nach
Terhardt
53
Spectral dominanceTerhardt CD track 23

• HCT, 200 Hz, 20 harmonics
• Non-harmonic CT
• lower harmonics shifted down
• upper harmonics shifted up
• Different boundary frequencies
• 500 Hz VP determined by upper SPs
• 1900 Hz VP determined by lower SPs
• 700 Hz ambiguous

54
Melody of residue tones (1)Terhardt CD track 24

• harmonics 2-4 or 3-5 or 4-6
• harmonics 5-7 or 6-8 or 7-9
• harmonics 8-10 or 9-11 or 10-12

55
Melody of residue tones (2) Terhardt CD track 25

• three randomly selected harmonics from harmonics
  2-9

56
Melody of residue tones (3)

• Chords in equal temperament
• pure tones
• HCTs VP becomes root of major triad

57
Acoustic bass of a church organ Terhardt CD
track 27

• A1 E2 A0?

58
Lecture 4, 3.5.06Consonance and dissonance of
sonorities in western music

• Roughness of harmonic intervals
• critical bandwidth
• pure versus complex tones
• frequency ratios
• Clarity of harmonic function
• fusion
• pitch salience
• cognition of pitch structures
• Familiarity
• historical development of tonal-harmonic syntax

59
Sound exampleTerhardt CD track 8

• harmonic interval of two pure tones

A harmonic tritone of two tones in the middle or
high register is quite smooth!
60
Superposition of two pure tones same amplitude,
similar frequency
f1 1/t1 f2 1/t2 beat freq. fb f2
f1 carrier freq. fc (f2 f1)/2
61
Roughness of a harmonic interval of pure tones

• 20 Hz lt fb lt 300 Hz
• e.g. semitone at 300 Hz, 300320 ? 20 Hz
• e.g. semitone at 600 Hz, 600640 ? 40 Hz
• Two HCTs many contributions to roughness
• fb lt 20 Hz individually audible beats
• e.g. mistuned piano strings
• most prominent near 4 Hz (cf. speech)
• fb gt 300 Hz no roughess
• Isolated HCTs above 300 Hz no roughness

62
Roughness of a harmonic interval of pure tones
Source Campbell Greated (1987). The musicians
guide to acoustics (p.58). New York Schirmer.
Roughness depends on overlap between excitation
functions
63
Roughness of a harmonic interval of pure tones

• Plomp Levelt (1965)

64
Critical bandwidth
65
Roughness of a harmonic interval of HCTs

• Sum roughness contributions from different
  critical bands

E.g. tritone (frequency ratio 11.414) Tone 1
1000 2000 3000
4000 5000 6000 7000 Tone 2
1414 2828
4242 5656
7070 Frequency ratios between almost coincident
harmonics 1.06
1.06
1.01 (1.06 corresponds to one semitone)
66
Roughness of a harmonic interval of HCTs

• Predictions according to Plomp Levelt

67
Frequency ratios of intervalsWhich one is the
right one?
interval note chr. pure ratio
Pythagorean P1 C 0
11 11 m2 C 1 1615
256243 M2 D 2 98
98 m3 D 3 65
3227 M3 E 4 54
8164 P4 F 5 43
43 TT F 6 4532
729512 P5 G 7 32
32 m6 G 8 85
12881 M6 A 9
53 2716 m7 A 10
95 169 M7 B
11 158 243128 P8 C
12 21 21
68
Frequency ratios of intervals

• Calculating intervals
• e.g. m7 P5 m3 3/2 x 6/5 9/5
• Pure tuning
• combinations of P8, P5, M3
• Pythagorean tuning
• combinations of P8, P5
• frequency ratio always in the form 2n/3m or 3m/2n
• Interval (cents) log2 (f1/f2) x 1200

69
Origins of musical scales

• Ancient western music assumptions
• vocal melody, oral tradition
• tuning of successive intervals by ear
• Role of successive P8, P5, P4 intervals
• theory of tonal affinity
• coinciding harmonics (Helmholtz)
• coinciding pitches (Terhardt)
• singers approach consonant intervals by trial and
  error
• audible difference between P8 M7/m9, P5
  TT/m6, P4 TT/M3
• Limitations on accuracy of intonation in vocal
  performance
• vocal limitations, e.g. jitter (even when no
  vibrato at all)
• perceptual limitations, e.g. (lack of)
  sensitivity to slow beats

70
Evolution of standard western scales

• Standard pentatonic/heptatonic
• a series of P5/P4s
• F C G D A ? F C G D A E B
• These P5/P4s are not very exact! ( 20-50 cents?)
• Chromatic scale
• add m2, M3 or P4 to diatonic tones, e.g. F/Gb
  is
• F m2, G - m2 (midway between F and G)
• D M3
• B P4
• Underlying assumption
• consonance is important and is preferred
• culture-specific concept and role of consonance

71
Clarity of harmonic function

• Theory of harmonic function
• Riemann (S D T usw.)
• Major and minor triads
• high clarity ? more common?
• Diminished and augmented triads
• low clarity ? less common?
• Clarity of harmonic function
• fusion (Stumpf)
• salience of virtual pitch at root (Terhardt)
• Cognitive theory
• Pitch structures are easier to understand (?
  more consonant) if they have clear reference
  pitches (roots and tonics).

72
Familiarity

• Historical development of tonal-harmonic syntax
• Historical listeners are familiar with the syntax
  of their period
• Example dominant seventh chord (e.g. GBDF)
• In musical practice
• in 1500 prepared or accidental
• in 1600 unprepared in Monteverdi
• in 1700 often unprepared but still dissonant
• in 1800 increasingly consonant
• in 1900 as if consonant
• In music theory
• before 1700 non-existent
• after 1800 universally recognized

73
Tenneys Consonance-Dissonance Concepts
74
Consonance-dissonance of sonorities in western
music

• Three perceptual factors
• 1. roughness
• peripheral origin
• 2. clarity of harmonic function
• central origin
• 3. familiarity
• depends on musical syntax
• Are they independent?
• 1 is perceptually independent of 2
• but 3 depends on 1 and 2

75
Lecture 5, 10.5.06Categorical perception

• Perception and cognition of music
• CP and the three worlds of Popper
• CP of relative pitch (versus intonation)
• CP of absolute pitch
• CP of rhythm (versus rubato)
• Evolutionary music psychology
• Why does music have pitch and time categories?
• Implications for origins and prehistory of music

76
Non-musical categorical perception

• Color
• red range of light wavelengths
• nature depends mainly on rods and cones
• nurture also depends on culture/learning
• Speech sounds
• The vowel /a/ has specific formant frequencies
• nature all formants are near 500, 1500, 2500
  Hz
• nurture formant frequencies of /a/ are learned
  from speech (? culture-specific)

77
Categorical perception and the three worlds of
Popper

• Psychophysics
• relationships between Worlds 1 2
• E.g. SPL of just audible pure tone
• Categorical perception
• conceptually between worlds 2 3
• empirically between worlds 1 3
• Examples
• range of frequency ratios of M3 interval
• scale step, duration, instrument, dynamic
• range of any continuous parameter corresponding
  to any label

78
Experiment on categorical perception of musical
intervals(Burns Campbell, 1994)
Stimuli Melodic intervals of complex tones
all ¼ tones up to one octave. Participants Mus
icians Question Which of 24 categories
(quarter tones)?
79
Results(Burns Campbell, 1994)

• All intervals on a continuous scale are
  categorized
• Familiar categories are
• broader
• more often selected
• Category centres
• familiar tuning (equal temperament)
• Category width
• distance between familiar categories

80
Another psychological definition of categorical
perception

• Heightened discrimination near category boundary
• Just noticeable difference (JND) is smaller at
  boundary
• E.g. frequency JND of successive pure tones,
  central range 110 cents
• This definition
• Does not necessarily hold for musical categories
• Is not assumed here

81
Categorical pitch perception versus intonation

• Hard to distinguish empirically. Whats the
  conceptual difference?
• Categorical perception
• label in World 3
• meaning
• Intonation
• pitch in World 2
• experience

82
What influences intonation?

• Real-time frequency adjustment in music
  performance
• depends directly on many factors!
• octave stretch
• beating of coinciding partials
• context, implication (leading tone)
• whether soloist (sharp) or accompaniment (flat)
• emotion (e.g. tension-release)
• timbre (deep low)
• clarity preference for equal spacing in
  chromatic or diatonic scale
• separation of major and minor modes
• pitch salience less stable tones are more
  variable
• ? Hard to investigate scientifically hard to
  isolate one factor

83
Intonation and enharmonic spelling

• E.g. F is usually sharper than Gb, but
• there are many different kinds of F and kinds of
  Gb
• enharmonic spelling is often ambiguous (and there
  is no clear rule)
• F can be lower than Gb if intonation approaches
  just (slow tempo, constant tones)
• ? Intonation does not depend directly on
  enharmonic spelling

84
When is a tone in tune?

• Two different ranges
• Category width corresponding to scale step
• say, 50-100 cents
• In-tune range (good timbre?)
• say, 10-30 cents
• Role of context
• Both category width and in-tune range are smaller
  when
• slower music (longer tones)
• less vibrato
• more familiar tuning
• more exact tuning
• higher pitch salience
• central pitch range

85
Absolute pitch

• Absolute perception is normal
• e.g. colour, vowel quality
• also across senses synaesthesia, chromasthesia
• AP is actually absolute chroma
• AP possessors are no better at naming register
• AP can apply either to individual tones or whole
  pieces
• E.g. ask listener if well-known piece is in the
  right key

86
AP is learned

• Pianists label white keys more easily
• because played more often or clearer label
• Everyone has some AP
• non-musicians tend to sing in right key (Levitin,
  1994)
• AP involves both long-term memory and labeling
• Only musicians can apply musical pitch labels
• AP is acquired in a critical period (like
  language?)
• provided there is sound-label relation and
  repetition
• Limits of AP also support learning
• semitone errors (from pitch shifts?)
• octave errors (from pitch ambiguity?)

87
Absolute versus relative pitch perception

• Both are
• examples of categorical perception (pitch or
  interval)
• defined by chromatic scale, accuracy 50-100
  cents
• Properties
• weak correlation with other musical or perceptual
  skills
• many have it to some extent (also non-musicians)

88
Musical rhythm as categorical perception

• Examples
• swing ratio 21 vs dotted rhythm 31
• triplet 111 vs 112
• Each category has
• a range of possible realisations (rubato)
• that depends on context
• triple meter makes 111 more likely
• duple meter makes 112 more likely

89
Pitch-rhythm analogy
90
Why does music have pitch time categories?

• Music must be stored and reproduced either as
• oral tradition or
• notation
• to acquire meaning in a cultural context
• Music can be stored in
• World 3 (memory in oral tradition)
• World 3 (notation) or
• World 1 (sound recording)
• categories are necessary
• amount of information is limited by cognitive
  capacity

91
Speech versus song

• Speech categories are phonemes, words
• Song categories are pitch, rhythm
• In both cases
• Categories have meaning
• Categories are part of culture

92
Origins of music

• What motivates/d people to create pitch/rhythm
  categories?
• Practice for cognitive system
• Emotional communication ? social cohesion
• Babies prelinguistic communication
• Fetus perception of maternal state

93
Prehistory of music

• Observation
• Songs in different oral traditions
• include P8, P5 and P4 intervals between scale
  steps
• Duple and triple rhythms, or time ratios of 12
  and 13
• How did this happen? A theory
• Arbitrary starting point
• Songs with arbitrary pitch and rhythm categories
• Process
• singers vary performance randomly or
  deliberately, by trial and error
• clearer structures are easier to remember
• pitch P8 or P5 between scale steps (pitch
  commonality)
• rhythm 21 and 31 ratios (pulse)
• Leads to
• simple scales (pentatonic or diatonic) and
  meters

94
Evolutionary theory
95
Musical diversity

• The described evolutionary process does not
  produce simplicity or monotony, but rather a wide
  range of musical styles. Possible explanation
• music has a wide range of social and cultural
  meanings and functions
• complexity can be preferred for representational
  or aesthetic reasons

96
Lecture 6, 17.5.06Test

• 5 questions _at_ 10 minutes 50 minutes
• Your options
• I will grade your paper if you want and give it
  back to you in my Sprechstunde.
• The grade for the test will not have any effect
  on your final grade.
• Tips on how to answer the questions
• Read the question carefully and ask yourself why
  exactly those words were chosen.
• Answer only the stated question dont talk
  around it.
• Think about your answer before you begin. Quality
  is more important than quantity.
• Structure your answer clearly, following the
  structure of the question (a, b).
• Write clearly and legibly. Begin each answer on a
  new page.
• If appropriate, incorporate diagrams and refer to
  them in the text.

97
Philosophy of perceptiona. Apply Poppers
concept of the three worlds to the art of
cooking b. to the description of a group of
people eating a meal in a restaurant c. to the
description of an experiment to investigate the
perception of i. the flavour of a piece of food
or ii. of an entire dish.

• POSSIBLE ANSWER
• Cooking involves ingredients (world 1), flavours
  (world 2) and recipes (world 3).
• The people sitting together at the table put the
  food in their mouths (world 1), experience the
  flavours, the feeling of being hungry or full,
  the company, etc. (2), and exchange information
  about the food and other topics (3).
• i. Participants are blindfolded and given
  different pieces of food whose texture is
  identical. They are asked to describe the taste
  in words (qualitative approach) or rate the
  similarity of two tastes on a 7-point scale
  (quantitative approach).
• ii. Gourmets rate the food in a restaurant
  qualitatively and/or quantitatively. Their
  ratings depend on the individual flavours, the
  combination, the visual impression, the ambience
  etc.

98
2. Spectral analysisa. Why does the ear separate
high frequencies from low frequencies?b. The
separation is imperfect and has limits. Why?

• POSSIBLE ANSWER
• The main function of hearing is to identify and
  describe sound sources in everday environments.
  The sound reaching the ear is mostly a
  superposition of directed and reflected sound. In
  this process, phase information is completely
  lost and amplitude information distorted. But
  provided the source and perceiver are moving much
  slower than the speed of sound, the ear can
  always rely on frequency information. Therefore
  the ear has evolved to be sensitive to frequency
  and to analyse a sound into its component
  frequencies.
• According to the uncertainty principle in
  physics, it is impossible to simultaneously
  extract both spectral and the temporal structure
  of a signal with perfect accuracy. If the
  effective window duration is long, the frequency
  information is more exact and the temporal
  information is less exact. The temporal envelope
  of the ear has evolved to allow both the
  important spectral and the important temporal
  aspects of environmental sounds that are
  important for humans, especially speech, to be
  perceived.

99
3. Pitcha. Describe the perception of the
pitches of a church bell using the terminology
spectral pitch and virtual pitch.b. Explain why
the bell is perceived in this way.

• POSSIBLE ANSWER
• The spectrum of a bell sound is inharmonic, but
  typically some of the partials correspond to an
  incomplete harmonic series. The pitch that we
  tend to hear at the start of a bell sound (the
  strike tone) corresponds to the fundamental of
  the clearest, most complete harmonic series
  within the spectrum and is therefore a virtual
  pitch. As we listen to the sound decay, we can
  sometimes hear individual partials, whose pitches
  are spectral pitches.
• The main function of hearing is to identify and
  describe sound sources. In general it helps if
  this happens as quickly as possible. Therefore
  pitch perception is geared toward holistic
  perception (corresponding to the sound source) of
  the onset of a sound (so that a quick decision
  can be made). The pitch at the start of a bell
  sound is this kind of pitch. Only once the bell
  has been identified and described can the
  listener hear the bell in a different (analytic,
  slow) way that is less closely related to
  evolution and survival.

100
4. Consonancea. Why and in what sense is a
harmonic tritone of pure tones in the middle or
upper register consonant? B. Why and in what
sense is a harmonic tritone of harmonic complex
tones in the middle or upper register dissonant?

• POSSIBLE ANSWER
• A harmonic tritone of pure tones in the middle or
  high register typically spans an interval greater
  than a critical band (which is 2-3 semitones in
  high registers). In general, such a dyad sounds
  completely smooth, because there is no
  interference between the two tones on the basilar
  membrane. The dyad is consonant in the sense that
  it has no roughness, but in a musical context it
  may be perceived as dissonant because of
  associations with musical syntax or because the
  harmonic function of the interval is ambiguous.
• The upper partials of a harmonic tritone of
  harmonic complex tones form several intervals of
  a semitone. For example, the third harmonic of
  the lower tone is one semitone away from the
  second harmonic of the higher tone. These
  semitone intervals are perceived as rough,
  especially if the amplitudes of the pure tones
  are similar. Therefore, the whole dyad is
  perceived as rough. The dyad is dissonant in this
  sense. But if it is presented in isolation it is
  not necessarily dissonant in the sense of
  harmonic clarity or unfamiliarity.

101
5. Intonation a. Give three possible reasons why
the tone F might be performed sharp relative to
Gb.b. To what extent and in what sense does
intonation depend on enharmonic spelling?

• POSSIBLE ANSWER
• i. F might be performed sharp relative to Gb
  because the performer wishes to communicate the
  expectation that it will rise to G (leading tone
  effect),
• ii because the performer wishes to make clear
  that the interval above D is a major and not a
  minor third, or
• iii because F is a perfect fifth above B, which
  in turn is a perfect fifth above E (summing
  fifths results in Pythagorean tuning and the
  corresponding major third, 8164, is bigger than
  the pure or just major third, 54).
• b. Intonation may depend on harmonic spelling if
  a performer is (sight-) reading believes that
  sharps are sharper than enharmonically equivalent
  flats. If not, the connection is indirect.
  Intonation is primarily determined by the sound
  and not by the notation. In some cases this can
  lead to the above effect, but it can sometimes
  lead to the reverse. For example if the tones are
  long and constant, beating between upper partials
  may be reduced if major thirds are tuned to just
  intonation (54). In this case, F is 5/4 times
  the frequency of D and Gb is 4/5 of the frequency
  of Bb. Since (5/4)3 125/64 lt 2, three just
  major thirds add to less than an octave, and F
  would be flatter than Gb in this case.

102
Lecture 9, 7.6.06

• Auditory scene analysis
• Perception of counterpoint

103
Auditory scene analysis

• How does the ear recognize and monitor sound
  sources?
• Thought experiment (Bregman, pers. comm.)
• Lake with two boat ramps (inlets)
• Leaf floating on water in each
• Task from his motion, identify and describe
• people and fish swimming
• boats and water skiers going past
• a stone or a feather hitting the water
• Impossible? Exactly analogous to auditory
  perception!

104
Gestalt principles in vision

• Proximity
• grouping of nearby dots
• Similarity
• grouping of similar dots
• Closure
• recognition of incomplete patterns
• Good continuation
• e.g. 2 lines crossing

105
Gestalt principles in music

• Perceptual coherence of melody
• Proximity small intervals in pitch and time
• Similarity constant timbre
• Closure hearing missing or inaudible tones
• Good continuation rising pattern continues to
  rise

106
Pitch proximity in melody
After Huron
107
Temporal proximity
Distribution of note durations in 52 instrumental
and vocal works (Huron) Dotted line upper and
lower voices of J.S. Bach's two-part Inventions
Dashed line 38 songs (vocal lines) by Stephen
Foster. Solid line mean Bin size 100 msec.
Assumed tempi typical recordings.
108
Proximity in pitch and timevan Noorden, 1975
the perceptual origin of the step-leap
distinction
109
Competition between Gestalt principles

• Proximity
• small intervals in pitch and time
• Good continuation
• rising pattern continues to rise
• Example of conflict between principles
• elements of rising pattern not proximate
• reversal of direction after leap
• crossing parts
• See next slide

110
Part crossing
Good continuation dominates
Pitch proximity dominates
111
Foreground and background

• foreground perceived object
• attention ? foreground
• Prerequisite for perception of object
• separation of foreground elements from background
  elements
• group elements within foreground
• perhaps also within background
• separate foreground from background

112
Perception of melody versus accompaniment

• grouping of foreground
• proximity, similarity
• separation of foreground from background
• common fate (assume non-parallel motion)

113
Explanation and generalizationThe auditory scene

• Graph of frequency (SP) against time (3rd dim.
  SPL?)
• showing patterns of
• audible partials (pure-tone components)
• noise
• Auditory scene analysis (ASA Bregman)
• separation of signal ( source) from noise
  (background) by
• integrating (grouping) signal (grouping events)
• segregating (separating) signal from background

114
Grouping principles in ASA

• Sequential (temporal, melodic) integration
• proximity (pitch, time, location)
• similarity (timbre, loudness)
• lack of sudden changes
• Simultaneous (spectral, harmonic) integration
• simultaneity of onsets
• coherence of changes
• frequency, SPL, spectral envelope
• harmonicity

115
Examples (Bregman CD)see Traube lecture

• Sequential integration (melody)
• Streaming and implied polyphony
• 1. melodic aspect
• 3. rhythmic aspect
• Musical examples
• 6. Telemann Sonata in C (from Der getreue
  Musikmeister)
• 7.-9. East African Xylophone
• Competition between principles
• 17. Part crossing (proximity versus good
  continuation)
• Spectral integration (VP)
• 18. Mistuning of a harmonic partial
• 24. Coherent modulation of frequency

116
Origins of ASA principles

• Interaction with physical and acoustical world
• Nature phylogenesis
• Nurture ontogenesis
• Domains
• human communication speech, music
• natural environment animal sounds
• artificial environment machines

117
Perception of Counterpoint

• Compositional rules and conventions
• History of music theory and pedagogical systems
• Modern normative harmony texts
• Dependence on perception versus style
• nature versus nurture
• universal versus culture-specific

118
Perception of Counterpoint

• Goals (what composers want to achieve)
• True counterpoint requires separately
  perceptible melodies
• clear voice-leading auditory streaming
• Means (compositional techniques)
• salient pitches
• harmonic complex tones, central range, legato
• within-voice coherence
• integration, fusion
• between-voice independence
• fission, segregation
• Rules (compositional conventions)
• sometimes explicit, sometimes not
• remarkably unchanged since medieval polyphony

119
Perception of Counterpoint

• Main source
• Huron, D. (2001). Tone and voice A derivation of
  the rules of voice-leading from perceptual
  principles. Music Perception, 19, 1-64.

120
Tone type

• Compositional rule
• Prefer harmonic complex tones
• Perceptual explanation
• High pitch salience
• Origin
• Human voice and speech communication
• Implication
• One of many culture-specific aspects

121
Salience of the strongest VP of a harmonic
complex tonecalculated after Terhardt et al.
(1982)
122
Registral compass

• Compositional rule
• Registral compass F2 to G5
• Perceptual explanation
• Virtual pitch salience of HCTs is maximum near
  300 Hz
• Origin
• Pitch range of human voice
• Implication
• middle C is the middle of something!

123
Temporal continuity

• Compositional rule
• Prefer sustained, legato tones
• Gaps between staccato tones lt 1 second
• Perceptual explanation
• Duration of echoic memory
• Coherence of melodic stream
• Implication
• Importance of legato for singing and instrument
  construction

124
Critical bandwidth in semitonesafter Moore
Glasberg 1983
PT-chord-spacing that minimizes masking and
roughnessafter Huron
125
Chord spacing

• Compositional rule
• More space between tenor and bass
• Perceptual explanation
• Minimum masking ? pitch salience
• Minimum roughness
• Both determined by critical bandwidth
• Implication
• active bass line is possible and ok

126
Doubling

• Compositional rule
• Dont double leading or chromatic tones
• Perceptual explanations
• Avoid parallel octaves (common fate)
• Clarify tonality by reinforcing tonally stable
  pitches (see later lecture on tonality)

127
Consonance and prevalence of harmonic intervals
Line sensory consonance of dyads of complex
tones (Kaestner, 1909) Bars interval prevalence
(Huron 1991) in the upper two voices of J.S.
Bach's three-part Sinfonias (BWV 787-801) Note
discrepance at P1 and P8!
128
Consonance

• Compositional rules
• Prefer consonances to dissonances
• But also avoid P1s and P8s (also P5s)
  contradition!
• Regardless of temporal context
• Perceptual explanation
• harmonicity
• ? more consonance usually desirable in western
  music
• ? more fusion not desirable in deliberately
  polyphonic music
• Implication
• Consonant sonorities are more prevalent (also
  triads, tetrads)
• Triads and sevenths should contain all pitch
  classes

129
Stepwise motion

• Compositional rule
• Prefer steps to leaps
• Fewer leaps at faster tempos
• Increase duration of tones forming leaps
• Both in composition and performance
• Perceptual explanation
• Proximity in pitch and time (cf. Noorden)
• Trill threshold corresponds to critical
  bandwidth?

130
Similar motion

• Compositional rule
• Prefer contrary to similar motion
• Perceptual explanation
• Avoid fusion
• Implication
• Two-part counterpoint favours thirds and sixths

131
Parallels

• Compositional rule
• Avoid parallel octaves and fifths
• Perceptual explanation
• Octaves/fifths AND parallel motion promote fusion

132
Part crossing

• Compositional rule
• Avoid part crossing
• Perceptual explanation
• Pitch proximity is stronger principle than good
  continuation

133
Outer voices

• Compositional rule
• Apply rules more strictly to outer voices
• Perceptual explanation
• Pitch salience masking from one side only

134
Leap resolution

• Compositional rule
• Follow leap by step in opposite direction
• Perceptual explanation
• Pitch proximity between non-successive tones

135
Onset asynchrony
Onset synchrony for 10 of Bach's 15 two-part
keyboard Inventions Non-zero phase means that one
voice is shifted relative to the other
136
Onset synchrony

• Compositional rule
• Avoid onset synchrony
• Perceptual explanation
• Cue to fusion
• Evidence
• Bach avoids onset synchrony in counterpoint When
  voices shifted relative to each other, onset
  synchrony is a minimum at zero shift

137
Perception of simultaneous tones
Stimuli sonorities of octave-comlex tones Task
how many tones? Source Parncutt (1993)
138
Number of active voices
Voice-tracking errors while listening to
polyphonic music (Huron) Listeners musicians
Task How many voices do you hear? Music
polyphonic textures with homogenous timbre Solid
columns mean errors expert musician subjects
Shaded columns unrecognized single-voice
entries
139
Number of active voices
Task how many voices do you hear? ? mean
auditory streams
140
Textural density

• Compositional rule
• No more than three voices can be active
• Perceptual explanation
• Listeners cannot count more than three
  simultaneous tones or voices

141
Timbral differentiation

• Compositional or performance rules
• A different timbre for each voice
• Vibrato only in the solo voice
• Instruments or loudspeakers at different
  locations
• Perceptual explanation
• Stream segregation

142
Combinations of rules

• Compositional rule
• If voice-leading weakened by violating one rule,
  compensate by obeying other rules more strictly
• E.g.
• Oblique or step motion to perfect consonances
• In similar motion, prefer steps to leaps
• When approaching a perfect consonance, avoid
  synchrony

143
Textural density

• Compositional rule
• Write in 3 to 6? parts
• Perceptual explanation compromise between
• Optimal roughness
• Optimal tonalness
• Maximum number of active voices

144
Conclusion

• Many rules have a perceptual basis
• Not necessarily universal
• Culture-specific
• Complexity and polyphony (notation)
• Independence of voices

145
Lecture 10, 14.6.06Western harmony and tonality

• An analogy between
• 1. Perception of harmonic complex tones
• salience and ambiguity of virtual pitches
• 2. Perception of musical chords
• salience and ambiguity of root
• 3. Perception of major-minor tonality
• salience and ambiguity of tonic

146
Background in music theory

• The root of a chord
• No general theory!
• desirable
• predict the root of any chord
• a problem that theorists never solved
• root of the minor triad
• Major-minor tonality
• Why two modes - not one or three?
• Why these scales - not others?

147
Musical pitch terminology

• Pitch classes or pcs
• Pitch in chromatic scale without specifying
  octave register
• 0C, 1C, 2D
• Pitch-class sets
• CEG 047
• CEbG 037
• CEbGb 036
• CEG 048

148
Background in music psychology Krumhansls tone
profiles
Stability of scale degrees in major and minor
scales
149
Krumhansls tone profilesExperimental method

• Musical context a well-defined major or minor
  key
• E.g. SDT cadence
• Probe tone
• Every degree of the chromatic scale
• How well does the tone go with the context?
• 1 very poorly 7 very well
• Mean results
• ? the relative stability of the 12 pcs

150
Octave generalizationoctave-complex tone (OCT,
Shepard tone)
151
Octave-complex tones (OCTs)
C
V
B
Z
D
W
A
CWG
CEG
E
Y
F
G
X
152
Origin of Krumhansls tone profiles
153
Background in psychoacoustics

• Pitch of a complex tone according to Terhardt
• Pitch Poppers world 2 (experiential, not
  physical!)
• Spectral pitch SP
• Pitch of a pure tone
• Virtual pitch VP
• Pitch of a complex tone
• salience
• Perceptual importance of a pitch
• Probability of perceiving a pitch spontaneously

154
Pitch perception Terhardts experimental method

• a complex test tone alternates with a pure
  reference tone
• The listener adjusts the frequency of the pure
  tone until the two tones have the same pitch
• The salience of a pitch the probability of
  matching it

155
Physical and experiential spectra 1. pure
tonePT (C4)2. harmonic complex tone HCT
(C4)3. octave complex tone OCT (C)
156
The harmonic series as a pattern-recognition
template
1
1
1
2
3
4
5
6
7
8
9
10
157
Terhardts virtual pitch algorithm

• Spectral analysis
• ? frequencies and amplitudes of pure tones
  (partials)
• Masking
• ? audibility of pure tones
• Spectral dominance region
• around 700 Hz (between the first two formants of
  vowels)
• ? salience of spectral pitches SPs
• Recognition of harmonic pitch patterns
• ? Virtual pitches VPs

158
Octave generalisation of the harmonic
template(Parncutt, 1988)
The five root-support intervals
P1
P5
M3
m7
M2
159
Circular representation of the harmonic template
P1
m7
M2
M3
P5
160
Major triad CEG 047 notes
pitches
161
Minor triad CEbG 037 notes
pitches
162
Diminished triad CEbGb 036 notes
pitches
163
Augmented triad CEG 048notes
pitches
164
Matrix multiplicationnotes x template saliences
notes 1 0 0 0 1 0 0
1 0 0 0 0
template
saliences 18 0 3 3 10 6 2 10 3 7 1 0
165
Experimental data(Parncutt, 1993)
Diamonds Mean ratings Squares Theoretical
predictions
166
Tonal stability and pitch salience in the tonic
triad
167
Tonal stability and pitch salience in the tonic
triad

• At the end of a phrase of tonal music
• Closure produced by last tone
• salience of that pitch within the tonic triad
• Tonic of major/minor tonality is a chord
• Tones of major/minor scales
• salient tones within tonic triad
• (exception leading tone)

168
Origins of major-minor tonality

• 1. Tonality in general (prehistory)
• preference for
• clear structures
• easy to remember (oral transmission)
• pitch hierarchies
• some pitches clearly more prevalent
• clear phrases
• more