Antnio Pedro Oliveira

1
Creative Generation ofAffective Music

• António Pedro Oliveira
• DEI - FCTUC
• 21/12/2006

2
Outline

• Introduction
• State of the Art
• Architecture
• Work to be done
• Experiments
• Examples
• Conclusion

3
Introduction

• Problem
• No automatic method to detect emotions related to
  music
• Music induce different emotions on different
  listeners
• Context, user profile
• Hypothesis
• Music is the main language of emotions

4
Introduction

• Motivations
• Music is a ubiquitous media
• Cinema, TV, radio, dance, transport, shopping
  centres, etc.
• Emotional content can be influenced by specific
  musical features
• Objectives
• Discover the isomorphic mapping between musical
  features structure and emotions structure
• Generate music to induce emotions
• Retrieve music features
• Sequence, compose music

5
Introduction

• Methodology
• Music - Select, manipulate, sequence music
  samples, adapting to intended emotion
• Audience - Compare intended and recognized
  emotions using Affective Computing technology

6
State of the Art

• Music Psychology
• Emotions and Music
• Music perception and cognition
• Music theory
• Computer Music
• Affective music generation
• Algorithmic composition

7
State of the Art

• Musical Signal Processing
• Music Features Extraction
• Melody
• Harmony
• Rhythm
• Instruments
• Dynamics

8
State of the Art

• Affective Computing
• Psychophisiological techniques for emotion
  recognition
• 2 Dimensional
  Space

9
System stage 1

• Architecture

10
System stage 2

• Architecture

11
System stage 3

• Architecture

12
System stage 4

• Architecture

13
Work to be done

• Research Topics
• Music Psychology
• Computational models to test isomorphic mappings
  between musical features structure and emotions
  structure
• Computer Music
• Algorithms for Affective Music Composition
• Automatic Affective DJ
• Bridge the semantic gap in Music Features
  Extraction
• Affective Computing
• Algorithms for emotion induction

14
Experiments

• Knowledge Base Construction
• N relations between musical features and emotions
• Music Base Labelling
• Music Features Extraction
• Music Generation
• Music composition, selection
• Emotions Recognition
• Affective Computing (Psychophisiological
  techniques)

15
Examples

• Rhythm
• Tempo X BPM
• Emotions

Bob Sinclair
Zero 7
16
Examples

• Melody
• Melodic motion - pitch variation, notes density
• Emotions

Sheryl Crow
R.E.M.
17
Examples

• Harmony
• Consonance music tension
• Emotions

Damien Rice
Faithless
18
Examples

• Instrumentation
• Timbre Spectral features
• Emotions

The Corrs
Louis Armstrong
19
Examples

• Dynamics
• Loudness X Energy
• Emotions

Green Day
U2
20
Conclusion

• Scientific contributions
• Knowledge base with relations between musical
  features and emotions
• System to be used by musicians, psychologists,
  health scientists
• Engineering contributions
• Unifying functional system
• Autonomous DJ application
• Artistic contributions
• Composition of new pieces of music

21
Conclusion

• Bring the knowledge of this cross-disciplinary
  approach into a computer system
• System advantages
• Automatic Affective Music Generation
• Computer games, movies, dance, etc.
• Flexibility to update
• Knowledge Base useful for Music Psychology
• Music Base - useful for Computer Music