GeneSynth

1
GeneSynth

• Sound Synthesis by Genetic Algorithm

2
Introduction

• Genetic Algorithms (GA)
• Search Algorithm
• Uses Darwin-inspired survival of the fittest
  heuristic.
• Used In a wide range of fields
• TSP, Financing, Music
• But not for sound synthesis
• Search space is too large

3
GeneSynth

• Experiment in using Gas to synthesize useful
  sounds
• Uses an input sound as a target to model as a
  heuristic
• Motivation
• Musical
• Analysis

4
Goal 1 Sound Fitting

• Take an audio file and fit other user-specified
  sounds using the audio file as a guide
• Example
• Beethovens Fifth symphony, 1st movement as a
  target
• Coin-jingling sounds as specified sounds
• Result Coin samples placed to sound as close as
  possible to the symphonic version.

5
Goal 2 Family Tree Exploration

• Many search algorithms have a history associated
  with each solution
• GAs are no exception
• A GA solutions history resembles a family tree
• Because GAs create solutions by pairing two other
  solutions together
• The ancestors and siblings of a good solution may
  yield interesting sounds for composers.

6
Goal 3 Modelling

• If a perfect or near-optimal solution is reached,
  it can be used as a model
• The model can then be modified in an interesting
  way and resynthesized to create a new useful
  result
• But how easily will a perfect model be reached?
• It depends on the sounds we select for use in the
  algorithm, as well as the efficiency of the GA.
• If we use a sinusoidal unit generator, the
  problem begins to resemble the phase vocoder.

7
The classical GA

• Origins in the mid 80s
• Many variants have since been created.
• GeneSynth is one of these variants
• The classical GA must be understood to appreciate
  the variants

8
GA Overview and Terms

• Chromosome a string of data which represents a
  solution. Example
• 00001011 (a set of 8 bits that represents the
  number 11)
• Individual the solution represented by a
  chromosome
• 11 in the above solution.
• Population a set of chromosomes. Example
• 00001011,00100011,10110000 (three chromosomes)

9
More GA Terms..

• Mutation function a function that changes
  sections of the chromosome based on some
  probability. Example
• M(01001011) -gt 00001010
• (Two bits are flipped, changing the chromosome)
• Crossover function a function that takes two
  chromosomes and combines them together to result
  in two new chromosomes
• C(10001010, 00001111)-gt 10001111,00001010
• (The original chromosomes swap halves)

10
GA terms and overview

• Fitness Function a function that assigns a score
  to an Individual, based on how good a solution it
  is.
• If the problem is to find the number close to 77,
  a fitness function could be
• F(x) 77 x

• Psuedocode
• Create Initial Population
• Repeat
• Score Each Individual
• Select pairs of good-ranking individuals
• Mutate the selected individuals
• Crossover the individuals
• Until goal score is met

11
A simple example

• Suppose our problem is to find the number 77.
• We will use the encoding AND functions in the
  above examples.
• We randomly create a set of 8 bit strings to
  start off the population of size four.
• 00100010, 11001010, 00010000,11110000
• We then score each individual and create the next
  four via mutation and then crossover.
• This process Is repeated until we have found the
  number 77.

12
Classical GA vs GeneSynth

• Fixed Length Chromosome
• Independent chromosome structure
• Fixed Mutation and crossover probabilities

• Variable Length chromosomes
• Self-referential chromosome structure
• Mutation and Crossover Probabilities vary over
  evolution

13
Chromosome Structure

• A bottom up example
• A SoundRoot is a unit generator, which takes
  parameters and creates audio by some method
• Ex. Sin(freq,phase), AudioFiles(FileName)
• A Transformation takes existing audio and
  parameters and processes it
• Ex. Distortion, Normalization, Amplitude
  Modulation
• SoundRoots and Transformations are the basis of
  sound generation for GeneSynth

14
Chromosome Structure

• A SoundCell is a unit that contains one
  SoundRoot, and any number of SoundCells and
  Transformation
• Note the recursive definition.
• A SoundCell specifies a
• sound entity that will be used
• some number of times in the
• synthesis

SoundCell
Transformations
SoundRoot
Child SoundCells
15
Chromosome Structure Genes

• A CellDefGene is a set of SoundCells.

• A PlaceGene is a set of several pieces of
  information
• A reference to a Cell In the CellDefGene,
• A time and volume to place the sound

CellDefGene
SoundCells
Two PlaceGenes
1.4 secs -6db
3.12 secs -8 db
16
The chromosome structure

• So, a chromosome is a CellDefGene and some number
  of PlaceGenes.

Chromosome
PlaceGenes
CellDefGene
1.4 s -6db
1.1 s -8db
0.4 s -2db
1.4 s -8db
3.1 s -6db
2.8 s -6db
17
Chromosome to Individual

• If we realize every SoundCell at the times and
  levels specified by the PlaceGenes, we have
  generated our individual, an audio file.

Chromosome
PlaceGenes
CellDefGene
1.4 s -6db
1.1 s -8db
0.4 s -2db
1.4 s -8db
3.1 s -6db
2.8 s -6db
18
Fitness Function

• The fitness function uses a target soundfile, and
  compares the individual soundfile against it.
  This is done by
• Taking FFTs of both, and comparing them
• Sums the decibel rating of the ratio of each
  bucket in the FFT.
• Taking the RMS value of both and comparing them

19
Mutation

• A chromosome mutates by randomizing some part of
  it. In GeneSynth,
• a new SoundCell can be created or deleted, Its
  parameters randomized, or one SoundCell might
  attach another to it.
• A PlaceGene will mutate by changing the SoundCell
  it refers to, or by changing its amplitude or
  time.

PlaceGenes
CellDefGene
1.4 s -6db
1.1 s -8db
0.4 s -2db
1.4 s -8db
3.1 s -6db
2.8 s -6db
PlaceGenes
CellDefGene
1.4 s -6db
1.1 s -8db
0.4 s -2db
1.4 s -8db
3.1 s -6db
2.8 s -12db
20
Crossover

• Pick two genes and taking a random number of
  SoundCells and the PlaceGenes that refer to them
  to make one new Chromosome, and leaving the
  remainder to make another.

1.4 s -6db
1.1 s -8db
0.4 s -2db
1.4 s -8db
3.1 s -6db
2.8 s -6db
3.1 s -8db
2.1 s -9db
2.5 s -7db
0.7 s -2db
1.1 s -7db
1.6 s -4db
1.4 s -6db
0.7 s -2db
1.1 s -7db
1.6 s -4db
3.1 s -6db
2.8 s -6db
0.4 s -2db
2.1 s -9db
2.5 s -7db
0.7 s -2db
1.1 s -8db
0.4 s -2db
21
Example of evolution

• score 0.407776, Eve Silverstone, 15 cells, 110
  gene
• score 0.062087, Akira Shakesphere, 3 cells, 13
  genes
• score 0.448460, Vincent Shakesphere, 19 cells,
  208 genes
• score 0.291499, Jack Terentino, 5 cells, 76
  genes
• score 0.287326, Alicia Joyce, 7 cells, 127
  genes
• score 0.458743, Akira Palestrina, 7 cells, 86
  genes
• score 0.395988, Samuel Silverstone, 24 cells,
  158 genes
• score 0.349573, Molly Silverstone, 12 cells, 82
  genes
• score 0.369854, Ikroop Dickens, 14 cells, 97
  genes
• score 0.438446, Leonardo Shakesphere, 19 cells,
  144 genes
• Best in generation 3 is 0.458743
• score 0.458743, Akira Palestrina, 7 cells, 86
  genes
• score 0.314851, Quentin Shakesphere, 8 cells,
  61 genes
• score 0.295788 , Ella Joyce, 6 cells, 41 genes
• score 0.476386, Jane Palestrina, 8 cells, 53
  genes
• score 0.133919, Ludwig Van Shakesphere, 3
  cells, 62 genes

• score 0.110832, Adam Joaquin, 9 cells, 22 genes
• score 0.220700, Jill Hayashi, 2 cells, 42 genes
• score 0.012515, Paul Berio, 16 cells, 3 genes
• score 0.150242, Nimrod Palestrina, 8 cells, 46
  genes
• score 0.303568, Rebecca Silverstone, 15 cells,
  111 genes
• score 0.252139, Alicia Ligeti, 9 cells, 95
  genes
• score 0.384353, John Shakesphere, 15 cells, 93
  genes
• score 0.245086, Kyoko Joyce, 2 cells, 107 genes
• score 0.073234, Rebecca Palestrina, 2 cells, 40
  genes
• score 0.363259, Leonardo Joyce, 13 cells, 105
  genes
• Best in generation 1 is 0.384353
• score 0.384353, John Shakesphere, 15 cells, 93
  genes
• score 0.328594, Samuel Terentino, 2 cells, 125
  genes
• score 0.295409, William Shakesphere, 15 cells,
  94 genes
• score 0.407776, Eve Silverstone, 15 cells, 110
  genes
• score 0.344386, Jack Terentino, 5 cells, 62
  genes

22
Sound Examples (early results)
Target
Generation 1, best
Generation 57, best
23
More to do..

• The algorithm is not yet complete
• Logging not implemented
• Evolution parameters need to auto adjusted
• More SoundRoots and Transformations needed.

24
Early Conclusions on GeneSynth

• The GA can produce meaningful results
• If a target is supplied it borrows some of Its
  meaning and uses It to search.
• The GA is useful for sound fitting and variation.
• Because there is no mathematically defined
  optimum for this, there will be many ways to
  search for it.
• Gas often find interesting niches and hot spots
  in the search space (local maxima,) some of
  whichh may be interesting.
• Generating a near perfect model has not been done
  yet
• It will be a while before we can tell this,
  because the algorithm does not run fast enough
  yet.