Melodic Matching Techniques for Large Music Databases

1
Melodic Matching Techniques for Large Music
Databases

• Alexandra Uitdenbogerd, Justin Zobel
• Department of Computer Science, RMIT University
• ACM Multimedia99

2
Outline

• Introduction
• Three-stage framework
• Experiment results analysis
• Conclusion

3
Introduction

• Propose a three-stage framework for matching
  pieces of music.
• -Melody extraction is used to reduce a MIDI
  
• representation to a sequence of non-chordal
  
• notes.
• -Standardisation is used to rewrite the
• sequence in a standard form.
• -Similarity function is then used to
  numerically
• score each piece in the collection against
  the
• query.

4
Melody extraction

• Combine all channels and keep the highest note
  from all simultaneous note events.
• (all-mono)
• Keep the highest notes from each channel, then
  select the channel with the highest first-order
  predictive entropy, that is, with-by a simple
  measure the most complex sequence of notes.
  (entropy-channel)

5
Melody extraction

• Use heuristics to split each channel into parts,
  then choose the part with the highest entropy.
  (entropy-part)
• Keep only the channel with highest average pitch,
  then keep only the highest notes as before.
  (top-channel)

6
Standardisation

• Pitch contour
• -replacing each note by up, down, or same.
• (Modulo) interval
• -represent the melody as a sequence of changes
  in pitch.
• -changes of more than an octave are reduced by
  twelve.
• Exact interval

7
Similarity measures

• Error is inherent in music retrieval, so exact
  matching is unlikely to be an option.
• Melody extraction is inaccurate.

8
Similarity measures

• Longest common substring
• -according to the length of the longest
• contiguous sequence that is identical to a
• sequence in the query.
• Longest common subsequence
• -no penalty for gaps of any size between the
• matching symbols.
• -has potential for this task because melody
  
• extraction often yields additional
  non-melody
• notes.

9
Similarity measures

• Local alignment

10
Local alignment example
11
Experimental design

• 10466 midi files.
• Query the 28 pieces of music chosen were
  randomly selected from a sub-collection of about
  one hundred pieces that we identified as having
  more than one distinct arrangement in the
  collection.
• All arrangements were designated as the relevant
  pieces for each of the queries.

12
Results
13
Analysis

• Contour is always the worst standardisation
  technique and is almost certainly not usable in
  practice.
• Modulo and exact intervals have similar
  performance to each other, with modulo better for
  long queries and exact interval better for short
  queries.
• Another clear result is that local alignment is
  the best similarity measure.

14
Results
15
Analysis

• All-mono was usually the best in conjunction with
  local alignment.
• Manual queries to a music retrieval system are
  likely to consist of a phrase, and thus will
  usually be less than 10 notes, and therefore good
  performance for short queries is vital.

16
Conclusion

• Combining simple melody extraction (all mono),
  relative pitch intervals, and local alignment
  gave excellent effectiveness for even short
  queries, the top 20 answers contained 12 correct
  matches on average.
• Results clearly answer the key question for music
  databases use of the three-stage framework
  allows effective retrieval of music by theme.

17
Conclusion (myself)

• Local alignment may be not good enough.
• Modulo interval may be useful to reduce the space
  that suffix tree need.
• But modulo interval will make more errors.

18
Thinking

• All mono Modulo interval
• a b b a a c
• a a b a c c
• a _ b a _ c

19
insertion cost 1 deletion 0.5mismatch
0.5 if ( _ v.s. a) 1 if (a
v.s. a)

• a _ b a _ c
• 0 0.5 1 2 3 3.5
  4.5
• a 1 0 0.5
• a 2
• b 3
• a 4
• a 5
• c 6