SoFI

1
SoFI Streaming Music using Song Form
Intelligence

• By Jonathan Doherty
• Supervisors Dr. Kevin Curran, Prof. Paul Mc
  Kevitt
• School of Computing and Intelligent Systems
• Faculty of Engineering
• University of Ulster, Magee

2
Objectives of SoFI

• Pattern matching within songs
• Determine replacements based on pattern matches
• Client based error concealment of packet loss
• Possibly model song semantics

3
Previous Research

• Liang et al. (2001) sender based by adjusting
  packet size
• Ngo et al. (1999) sender and receiver based
  using a random order for sending packets
• Windows Media Player, Apples Quick Time
• Real Audio, Shoutcast

4
Song Form and Structure

• Western tonal music
• Verse, chorus, verse, chorus
• Music pattern matching
• SEMEX (Lemstrom and Perttu 2000)
• CAMUS 3D (Miranda 2001)
• Ghias et al. (1995) Query by humming

5
Project Proposal

• To match lost/late packets with previously
  received similar portions of songs
• Cognitive approach to error concealment
• MPEG-7 descriptors for tagging sections of songs

6
SoFI General Architecture
Pattern Matching and Replacement
Audio Player
Packet Loss Identifier
BUFFER
7
Comparison of Streaming Approaches
Systems Year Music Information Retrieval / Pattern Matching Streaming Error Concealment Streaming Error Concealment Streaming Error Concealment Streaming Error Concealment
Systems Year Music Information Retrieval / Pattern Matching Sender Based Receiver Based On-edge Server Packet Re-send
Streaming Approaches Ngo et al. 1999
Streaming Approaches Liang et al. 2003
Streaming Applications Media Player 2004
Streaming Applications Real Player "
Streaming Applications Shoutcast "
Streaming Applications Quick Time "
This project SoFI ? ? ?
8
Software Analysis

• Programming Tools
• Java
• C
• VB .Net
• Streaming Applications
• Shoutcast
• Real Player
• Quick Time

9
Project Plan
2004 2005 2005 2005 2005 2006 2006 2006 2006 2007 2007 2007
Research Activities Oct-Dec Jan-Mar Apr-Jun Jul-Sep Oct-Dec Jan-Mar Apr-Jun Jul-Sep Oct-Dec Jan-Mar Apr-Jun Jul-Sep
Literature survey
Literature Review write-up
Write-up and submission of paper to conferences
Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection Analysis and Tool Selection
Learning MPEG-7 implementation
Java, C, VB .Net evaluation
Analysis/design of pattern matching algorithm
Selection of other reusable components (e.g. streaming audio applications)
System design (Object Orientated Based)
Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation Unit implementation
Construction and testing of core streaming modules
Pattern matching construction
Construction of audio modules
Unforeseen modules
Integration and testing
Performance analysis
Write up Ph.D. thesis
Improving system
Modifying thesis
10
Conclusion

• The objectives of SoFI meet challenging problems
  in streaming audio
• Maintain high quality audio streaming on bursty
  and bandwidth constrained networks
• Extend the QoS (Quality of Service) protocols to
  improve users listening experience
• Success of SoFI will be tested against accuracy
  in pattern matching and amount of packets
  replaced in relation to total number of packets
  lost