Bradley W. Vines

1
Functional Data AnalysisTechniques for
Exploring Temporal Processes in Music
Bradley Vines Good afternoon. I will be
discussing (title here). This talk is intended
as an introduction and an overview of FDA.

• Bradley W. Vines
• McGill University

2
Collaborators
Bradley Vines I have collaborated with the
following researchers to develop applications of
FDA techniques for use in music cognition
researche

• Daniel Levitin (McGill University)
• Carol Krumhansl (Cornell University)
• Jim Ramsay (McGill University)
• Regina Nuzzo (McGill University)
• Stephen McAdams (IRCAM)

3
Talk Outline

• What is Functional Data Analysis?
• Steps of a typical FDA
• Demonstrate some of the major FDA tools
• Smoothing
• Registration
• General Linear Modeling (significance testing)

4
An Example of Functional Data
The idea of tension has a shared meaning across
participants, based upon extra-musical
experiences like tension in physical objects, in
social situations and in the body.

• Solo clarinet performances
• 3 Treatment Groups
• Auditory only
• Visual only
• Auditory Visual
• Continuous Tension Judgments

I will begin with an example of functional data.
This data comes from a study that I presented
earlier this week in which we explored the impact
of seeing a musician perform. I will be using
these data to demonstrate the Functional Data
Analysis tools throughout the talk
5
An example of functional data
The question is How can this data be analyzed?
Correlations are useful for identifying
similarities between data sets as are multiple
regressions, but they reduce all of the
information to a summary statistic. Here we are
also interested in how the relations between the
different groups changes over time. This is
where Functional Data Analysis is well suited.
With functional data analysis software tools and
analysis techniques, it is possible to explore
changes over time and to understand WHEN
important changes or relations are occurring.
6
What is Functional Data Analysis? (Ramsay
Silverman, 1997)
The meaning of the music and its impact depends
upon the relations between events over time and
the way that those relations change.
Bradley Vines Temporal dynamics are an important
aspect of music and they are the focus of much
music cognition research
Examples of time dependent measures in musical
stimuli

• For data drawn from continuous processes
• Growth curves, market value, movement, ERPs
• Model data as functions of time
• Temporal dynamics in music
• (Vines, Nuzzo, Levitin, under review)
• continuous measurements of emotion
• expressive timing profiles
• physiological measurements
• movement tracking
• Software tools available in Matlab and in S-Plus

Including measurements like growth curves

• To understand music, we need to know how changes
  in sound effect a listener and the performer -
• Music necessarily occurs through time as the
  unfolding of related events.

Bradley Vines All of these data involve
processes that evolve over time and therefore
that may be intuitively thought of as functions
of time, which makes FDA techniques a useful and
meaningful way to analyze and explore such
measures.
Because we are interested not only in the current
moment in music but also how events are changing
over time and even how changes are changing over
time (as in expressive timing profiles), it is
meaningful and intuitive to think of continuous
measurements as functions of time and to model
them as such.
It makes intuitive sense to think about this kind
of data in terms of functions of time
I will give the ftp site later in the talk.
Tools for visualizing the data, revealing trends
in variation and for significance testing
7
Modeling data as functions of time

• Bradley Vines
• Using basis-fitting techniques
• Take a number of basic functions and add them
  together in such a way as to create the desired
  function (Add together basic functions)
• Same process as in sound synthesis
• Fourier analysis does just the opposite

• Basis functions
• Element functions that can be added together to
  approximate the data.
• W1F1(t) W2F2(t) W3F3(t)
• A least squares algorithm is used to determine
  the weighting coefficients.

8
Two basis types

• Bradley Vines
• Using basis-fitting techniques
• Take a number of basic functions and add them
  together in such a way as to create the desired
  function (Add together basic functions)
• Same process as in sound synthesis
• Fourier analysis does just the opposite

• Fourier
• B-spline
• Polynomial functions
• Knots

Usually, however, data is messy, and
non-periodic. B-spline bases are useful for data
that do not have a simply periodicity.
I will be concentrating on the use of B-spline
bases to model functional data
9
Visualizing B-spline Bases
10
Visualizing B-spline Bases
11
Steps in a typical FDA

• Representing the data in Matlab Matrices
• Each row a sample point in time
• Each column an observation (participant/performer
  )
• Third dimension for multivariate observations
• As in Schuberts multi-dimensional continuous
  interface
• Valence
• Arousal
• (Schubert, 1999)

12
Steps in a typical FDA

• Modeling the data with functions
• Two major considerations
• Order of the B-spline bases
• The number of basis functions

13
Steps in a typical FDA

• Modeling the data with functions
• Two major considerations
• Order of the B-spline bases
• The number of basis functions
• The order of B-spline bases
• Determines how many derivatives will be smooth.

14
Steps in a typical FDA

• The number of basis functions
• Affects the quality of fit to the data
• The more B-splines, the smaller the error
• Tradeoff
• Modeling data accurately
• Excluding unimportant noise in the data

15
Original Data
Remember to mention that there were 800
samples. There are 800 samples shown here
16
Modeled Data
Correlation .9975
17
Modeled Data
The choice of of B-splines really depends upon
the assumptions that the researcher can make
about the data. If it is known that there is a
single objective event that lead to two peaks,
then it might be ideal to treat those two peaks
with a single curve.
Correlation .97
18
Major FDA Tools
Bradley Vines With the data all prepared and
modeled with functions, it is possible to go on
to use the tools available in FDA.
19
Controlling Unwanted Variability

• Curvature (high frequency noise)
• Smoothing
• Amplitude
• Scaling
• Phase
• Registration

20
Nine Tension Judgments
21
(No Transcript)
22
(No Transcript)
23
Nine Tension Judgments
24
Nine Tension Judgments
25
Nine Tension Judgments
Bradley Vines Note that some of the judgments
were shifted ahead or backwards at the two points.
26
Time Warping
27
General Linear Modeling

• Functional regression
• Functional significance test (F-test)

28
The effect of adding video
29
Functional Linear Model

• Y(t) U(t) B1(t) if video is added

30
Results
The question is When is the difference from
zero significant?
31
Significance Testing

• Analogous components to traditional F-testing
• MSE(t) SSE(t) / df(error)
• -with df(error) N participants - P parameters
• MSR(t) SSY(t) SSE(t)/df(model)
• -with df(model) P parameters - 1
• FRATIO(t) MSR(t)/MSE(t)

32
Significance Testing
An F-value that is itself a function of time.
33
Other FDA techniques that are available

• Analysis of covariance
• Functional correlation analysis
• Canonical correlation analysis
• Principal Components Analysis

34
FUNCTIONAL DATA ANALYSIS TECHNIQUES FOR
EXPLORING TEMPORAL PROCESSES IN MUSIC

• Prof. James Ramsays ftp site
• http//www.psych.mcgill.ca/faculty/ramsay/fda.html
  

bradley.vines_at_mail.mcgill.ca
35
Smoothing

• The smoothing parameter, lambda, controls the
  curvature of a function.
• Trade off between perfect fit to the original
  data and a best linear approximation for the
  data. Penalizes variance

36
Smoothing

• Examples of curves before and after smoothing
  (try to find a good singly participant who is
  nice and dynamic for all of this, or a mean
  curve, I suppose)

37
Principal Components Analysis

• Traditional statistics
• Identifying major modes of variation
• Reducing the number of dimensions in the data
• Determine which variables are related
• Functional analogue
• Reveals major modes of variation
• Can reveal trends in phase and in magnitude

38
Principal Components Analysis

• Monthly temperature data
• (available on the ftp website)
• Weather stations across Canada
• Exploring trends in the data and grouping weather
  stations

39
Monthly Weather Data
Bradley Vines 32 weather stations
40
Eigenvalues, VARIMAX PCA
41
VARIMAX Principal Components
42
VARIMAX Principal Components
43
VARIMAX Principal Components
44
VARIMAX Principal Components
45
Component Scores