Title: ESTIMATION OF LYAPUNOV SPECTRA FROM A TIME SERIES
1ESTIMATION OF LYAPUNOV SPECTRAFROM A TIME SERIES
S. Srinivasan, S. Prasad, S. Patil, G. Lazarou
and J. Picone Intelligent Electronic
Systems Center for Advanced Vehicular
Systems Mississippi State University URL
http//www.cavs.msstate.edu/hse/ies/publications/c
onferences/ieee_secon/2006/lyapunov_exponents/
2- Analysis of chaotic signals
- Reconstruction a phase-space from a scalar
observable - Lyapunov exponents as a tool to analyze chaos
- Lyapunov spectra of chaotic and non-chaotic time
series - Optimize parameters of Lyapunov spectra
estimation
3- A deterministic signal or system every event is
the result of preceding events and actions hence
predictable completely - Stochastic noise signal that is not
deterministic, i.e., inherently unpredictable - A chaotic signal or system sensitive to initial
conditions (Butterfly Effect) - Chaos says predictability holds only in
principle, hence chaotic signals are also called
deterministic noise. - Dimension of a system number degrees of freedom
possessed by the system - Deterministic Chaos or Stochastic Noise?
- Both have continuous power spectra (and not
easily distinguishable) - Noise is infinite-dimensional.
- Chaotic signals are finite dimensional, but
dimension no longer associated with number of
independent frequencies, but a statistical
feature related to both temporal evolution and
geometric aspect (self-similar structure of the
attractor)
4- Power Spectrum of a Lorentz Signal
- Power spectra of chaotic signals are continuous,
though the system is finite dimensional. For
example, the power spectrum of a 3-dimensional
chaotic Lorentz signal is shown below. Stochastic
systems have similar spectra even though they are
infinite dimensional.
5- Attractors for Dynamical Systems
- System Attractor Trajectories approach a limit
with increasing time, irrespective of the initial
conditions within a region - Basin of Attraction Set of initial conditions
converging to a particular attractor - Attractors Non-chaotic (point, limit cycle or
torus),or chaotic (strange attactors) - Example point and limit cycle attractors of a
logistic map (a discrete nonlinear chaotic map)
6- Strange Attractors attractors whose shapes are
neither points nor limit cycles. They typically
have a fractal structure (i.e., they have
dimensions that are not integers but fractional) - Example a Lorentz system with parameters
7- Exploit geometrical (self-similar structure)
aspects of an attractor or the temporal evolution
for system characterization - Geometry of a Strange Attractor
- Most strange attractors show a similar structure
at various scales, i.e., parts are similar to the
whole. - Fractal dimensions can be used to quantify this
self-similarity. - e.g., Hausdorff, correlation dimensions.
- Temporal Aspect of Chaos
- Characteristic exponents or Lyapunov Exponents
(LEs) - captures rate of divergence (or
convergence) of nearby trajectories - Also Correlation Entropy captures similar
information. - Any characterization presupposes that phase-space
is available. - What if only one scalar time series measurement
of the system (and not its actual phase space) is
available?
8- Reconstructed Phase Space (RPS) Embedding
- Embedding A mapping from an one-dimensional
signal to an m-dimensional signal - Takens Theorem
- Can reconstruct a phase space equivalent to the
original phase space by embedding with m 2d1
(d is the system dimension) - Embedding Dimension a theoretically sufficient
bound in practice, embedding with a smaller
dimension is adequate. - Equivalence
- means the system invariants characterizing the
attractor are the same - does not mean reconstructed phase space (RPS) is
exactly the same as original phase space - RPS Construction techniques include differential
embedding, integral embedding, time delay
embedding, and SVD embedding
9- Reconstructed Phase Space (RPS) Time Delay
Embedding
- Uses delayed copies of the original time series
as components of RPS to form a matrix - m embedding dimension, delay parameter
- Each row of the matrix is a point in the RPS
10- Reconstructed Phase Space (RPS)
Time Delay Embedding of a Lorentz time series
11- Reconstructed Phase Space (RPS) Time Delay
Embedding
- Setting very small delay value leads to highly
correlated vector elements, concentrated around
the diagonal in embedding space. Structure
perpendicular to the diagonal not captured
adequately. - Setting very large delay value leads elements of
the vector to behave as if they are independent.
Evolutionary information in the system is lost. - Quantitative tools for fixing delay plots of
autocorrelation and auto-mutual information are
useful guides. - Advantages easy to compute the attractor
structure is not distorted since no extra
processing is done on it. - Disadvantages choice of delay parameter value is
not obvious leads to poor RPS in presence of
noise.
12- Reconstructed Phase Space (RPS) SVD-based
Embedding
- Works in two stages
- Delay embed, with one sample delay, to a
dimension larger than twice the actual embedding
dimension - Reduce this matrix using SVD to finally have
number of columns equal to embedding dimension. - (SVD-based matrix reduction is done by
projecting each row onto only the first few
eigenvectors and then reconstructing it to a
lower-dimensional space) - SVD window size dimension of time delayed
embedded matrix over which SVD operates - Advantages No delay parameter value to be set
more robust to noise due to SVD stage - Disadvantages Noise reducing property of SVD may
also distort the attractor properties
13Reconstructed Phase Space (RPS) Reconstruction
Attractor reconstruction using SVD embedding
(for a Lorentz system)
14- Quantifies separation in time between
trajectories, assuming rate of growth (or decay)
is exponential in time, as - where J is the Jacobian matrix at point p.
- Captures sensitivity to initial conditions.
- Analyzes separation in time of two trajectories
with close initial points - where is the systems evolution function.
15- Lyapunov Exponents Some Properties
- m-dimensional system has m LEs
- LE is a measure averaged over the whole attractor
- Sum of first k LEs rate of growth of
k-dimensional Euclidean volume element - Bounded attractor Sum of all LEs equals zero
(conservative) or negative (dissipative) - Zero exponents indicate periodic attractor (limit
cycle) or a flow - Negative exponents pull points in the basin of
attraction to the attractor - Positive exponents indicate divergence signature
for existence of chaos
16- Lyapunov Exponents Computation
- Embed time series to form RPS matrix. Rows
represent points in phase space - Take first point as center
- Form neighborhood matrix, each row obtained by
subtracting a neighbor from the centre - Find evolution of each neighbor and form the
evolved neighborhood matrix by subtracting each
evolved neighbor from the evolved centre - Compute trajectory matrix at the center by
multiplying pseudo-inverse of neighborhood matrix
with evolved neighborhood matrix - Advance center to a new point and go to step 3,
averaging the trajectory matrix in each iteration - The LEs are given by the average of the
eigenvalues from each R matrix. Direct averaging
has numerical issues, hence an iterative QR
decomposition method (treppen-iteration) is used.
17- Lyapunov Exponents Computation Flowchart
18- Three systems tested two chaotic (Lorentz and
Rossler) and one non-chaotic (sine signal) - Two test conditions clean and noisy (10 dB white
noise) - Lorentz system
- Parameters
- Expected LEs (1.37, 0, -22.37)
- Rossler system
- Parameters a 0.15, b 0.2, c 10
- Expected LEs (0.090, 0.00, -9.8)
- Sine Signal
- Parameters Freq1Hz, Samp freq16Hz, Amp1
- Expected LEs (0.00, 0.00, -1.85)
19- Experiments performed to optimize parameters of
estimation algorithm - 30,000 points were generated for each series in
both the conditions - 5,000 iterations (or the number of evolution
steps) were used for averaging using QR
treppen-iteration - Variation of LEs with SVD window size and number
of nearest neighbors - Varied number of neighbors with SVD window size
15 for clean data 50 for noisy data - Varied SVD window size with number of neighbors
15 for clean data 50 for noisy data
20 Lyapunov Exponents (LEs) for a Lorentz System
- For clean data Positive and zero exponents are
almost constant at the expected values - For noisy data Positive and zero exponents
converge to the expected value for SVD window
size about 50 and number of neighbors also about
50 - Negative LE estimation not reliable
21Lyapunov Exponents (LEs) for a Rossler System
- For clean data Positive and zero exponents are
almost constant at the expected values - For noisy data Positive and zero exponents
converge to the expected value for SVD window
size about 60 and number of neighbors also about
50 - Negative LE estimation not reliable
22Lyapunov Exponents (LEs) for Sine Signal
- For clean data Positive and zero exponents are
almost constant at the expected values - For clean data Positive and zero exponents
converge to the expected value for SVD window
size about 40 and number of neighbors also about
30 - Negative LE estimation not reliable
23- LEs are useful in quantifying how chaotic a
system is. - SVD embedding helps reconstructing phase spaces
in noisy conditions. - Parameters of the LE computation algorithm are
optimized experimentally to get reliable
estimates. - Both the positive and zero LEs are estimated
near the actual values using optimized
parameters. - Negative LE estimation is unreliable (but this is
of little concern in chaotic systems). - The code for LE estimation is publicly available.
- Our future work will be to apply Lyapunov
exponents to model nonlinearities in speech for
better automatic speech recognition.
24 25- J. P. Eckmann and D. Ruelle, Ergodic Theory of
Chaos and Strange Attractors, Reviews of Modern
Physics, vol. 57, pp. 617-656, July 1985. - M. Banbrook, Nonlinear analysis of speech from
a synthesis perspective, PhD Thesis, The
University of Edinburgh, Edinburgh, UK, 1996. - E. Ott, T. Sauer, J. A. Yorke, Coping with
chaos, Wiley Interscience, New York, New York,
USA, 1994. - M. Sano and Y. Sawada, Measurement of the
Lyapunov Spectrum from a Chaotic Time Series,
Physical Review Letters, vol. 55, pp. 1082-1085,
1985. - G. Ushaw, Sigma delta modulation of a chaotic
signal, PhD Thesis, The University of Edinburgh,
Edinburgh, UK, 1996.