Parametric Methods

1
Parametric Methods

• for
• Power Spectrum Estimation

2
Advantages of Nonparametric Method

• Relatively simple
• Well understood
• Easy to compute via the FFT algorithm.

3
Limitations of Nonpara. Method

• Require long data records in order to yield the
  necessary frequency resolution
• Suffer from spectral leakage effects due to
  windowing, and weak signals is masked by the
  spectral leakage.

4
Reasons for the Limitation

• The inherent assumption
• the estimate is zero for
  .
• This assumption severely limits the frequency
  resolution and the quality of PSD estimate.

5
Reasons for the Limitation

• The inherent assumption in the periodogram
  estimate is that the data is periodic with period
  N.
• Neither one of these inherent assumptions is
  realistic.

6
Parametric Methods

• PSD estimation methods do not require these
  assumptions.
• Extrapolate the values of the autocorrelation for
  lags .

7
Possibility for Extrapolation

• If we have some a priori information on how the
  data was generated, a model for the signal
  generation may be constructed with number of
  parameters that can be estimated from the
  observed data.
• From the model and the estimated parameters, we
  can compute the power density spectrum implied by
  the model.

8
Effects for Extrapolation

• The modeling approach eliminates
• the need for window functions
• the assumption that the autocorrelation sequence
  is zero for .

9
Effects for Extrapolation

• Parametric (model-based) power spectrum
  estimation methods provide better frequency
  resolution than the FFT-based, nonparametric
  methods.
• Avoid the problem of leakage.

10
Effects for Extrapolation

• Parametric method is especially true in
  applications where short data records are
  available due to time-variant or transient
  phenomena.

11
Basics

• of
• Parametric Methods

12
Modeling

• The data sequence is regarded as the
  output of a linear system characterized by a
  rational system function of the form

13
Modeling

• The corresponding difference equation is
• where is the input sequence to the
  system and the observed data, ,
  represents the output sequence.

14
Modeling

• Here is the power density spectrum of
  the input sequence and is the frequency
  response of the model.

15
Modeling

• Assume that the input sequence is a
  zero-mean, white noise sequence
• The power density spectrum of the observed data

16
Model-Based PSD Estimation

• Given the data sequence
  , its model is
• The key to PSD estimation is to estimate the
  parameters and of the model.

17
Model-Based PSD Estimation

• The procedure consists of two steps
• 1. Estimate the parameters and of
  the model.
• 2. Compute the PSD

18
Decomposition Theorem

• Based on the decomposition theorem due to Wold,
  the ARMA, MA and AR model are equal and can be
  converted to each other.
• Of these three linear models, the AR model is by
  far the most widely used.

19
Model-Based PSD Estimation

• The reasons for AR model widely used are
• First, the AR model is suitable for representing
  spectra with narrow peaks.
• Second, the AR model results in very simple
  linear equations for the AR parameters

20
Model-Based PSD Estimation

• The MA model, requires many more coefficients to
  represent a narrow spectrum.
• Consequently, it is rarely used alone as a model
  for spectrum estimation.

21
Model-Based PSD Estimation

• The ARMA model provides a more efficient
  representation from the viewpoint of the number
  of model parameters needed to represent the
  spectrum of a random process.
• It is not easy to get the parameters of ARMA
  model.

22
Relationships

• Between the Autocorrelation and the Model
  Parameters

23
AR (p) Process

• The AR parameters are obtained from the
  solution of the Yule-Walker or normal equations.

24
AR (p) Process

• Yule-Walker equations can be efficiently solved
  by use of the Levinson-Durbin algorithm.

25
AR (p) Process

• All the system parameters in the AR (p) model are
  easily determined from the autocorrelation
  sequence for .
• It may be used to extend the autocorrelation
  sequence for , once are
  determined.

26
Yule-Walker Method

• for
• the AR Model Parameters

27
Yule-Walker Method

• In the Yule-Walker method, estimate the
  autocorrelation from the data and use the
  estimates to solve for the AR model parameters.

28
Yule-Walker Method

• The Levinson-Durbin algorithm with substituted
  for , yields the AR
  parameters.

29
Yule-Walker Method

• where are estimates of AR
  parameters obtained from the Levinson-Durbin
  recursions, and the estimated minimum mean-square
  value for the pth-order predictor

30
Characteristics

• Spectral peaks are proportional to the square of
  the power of the sinusoidal signal.
• The area under the peak in the power density
  spectrum is linearly proportional to power of the
  sinusoid.
• The behavior holds for all AR model-based
  estimation methods.

31
Burg Method

• for
• the AR Model Parameters

32
Burg Method

• To estimate the AR parameters
• order-recursive
• lattice method
• the minimization of the forward and backward
  errors in linear predictors

33
Burg Method

• Given the data
  , the forward linear prediction estimates of
  order m,
• where , are
  the prediction coefficients.

34
Burg Method

• The corresponding forward and backward errors
• , defined as
• The least-squares error is

35
Burg Method

• This error is to be minimized by selecting the
  prediction coefficients, subject to the
  constraint that they satisfy the Levinson-Durbin
  recursion given by

36
Burg Method

• where is the mth
  reflection coefficient in the lattice filter
  realization of the predictor.

37
Burg Method

• Perform the minimization of with
  respect to the complex-valued reflection
  coefficient , we obtain the result

38
Burg Method

• where is an estimate of the
  total squared error .

39
Summarize

• The Burg algorithm computes the reflection
  coefficients in the equivalent lattice structure.
• The Levinson-Durbin algorithm is used to obtain
  the AR model parameters.

40
Advantages of Burg Method

• For estimating the parameters of the AR model
  are
• (1) it results in high frequency resolution
• (2) it yields a stable AR model
• (3) it is computationally efficient (due to the
  Levinson-Durbin algorithm).

41
Disadvantages of Burg Method

• Exhibits spectral line splitting at high SNR.
• For high-order models, the method also introduces
  spurious peaks.
• For sinusoidal signals in noise, the Burg method
  exhibits a sensitivity to the initial phase of a
  sinusoid, especially in short data records.

42
Modifications of Burg Method

• Basically, the modifications involve the
  introduction of a weighting (window) sequence on
  the squared forward and backward errors.

43
Modifications of Burg Method

• Results in the reflection coefficient estimates

44
Modifications of Burg Method

• Use of a Hamming window, a quadratic or parabolic
  window, the energy-weighting method, and the
  data-adaptive energy weighting.

45
Modifications of Burg Method

• These windowing and energy-weighting methods have
  proved effective in reducing the occurrence of
  line splitting and spurious peaks, and are also
  effective in reducing frequency bias.

46
Maximum Entropy Spectrum Estimation

• MESE

47
MESE

• MESE is a criterion as a basis for the AR model
  in parametric spectrum estimation.

48
MESE

• Burg postulated that the extrapolation be made on
  the basis of maximizing uncertainty (entropy) or
  randomness, in the sense that the spectrum
  of the process is the flattest of all
  spectra that have the given autocorrelation
  values

49
MESE

• When the process is Gaussian, the entropy per
  sample is proportional to the integral

50
MESE

• Burg found that the maximum of this integral,
  subject to the constraints,
• is the AR (p) process for which the given
  autocorrelation sequence

51
MESE

• This solution provides an additional
  justification for the use of the AR model in
  power spectrum estimation.

52
MESE

• In view of Burgs basic work in maximum entropy
  spectral estimation, the Burg power spectrum
  estimation procedure is often called the maximum
  entropy method (MEM).

53
MESE

• The maximum entropy spectrum is identical to the
  AR-model spectrum only when the exact
  autocorrelation is known.

54
MESE

• The general formulation for the maximum entropy
  spectrum based on estimates of the
  autocorrelation sequence results in a set of
  nonlinear equations.

55
Unconstrained Least-Square Method

• for
• the AR Model Parameters

56
Burg method

• The Burg method for determining the parameters of
  the AR model is basically a least-squares lattice
  algorithm, with the added constraint that the
  predictor coefficients satisfy the
  Levinson-Durbin recursion.

57
Burg method

• As a result of this constraint, an increase in
  the order of the AR model requires only a single
  parameter optimization at each stage.

58
Unconstrained LS Algorithm

• The sum of squares of the forward and backward
  linear prediction error

59
Unconstrained LS Algorithm

• The unconstrained minimization of with
  respect to the prediction coefficients yields the
  set of linear equations

60
Unconstrained LS Algorithm

• By definition, the autocorrelation
• The resulting residual least-squares error is

61
Unconstrained LS Algorithm

• The unconstrained least-squares power spectrum is

62
Unconstrained LS Algorithm

• The form of the unconstrained least-squares
  method just described has also been called the
  unwindowed data least-squares method.
• It has been proposed for spectrum estimation in
  several papers, including Burg, Nuttal, and
  Ulrych and Clayton.

63
Unconstrained LS Algorithm

• The performance have been found to be superior to
  the Burg method, in the sense that it does not
  exhibit the same sensitivity to such problems as
  line splitting, frequency bias, and spurious
  peaks.

64
Selection of AR Model Order
65
Selection of AR Model Order

• As a general rule, select a model with too low an
  order, a highly smoothed spectrum is obtained.
• If p is selected too high, we run the risk of
  introducing spurious low-level peaks in the
  spectrum.

66
Selection of AR Model Order

• One indication of the performance of the AR model
  is the mean-square value of the residual error.
• The mean-square value of the residual error
  decreases as the order of the AR model is
  increased.

67
Selection of AR Model Order

• Monitor the rate of decreases for error square
  and decide to terminate the process when the rate
  of decrease becomes relatively slow.
• The approach may be imprecise and ill-defined.

68
Selection of AR Model Order

• Two of the better known criteria for selecting
  the model order have been proposed by Akaike.

69
Selection of AR Model Order

• First, called the final prediction error (FPE)
  criterion, the order is selected to minimize the
  performance index
• where is the estimated variance of the
  linear prediction error.
• This performance index is based on minimizing the
  MSE for a one-step predictor.

70
Selection of AR Model Order

• The second criterion proposed by Akaike, called
  the Akaike information criterion (AIC), is based
  on selecting the order that minimizes

71
Selection of AR Model Order

• The term and decreases, as
  the order of the AR model is increased.
• 2p/N increases with an increase in p.
• Hence, a minimum value is obtained for some p.

72
Selection of AR Model Order

• Another criterion proposed by Rissanen, is based
  on selecting the order that minimizes the
  description length, (MDL), where MDL is defined
  as

73
Selection of AR Model Order

• A fourth criterion has been proposed by Parzen is
  called the criterion autoregressive transfer
  (CAT) function
• Where
• The order p is selected to minimize CAT (p).

74
Selection of AR Model Order

• In applying the preceding criteria, the mean
  should be removed from the data.
• Since depends on the type of spectrum
  estimate we obtain, the model order is also a
  function of the criterion.

75
Selection of AR Model Order

• The experimental results indicate that the
  model-order selection criteria do not yield
  definitive results.

76
Selection of AR Model Order

• It is found that FPE (p) criterion tends to
  underestimate the model order.
• Kashyap showed that the ACI criterion is
  statistically inconsistent as .

77
Selection of AR Model Order

• The MDL information criterion proposed by
  Rissanen is statistically consistent.
• Other experimental results indicate that for
  small data length, the order of the AR model
  should be selected to be in the range N/3 to N/2
  for good results.

78
Selection of AR Model Order

• It is apparent that in the absence of any prior
  information regarding the physical process that
  resulted in the data, one should try different
  model orders and different criteria and,
  ultimately, interpret the different results.

79
Experimental Results
80
Experimental Results

• Present some experimental results on the
  performance of AR power spectrum estimates that
  were obtained with artificially generated data.

81
Experimental Results

• Compare the spectral estimation methods on the
  basis of their frequency resolution, bias and
  robustness in the presence of additive noise.

82
Experimental Results

• The data consist of either one or two sinusoids
  and additive Gaussian noise. The two sinousoids
  are space apart.
• Clearly, the underlying process is ARMA(4,4).
• The results that are shown employ an AR(p) model
  for this data.

83
Experimental Results

• For high signal-to-noise ratios (SNR), we expect
  the AR(4) to be adequate.

SNR20dB
84
Experimental Results
SNR20dB
85
Experimental Results

• The Yule-Walker method no longer resolves the
  peaks.
• Some bias is also evident in the Burg method.
• Burg and least-squares methods are clearly
  superior for short data records.

86
Experimental Results

• The effect of filter order on the Burg method
• It exhibits spurious peaks

87
Experimental Results
Line Splitting
88
Experimental Results

• The resolution properties of the Burg method for
• and at low SNR
  (3dB).

89
Experimental Results

• Since the additive noise process is ARMA, a
  higher-order AR model is required to provide a
  good approximation at low SNR.
• The frequency resolution improves as the order is
  increased.

90
Experimental Results
The FPE for the Burg method For this SNR, the
optimum valued is according to
the FPE criterion.