DSP-CIS Chapter-6: Filter Implementation

1
DSP-CISChapter-6 Filter Implementation

• Marc Moonen
• Dept. E.E./ESAT, KU Leuven
• marc.moonen_at_esat.kuleuven.be
• www.esat.kuleuven.be/scd/

2
Filter Design/Realization

• Step-1 define filter specs
• (pass-band, stop-band, optimization
  criterion,)
• Step-2 derive optimal transfer function
• FIR or IIR design
  
• Step-3 filter realization (block scheme/flow
  graph)
• direct form realizations, lattice
  realizations,
• Step-4 filter implementation (software/hardware)
  
• finite word-length issues,
• question implemented filter designed
  filter ?
• You cant always get what you want
  -Jagger/Richards (?)

Chapter-4
Chapter-5
Chapter-6
3
Chapter-6 Filter Implementation

• Introduction
• Filter implementation finite wordlength problem
• Coefficient Quantization
• Arithmetic Operations
• Scaling
• Quantization noise
• Limit Cycles
• Orthogonal Filters

4
Introduction

• Filter implementation finite word-length
  problem
• So far have assumed that signals/coefficients/arit
  hmetic operations are represented/performed with
  infinite precision.
• In practice, numbers can be represented only to a
  finite precision, and hence signals/coefficients/a
  rithmetic operations are subject to quantization
  (truncation/rounding/...) errors.
• Investigate impact of
• - quantization of filter coefficients
• - quantization ( overflow) in arithmetic
  operations
• PS Chapter partly adopted from
  
• A course in digital signal
  processing, B.Porat, Wiley 1997

5
Introduction

• Filter implementation finite word-length
  problem
• We consider fixed-point filter implementations,
  with a short word-length.
• In hardware design, with tight speed
  requirements, finite word-length problem is a
  relevant problem.
• In signal processors with a sufficiently long
  word-length, e.g. with 16 bits (4 decimal
  digits) or 24 bits (7 decimal digits) precision,
  or with floating-point representations and
  arithmetic, finite word-length issues are less
  relevant.

6
Introduction

• QWhy bother
• about many different realizations
• for one and the same
  filter?

Back to Chapter-5
7
Introduction Example

• IIR Elliptic Lowpass filter designed using
• ELLIP function.
• All frequency values are in Hz.
• Fs 48000 Sampling Frequency
• L 8 Order
• Fpass 9600 Passband Frequency
• Apass 60 Passband Ripple (dB)
• Astop 160 Stopband Attenuation (dB)

Transfer function
Poles zeros
8
Introduction Example
Filter outputs
Direct form realization _at_ infinite precision
Lattice-ladder realization _at_ infinite precision
Difference
9
Introduction Example
Filter outputs
Direct form realization _at_ infinite precision
Direct form realization _at_ 10-bit precision
Difference
10
Introduction Example
Filter outputs
Direct form realization _at_ infinite precision
Direct form realization _at_ 8-bit precision
Difference
11
Introduction Example
Filter outputs
Direct form realization _at_ infinite precision
Lattice-ladder realization _at_ 8-bit precision
Difference
Better select a good realization !
12
Coefficient Quantization

• The coefficient quantization problem
• Filter design in Matlab (e.g.) provides filter
  coefficients to 15 decimal digits (such that
  filter meets specifications)
• For implementation, have to quantize coefficients
  to the word-length used for the implementation.
• As a result, implemented filter may fail to meet
  specifications ??

13
Coefficient Quantization

• Coefficient quantization effect on pole locations
  
• example 2nd-order system (e.g. for
  cascade/direct form realization)
• triangle of stability denominator
  polynomial is stable (i.e.
• roots inside unit circle) iff coefficients
  lie inside triangle
• Proof Apply Schur-Cohn stability test (see
  Chapter-5).

14
Coefficient Quantization

• example (continued)
• with 5 bits per coefficient, all possible
  quantized pole positions are...
• Low density of quantized pole locations at
  z1, z-1, hence problem for
  narrow-band LP and HP filters (see Chapter-4).

15
Coefficient Quantization

• example (continued)
• possible remedy coupled realization
• poles are where
  are realized/quantized
• hence quantized pole locations are (5
  bits)

uk
-
yk
coefficient precision pole precision
16
Coefficient Quantization

• Coefficient quantization effect on pole locations
  
• example higher-order systems (first-order
  analysis)
• ? tightly spaced poles (e.g. for narrow band
  filters) imply
• high sensitivity of pole locations to
  coefficient quantization
• ? hence preference for low-order systems (e.g.
  in parallel/cascade)

17
Coefficient Quantization

• PS Coefficient quantization in lossless lattice
  realizations
• In lossless lattice, all coefficients are sines
  and cosines, hence all
• values between 1 and 1, i.e. dynamic
  range and coefficient
• quantization error well under control.

o original transfer function transfer
function after 8-bit truncation of
lossless lattice filter coefficients -
transfer function after 8-bit truncation
of direct-form coefficients (bis)

18
Arithmetic Operations

• Finite word-length effects in arithmetic
  operations
• In linear filters, have to consider additions
  multiplications
• Addition
• if, two B-bit numbers are added, the result
  has (B1) bits.
• Multiplication
• if a B1-bit number is multiplied by a B2-bit
  number, the
• result has (B1B2-1) bits.
  
  
• For instance, two B-bit numbers yield a
  (2B-1)-bit product
• Typically (especially so in an IIR (feedback)
  filter), the result of an addition/multiplication
  has to be represented again as a B-bit number
  (e.g. BB). Hence have to remove most
  significant bits and/or least significant bits

19
Arithmetic Operations

• Option-1 Most significant bits (MSBs)
• If the result is known to be (almost) always
  upper bounded such that 1 or more MSBs are
  (almost) always redundant, these MSBs can be
  dropped without loss of accuracy (mostly).
  Dropping MSBs then leads to better usage of
  available word-length, hence better SNR.
  
  This implies we
  have to monitor potential overflow (dropping
  MSBs that are non-redundant), and possibly
  introduce additional scaling to avoid overflow.
• Option-2 Least significant bits (LSBs)
• Rounding/truncation/ to B bits introduces
  quantization noise.
• The effect of quantization noise is usually
  analyzed in a statistical manner.
• Quantization, however, is a deterministic
  non-linear effect, which may give rise to limit
  cycle oscillations.

20
Quantization Noise

• Quantization mechanisms
• Rounding Truncation
  Magnitude Truncation
• mean0
  mean(-0.5)LSB (biased!) mean0
• variance(1/12)LSB2 variance(1/12)LSB
  2 variance(1/6)LSB2

output
input
probability
error
21
Quantization Noise

• Statistical analysis is based on the following
  assumptions
• - each quantization error is random, with
  uniform probability distribution function (see
  previous slide)
• - quantization errors at the output of a given
  multiplier are uncorrelated/independent (white
  noise assumption)
• - quantization errors at the outputs of
  different multipliers are uncorrelated/independent
  (independent sources assumption)
• One noise source is inserted
• for each multiplier(/adder)
• Since the filter is a linear filter the output
• noise generated by each noise source can be
• computed and added to the output signal

22
Limit Cycles

• Statistical analysis is simple/convenient, but
  quantization is truly a non-linear effect, and
  should be analyzed as a deterministic process.
• Though very difficult, such analysis may reveal
  odd behavior
• Example yk -0.625.yk-1uk
• 4-bit rounding
  arithmetic
• input uk0,
  y03/8
• output yk 3/8,
  -1/4, 1/8, -1/8, 1/8, -1/8, 1/8, -1/8, 1/8,..
• Oscillations in the absence of input (uk0) are
  called
• zero-input limit cycle oscillations

23
Limit Cycles

• Example yk -0.625.yk-1uk
• 4-bit truncation
  (instead of rounding)
• input uk0,
  y03/8
• output yk 3/8,
  -1/4, 1/8, 0, 0, 0,.. (no limit cycle!)
• Example yk 0.625.yk-1uk
• 4-bit rounding
• input uk0,
  y03/8
• output yk 3/8,
  1/4, 1/8, 1/8, 1/8, 1/8,..
• Example yk 0.625.yk-1uk
• 4-bit truncation
• input uk0,
  y0-3/8
• output yk -3/8,
  -1/4, -1/8, -1/8, -1/8, -1/8,..
• Conclusion weird, weird, weird, !

24
Limit Cycles

• Limit cycle oscillations are clearly unwanted
  (e.g. may be audible in speech/audio
  applications)
• Limit cycle oscillations can only appear if the
  filter has feedback. Hence FIR filters cannot
  have limit cycle oscillations.
• Mathematical analysis is very difficult ?
• Truncation often helps to avoid limit cycles
  (e.g. magnitude truncation, where absolute value
  of quantizer output is never larger than absolute
  value of quantizer input (passive quantizer)).
• Some filter realizations can be made limit cycle
  free, e.g. coupled realization, orthogonal
  filters (see below).

25
Orthogonal Filters
Skip this slide

• Orthogonal filter state-space realization with
• orthogonal realization matrix
• PS lattice filters and lattice-part of
  lattice-ladder .
  
• Strictly speaking, these are not state-space
  realizations (cfr. supra), but orthogonal R is
  realized as a product of matrices, each of which
  is again orthogonal, such that useful properties
  of orthogonal states-space realizations indeed
  carry over (see p. 38)
• Why should we be so fond of orthogonal filters ?

26
Orthogonal Filters
Skip this slide

• Scaling
• - in a state-space realization, have to monitor
  overflow in internal states
• - transfer functions from input to internal
  states (scaling functions) are
• defined by impulse response sequence (see p.
  22)
• - for an orthogonal filter (R.RI), it is
  proven (next slide) that
• which implies that all scaling functions
  (rows of Delta) have L2-norm1
• (diagonal elements of )
• conclusion orthogonal filters are scaled in
  L2-sense

27
Orthogonal Filters
Skip this slide

• Proof
• First
• Then

28
Orthogonal Filters
Skip this slide

• Quantization noise
• - in a state-space realization, quantization
  noise in internal states may be
• represented by equivalent noise source
  (vector) Exk
• - transfer functions from noise sources to
  output (noise transfer
• functions) are defined by impulse response
  sequence
• - for an orthogonal filter (R.RI), it is
  proven that (try it)
• which implies that all noise gains are 1
  (all noise transfer functions
• have L2-norm 1 diagonal elements of
  ).
• - This is then proven to correspond to minimum
  possible output noise
• variance for given transfer function, under
  L2 scaled conditions (i.e.
• when states are scaled such that scaling
  functions have L2-norm1) !! (details omitted)

29
Orthogonal Filters

• Limit cycle oscillations
• if magnitude truncation is used (passive
  quantization),
• orthogonal filters are guaranteed to be free of
  limit cycles
• (details omitted)
• intuition quantization consumes energy/power,
  orthogonal filter does not generate power to feed
  limit cycle.

30
Orthogonal Filters

• Orthogonal filters
• L2-scaled, minimum output noise,
  limit cycle oscillation free filters
  !
• It can be shown that these statements also hold
  for
• lossless lattice realizations of a general
  IIR filter
• lattice-part of a general IIR filter in
  lattice-ladder realization