Voice over IP and Voice Quality Measurement

1
Voice over IP and Voice Quality Measurement
2
Outline of Talk

• Introduction
• VoIP Networks
• What is QoS or Perceived QoS?
• How to Measure/Predict Voice Quality?
• Subjective
• Objective (intrusive and non-intrusive methods)
• QoS Prediction and Control Research in Plymouth

3
Introduction the problem

• Internet Protocol (IP) networks
• On a steep slope of innovation long term
  carriers of all traffic including voice traffic.
• IP is now the universal communications protocol
  because it facilitates convergence of networks
  and the ability to offer multiple services on the
  same networks.
• Not designed to carry real-time traffic, such as
  voice and video, because of their variable
  characteristics (e.g. delay, delay variation and
  packet loss) . These have adverse effects on
  voice quality.

4
Introduction Voice Quality in IP networks

• User perceived quality is the key QoS metric in
  VoIP applications - The end-user of a VoIP
  service expects
• voice quality to be as good as in traditional
  networks, and
• the service to be as reliable.
• This is not the case at present. This makes it
  necessary to be able to predict/measure, and if
  appropriate, control voice quality in order to
  deliver the desired QoS.

5
VoIP Network and Perceived QoS

• Network QoS
• Perceived QoS is measured from mouth to ear,
  i.e. end-to-end and depends on the performance of
  IP network and terminal/gateway.

6
VoIP New Applications
Mobile network
PSTN
IP Network /MPLS
MGW
GW
DSLAM
VoWLAN
IAD
IAD Integrated Access Device DSLAM DSL Access
Multiplexer MGW Media Gateway MPLS
Multi-protocol Label Switching
Enterprise LAN
VoDSL
7
VoIP Protocol Stack
Audio /video
Application layer
Transport layer
IP
Network layer
e.g. Ethernet/SDH
Physical layer
8
What is QoS?

• The ISO standard defines QoS as a concept for
  specifying how good the offered networking
  services are. QoS can be characterised by a
  number of specific parameters.
• For Multimedia Communication System (MCS), QoS
  concept can be extended to User QoS or
  Perceived QoS.
• For VoIP, Perceived QoS user perceived voice
  quality (e.g. MOS)

9
Factors affect voice quality
End-to-end perceived voice quality (MOS)
Sender
Receiver
Jitter buffer
Decoder
De- packetizer
Encoder
Packetizer
Voice source
Voice receiver
coding distortion codec delay
packet loss network delay jitter
codec impairment delay
delay
delay
buffer-delay buffer-loss

• Other impairments echo, sidetone, background
  noise
• Other factors language, gender, FEC, packet
  loss concealment

10
Inter-relationships between the QoS Parameters 1
11
QoS parameters 1
12
Key QoS parameters and how they arise

• Packet Loss
• Network packet loss (as a result of congestion or
  rerouting in the IP network)
• Late arrival loss (dropped at receiver)
• Link failures and system errors.
• End-to-end Delay
• Network delay (transmission and queuing delay)
• Buffer delay
• Codec processing delay
• Packetizing/depacketizing delay
• Jitter (delay variation)
• Caused by queuing delays within the IP network

13
Delay impact on multimedia quality 7
Interactive
Responsive
Timely
Non-critical
5
Packet Loss
Conversational voice and video
Voice/video messaging
Streaming audio/video
Delay
Fax
100 msec
1 sec
10 sec
100 sec
0

• For VoIP applications, delay lt 150 ms,
  imperceptible, delay gt 400 ms, quality
  unacceptable for most users.

14
How to Enhance QoS?

• Application-level QoS mechanisms
• Packet loss compensation (e.g. FEC, loss
  concealment)
• Jitter compensation (e.g. buffer algorithms)
• Adaptive source coding
• Network-level QoS mechanisms
• How to guarantee IP network performance
• Diffserv (Differentiated Services)
• Intserv (Integrated Services)

15
How to Measure Voice Quality?

• Why need to measure voice quality?
• For QoS monitoring and/or control purposes to
  ensure that the technical and commercial
  requirements (e.g. SLA) are met.
• How to measure voice quality?
• Subjective methods (e.g. MOS)
• Objective methods (e.g. PESQ or E-model)

16
Subjective or objective measurement

• Subjective Voice Quality Measurement
• Subjective listening tests by a group of people
• Provides a benchmark for objective test methods
• Expensive and time-consuming
• Objective Speech Quality Measurement
• Repeatable, automatic, and predicts subjective
  score
• Suitable for online quality measurement/monitoring
  
• Can be used for intrusive and Non-intrusive
  measurements.

17
Voice quality measurement
18
Voice quality measurement (cont.)
19
Subjective voice quality measurement

• Mean Opinion Score (MOS)
• The most widely used subjective measure of voice
  quality.
• Provides a direct link to voice quality as
  perceived by the end user.
• Gives average opinion of quality based on asking
  people to grade the quality of speech on a
  five-point scale Excellent, Good, Fair, Poor
  and Bad.
• Slow, time-consuming, expensive, not repeatable
  and cannot be used to monitor voice quality
  on-line in a large network.
• Different Categories of MOS Test (ITU P.8002)
• Absolute Category Rating (ACR) only listen to
  the degraded speech signals (most commonly used)
• Degradation Category Rating (DCR) rate annoyance
  or degradation level between the reference and
  degraded signal

20
MOS Test Based on ACR
Category Speech Quality
5 Excellent
4 Good
3 Fair
2 Poor
1 Bad
Absolute Category Rating (ACR)
21
MOS Test based on DCR
Category Degradation level
5 Inaudible
4 Audible but not annoying
3 Slightly annoying
2 Annoying
1 Very annoying
Degradation Category Rating (DCR)
22
Online MOS Test Website

• http//www.tech.plymouth.ac.uk/spmc/people/lfsun/m
  os
• This is our research on subjective tests. The aim
  is to provide a more efficient method to carry
  out subjective tests compared to standard MOS
  test (e.g. ITU P.800).
• Standard MOS measurement requires a stringent
  test requirement (e.g. sound proof room, a large
  number of subjects, test procedures). Thus, it is
  very time consuming, expensive, and difficult to
  organise a test.

23
Objective voice quality measurement

• Automated measure of speech quality using an
  appropriate model.
• Conventional methods, e.g. SNR-based approach,
  are not appropriate as they fail to reveal
  quality as perceived by the end user.
• Emerging methods for voice quality prediction are
  based on models of human auditory perception or
  psychologically-derived computational models.
• Can be intrusive (e.g. ITU P.862, PESQ 3) or
  Non-intrusive (e.g. ITU P.563 4 formerly
  P.SEAM) .

24
Intrusive measurement
Reference signal/speech
PESQ
PESQ quality score (MOS)
System under test
Degraded signal/speech

• PESQ (Perceptual Evaluation of Speech Quality),
  ITU P.862, Feb, 2001
• Intrusive (active) test, listening-only quality
• uses test stimuli, such as speech signal

25
Perceptual Evaluation of Speech Quality

• Transforms the original and degraded speech
  signals into a psychophysical representation that
  approximates human perception.
• Calculates their perceptual distance and maps
  this into an objective MOS score.

26
PESQ (perceptual difference)
reference speech
Loss position
degraded speech
PESQ
PSQM
27
OPTICOM- Opera system
                                      Opera
system "Digital Ear http//www.opticom.de
Perceptual Voice/Audio Quality PESQ/PSQM/PEAQ
28
Non-intrusive measurement

• Non-intrusive (passive) test
• Output-based (speech signal based) or
  parameter-based
• Low accuracy if compared to the intrusive methods
• Adequate for real-time, online monitoring purposes

29
Non-intrusive Speech Quality Prediction
Gateway
IP
T1/E1
Signal-based method
Signal-based method

• Signal-based (output-based) to predict/measure
  voice quality directly from degraded speech
  signal (e.g. from T1/E1).
• Parameter-based to predict/measure voice quality
  directly from IP network impairment parameters
  (e.g. loss, delay, jitter).

30
Signal based (output-based) Method

• Assess/predict speech quality non-intrusively
  from degraded speech signal only
• Need to extract speech features (e.g.
  unnaturalness voice, noises, time clipping)
• Mapping to MOS via quality prediction model
• ITU P.563 May 2004 (single-end, signal-based or
  output-based)

31
Parameter based Method

• Access/predict speech quality from IP network
  impairments (e.g. loss, delay) and codec etc.
• Neural network model, non-linear regression
  model, ITU-T E-model 5
• External or built-in approach (be located
  before/after jitter buffer)

32
E-model (ITU G.107, G.108)

• Computational model can be used to compute the
  Mouth-to-ear transmission quality.
• Overall Transmission Quality Rating given by
  model is referred to as the R factor. R lies in
  the range 0-100 and can be mapped to MOS.
• Designed for network planning, but may be used
  for non-intrusive quality monitoring/measurement.
• Based on the principle that Psychological
  factors on the psychological scale are additive

33
E-model equation

• Ro base R value (noise level)
• Id impairments that are delayed with respect to
  speech (e.g. talker/listener echo and absolute
  delay)
• Is impairments that occur simultaneously with
  speech (e.g. quantization noise, received speech
  level and sidetone level)
• Ie equipment impairment (e.g. codec, packet
  loss, jitter)
• A Advantage factor (e.g. 0 for wireline and 10
  for GSM)

34
Loss model - maps loss to Ie
Curve is CODEC dependant
35
Delay model
R Factor Reduction
End to end delay (ms)
36
E-model (a simplified version)
Id
Delay model
Delay (d)
MOS
R?MOS
Packet loss rate
Loss model
Codec type
Ie
37
E-model (R factor) and MOS
TIA 2000
38
Extended E-model

• Simplified E-model
• consider only effects from codec, packet loss
  (random packet loss) and end-to-end delay.
• Extended E-model 6
• Further consider burst loss effects (e.g. 2-state
  Gilbert model, 3 or 4 states Markov models)
• Further consider recency effects.
• Telchemy (http//www.telchemy.com/)

39
Burst Loss vs. Random Loss
40
Recency Effect 6
41
Extended E Model 6
42
VQmon Embedded Monitoring6
Gateway
Gateway
IP Network
QoS metrics
VQmon Agent embedded into VoIP Gateway
NMS
Telchemy (http//www.telchemy.com/)
43
Voice and Video quality Assessment in Psytechnics

• Psytechnics spin off from BT
• http//www.psytechnics.com
• Intrusive model (e.g. PESQ)
• Non-intrusive model
• psyVoIP (parameter-based)
• E-model
• NiQA (signal-based)
• CCI (Call Clarity Index)/INMD (In-service
  Non-intrusive Measurement Device)

44
QoS Prediction and Control - Research in Plymouth

• Aims and objectives
• To research and develop novel, generic methods
  for objective measurement, prediction and control
  of user-perceived quality.
• To apply the methods to real world problems in
  communications, audio and healthcare.
• Examples
• Non-intrusive voice quality prediction and
  measurement for VoIP
• QoS prediction and control for wireless VoIP
• Multimedia quality prediction (voice, audio and
  video)

45
Signal Processing Multimedia Communications
Group

• Research within the Group is concerned with the
• development of novel, generic signal and
  information
• processing methods and their applications to real
  world
• problems.
• Main application areas
• Multimedia communications quality of service
  prediction and control
• Audio sound synthesis, audio quality assessment
• Biomedicine intelligent biosignal analysis,
  biomedical informatics, decision support.

46
About my PhD project

• To develop novel and efficient method/models for
  non-intrusive quality prediction,
• To apply the models for perceptual optimization
  control( e.g. buffer optimization and adaptive
  sender-bit-rate QoS control)

47
A New Methodology
MOS(PESQ)
Intrusive method
Measured MOSc
E-model
delay
PESQ
Reference speech
Degraded speech
(packet loss, delay, codec )
Non-intrusive method
New model
(regression or ANN models)
Predicted MOSc

• Based on intrusive quality measurement (e.g.
  PESQ) to predict voice quality non-intrusively
  which avoids subjective tests.
• A generic method which can be easily applied to
  audio, image and video.

48
Two Non-intrusive Models

• Artificial neural network models for predicting
  listening and conversational voice quality
• Simplified regression models to predict voice
  quality

49
Three Applications

• Voice quality monitoring/prediction for real
  Internet VoIP traces
• Perceived voice quality driven jitter buffer
  optimization
• Perceived voice quality driven QoS control
  (combined adaptive sender-bit-rate and priority
  marking control)

50
References

1. M. Buckley, End-to-end QoS control in VoIP
   systems, Workshop on QoS and user perceived
   transmission quality in evolving networks, Oct.
   2002.
2. ITU-T Rec. P.800, Methods for subjective
   determination of transmission quality, Aug.1996.
3. ITU-T Rec. P. 862, Perceptual evaluation of
   speech quality (PESQ), an objective method for
   end-to-end speech quality assessment of
   narrow-band telephone networks and speech codecs,
   Feb. 2001
4. ITU-T Rec. P.563, Single-ended method for
   objective speech quality assessment in
   narrow-band telephony applications, May 2004.
5. ITU-T Recommendation G.107, The E-model, a
   computational model for use in transmission
   planning, 2000.
6. A. Clark, Modeling the Effects of Burst Packet
   Loss and Recency on Subjective Voice Quality, 2nd
   IPTel Workshop, 2001, pp.123 127.
7. H. Schink, Characterising end to end quality of
   service in TIPHON systems, IP Networking
   Mediacom Workshop, April 2001.

51
References

• L Sun and E Ifeachor, "New Models for Perceived
  Voice Quality Prediction and their Applications
  in Playout Buffer Optimization for VoIP Networks
  Proceedings of IEEE ICC 2004, Paris, France, June
  2004, pp.1478 - 1483.
• Z Qiao, L Sun, N Heilemann and E Ifeachor "A New
  Method for VoIP Quality of Service Control Based
  on Combined Adaptive Sender Rate and Priority
  Marking Proceedings of IEEE ICC 2004, Paris,
  France, June 2004, pp.1473 - 1477.
• L Sun and E Ifeachor, "New Methods for Voice
  Quality Evaluation for IP Networks"  Proceedings
  of the 18th International Teletraffic Congress
  (ITC18), Berlin, Germany, 31 Aug - 5 Sep 2003,
  pp. 1201 - 1210.
• L Sun and E Ifeachor, "Prediction of Perceived
  Conversational Speech Quality and Effects of
  Playout Buffer Algorithms, Proceedings of IEEE
  ICC 2003, Anchorage, USA, May 2003, pp. 1- 6.
• L Sun and E Ifeachor, "Perceived Speech Quality
  Prediction for Voice over IP-based
  Networks" Proceedings of IEEE ICC 2002, New York,
  USA, April 2002, pp.2573-2577.
• L Sun, G Wade, B Lines and E Ifeachor, "Impact of
  Packet Loss Location on Perceived Speech
  Quality, Proceedings of 2nd IP-Telephony
  Workshop (IPTEL '01), New York, April 2001,
  pp.114-122. 

52
Contact details

• SPMC Group website http//www.tech.plymouth.ac.uk
  /spmc
• Professor Emmanuel Ifeachor, Head of Group,
• E-mailE.Ifeachor_at_plymouth.ac.uk
• Dr. Lingfen Sun
• E-mailL.Sun_at_plymouth.ac.uk
• Homepage
• http//www.tech.plymouth.ac.uk/spmc/people/lfsun
  /

53
Thank you!