Omnitele Presentation

1
Analysis of Service Quality in 3G Mobile Networks
Heidi Lagerström Supervisor Professor Heikki
Hämmäinen Instructors M.Sc. Sami Vesala M.Sc.
Katja Koivu
2
Contents

• Introduction to the study
• Background, research problem, research methods
• Quality of Service (QoS) in UMTS Networks
• Measuring service quality
• Defining Key Performance Indicators (KPI)
• Case study

3
Background

• UMTS introduces new real time services to mobile
  networks, such as video telephony.
• These real time services require QoS guarantees
  to function properly.
• For operators to maintain satisfactory service
  quality constant network monitoring is needed.
• Network measurements are based on correctly
  defined KPIs for each service.

Operators possibilities to utilise QoS in
practice have not been widely researched. Key
Performance Indicators have not been defined for
the new services from the end-user perspective.
4
Research problem
How should service quality be measured in 3G
networks and how the QoS mechanisms can be used
to affect the service quality perceived by
subscribers.

• Objectives
• What are the KPIs that measure service quality,
  from end user perspective, in 3G networks for the
  key services (AMR voice, video telephony, video
  streaming, web browsing and e-mail)?
• What are the QoS mechanisms in Release 99 and how
  can they be used to improve service quality?

5
Research methods

• Literature study
• 3GPP, ETSI, ITU specifications
• Several books and publications
• Interviews
• Network equipment vendors Ericsson, Nokia
• Operators Elisa
• Several other radio network experts
• Case study
• Field measurements for two operators in live
  networks

6
Contents

• Introduction to the study
• Background, research problem, research methods
• Quality of Service (QoS) in UMTS Networks
• Measuring service quality
• Defining Key Performance Indicators (KPI)
• Case study

7
Why do we need QoS?

• UMTS networks support services with very
  different performance requirements
• Real-time services require performance guarantees
• Customer acceptance closely tied to service
  quality
• Optimal usage of network resources
• Radio resources scarce
• Cost-effectiveness
• Return of investment
• Service and user differentiation
• Meet different needs of customers (e.g. business
  vs. consumer)
• Support different services (real-time vs. best
  effort)
• Competitive advantage!

8
QoS Traffic Classes
Traffic class Characteristics Example application
Conversational Preserve time relation between information entities of the stream. Conversational pattern (stringent and low delay) Speech Video calls
Streaming Preserve time relation between information entities of the stream. Real-time streaming video
Interactive Request-response pattern. Preserve payload content. Web browsing
Background Destination is not expecting the data within a certain time. Preserve payload content. E-mail File downloading

• Demanding
• Delay
• Jitter

• Demanding
• Bit rate
• Jitter

• Tolerant
• Delay and bit rate can vary
• Integrity

• Easiest
• Delay and bit ratecan vary
• Integrity

9
QoS Profile Attributes
R99 QoS attribute Example value
Residual BER 10 5
SDU error ratio 10 4
Delivery of erraneous SDUs No
Maximum SDU size (octets) 1500
Delivery order No
Transfer delay 100 ms (conversational) 280 ms (streaming)
ARP 1, 2 or 3
Traffic Class Conversational, streaming, interactive, background
THP 1, 2 or 3 (same as ARP)
Maximum allowed bit rate e.g. 64, 128 or 384 kbps
Maximum guaranteed bit rate e.g. 64, 128 or 384 kbps
Depends on operators QoS strategy
Depends on the QoS strategy and UE/RNC
capabilities
10
QoS Differentiation

• Each service gets the treatment it requires
  according to the QoS profile
• Network resources are shared according to the
  service needs
• Network resources can be used more efficiently

11
QoS Mechanisms
PDP context with the requested QoS capabilities
RRM

• Different QoS techniques in different parts
  of the network
• Appropriate QoS must be provided in every
  network so that the user can experience good
  service quality

12
Operators QoS Strategy
User profiles are stored in HLR. Each user can
have several user profiles, which correspond to
different services and are mapped to different
bearers according to the operators strategy.

• Operators can practise user differentiation by
  giving each user set of QoS profiles, which
  he/she is entitled to use
• Operators can practise service differentiation
  by mapping each service to the bearer that meets
  its requirements

• Meet the needs of different customers
• Offer each service the quality it requires
• Optimise network resource usage

13
Contents

• Introduction to the study
• Background, research problem, research methods
• Quality of Service (QoS) in UMTS Networks
• Measuring service quality
• Defining Key Performance Indicators (KPI)
• Case study

14
Measuring network performance
15
Defining the right KPIs

• Different services have different quality
  requirements
• KPIs must be defined separately for each of the
  key services
• KPI categories
• Service Accessibility
• Service Integrity
• Service Retainability
• With inadequate performance indicators and
  monitoring
• Hidden problems in network performance and user
  perceived quality of service
• Poorly defined indicators may show better quality
  than in the reality
• Incorrect formulas and counters
• Unreasonable measurement periods (too much
  averaging etc.)

16
Example Voice Services - CS
Customer demand Indicator Measure
Service accessability Availability Coverage Call setup success rate Call setup delay Ec/No, RSCP Admission control RAB assignment
Service integrity Voice quality Noisy frames (FER), MOS
Service retainability Call drop rate Handover failure No coverage Interference
17
Example Data Services - PS
Customer demand Indicators Measures
Service accessability Availability Coverage Access success rate Service access delay Ec/No, RSCP Admission control Attach, PDP context activation, IP service setup
Service integrity Video quality Audio quality Web page download time E-mail sending time, etc. BLER, FER, throughput, delay, jitter
Service retainability Dropped data connection Connection timeouts Dropped PDP context/attach No coverage etc. Handover failure
18
Contents

• Introduction to the study
• Background, research problem, research methods
• Quality of Service (QoS) in UMTS Networks
• Measuring service quality
• Defining Key Performance Indicators (KPI)
• Case study

19
Measurement plan
Operator 1 Operator 2
Drive test AMR speech X X
Drive test FTP download X X
Video telephony X N/A
Streaming X X
Web page download X X
E-mail X X
Data connection attach, PDP context activation, RTT, FTP DL UL X X
Tools Nemo Outdoor, Optimi x-AppMonitor, Ethereal
20
AMR voice Drive test statistics
Shoud be 100
Good gt -10dB
Good gt -92dBm
Good lt 21dBm

• Reasons for call failure
• Ec/N0 was not at adequate level
• Call setup was unsuccessful (unsuccessful RACH
  procedure)
• Look at L3 signalling

21
CPICH coverage Ec/N0
Operator 1
According to Ec/N0 values both operators have
good coverage. Couple of RED areas, which need to
be further investigated!
Operator 2
If large interference areas are generated, the
problem could be minimised later by adjusting the
antenna direction or height, or by down tilting
the antenna or by slightly tuning the pilot power
levels.
22
Data Connection
Typical RTT in UMTS network is 200ms, which
enables good quality conversational PS services,
such as VoIP.
23
Video streaming
In mobile phone display 60 kbps streaming bit
rate produces good video quality.
24
Web browsing
Sample web page 319 kB
25
Conclusions

• In 3G networks QoS management is required
• Real-time services require QoS guarantees
• Need to support different kinds of services
• With QoS mechanisms operators can use their
  network resources more efficiently and gain
  competitive advantage
• To maintain and improve the network performance
  and user experienced service quality constant
  monitoring and performance follow-up is needed
• Successful network measurements are based on
  correct KPI definitions
• A combination of end-to-end field measurements,
  interface probes, network element counter
  statistics and customer feedback is required
• The measurement results show that there are big
  differences in the performance of operators UMTS
  networks
• Currently UMTS networks are not fully optimised ?
  there is a clear need for optimisation!
• Majority of 3G measuring equipment and terminals
  are still quite immature

26
For more information about Omnitele, please visit
our web site www.omnitele.fi
27
KPI Definitions
28
AMR Speech KPIs
Parameters
Service coverage
Speech quality
Service accessibility
Codec usage
Service access time
Service retainability
Trigger points
Place a call
Alerting message
Speech interchange
Intentional termination of session
T2
T3
T1
T0
Channel request
ALERTING
Start of audio stream
RELEASE
29
Video Telephony KPIs
Parameters
Speech quality
Video quality
Service coverage
Video call setup success ratio
Audio/video synchronisation
Service accessibility
Video call setup time
Service retainability
Service access time
Trigger points
Video call Request
Alerting message
Audio/video output
Intentional termination of session
T2
T3
T0
T1
Audio/video output starts
Audio/video output ends
RELEASE
Channel request
ALERTING / Call accepted
30
Video Streaming KPIs
Parameters
Video quality
Service coverage
Audio quality
Streaming reproduction start failure
Service accessibility
Audio/video synchronisation
Streaming reproduction start delay
Service access time
Streaming reproduction cut-off ratio
Trigger points
Stream Request
Buffering message appears on player
Stream reproduction
Intentional termination of session
T2
T3
T0
T1
RTP payload 1st data packet BUFFERING
Streaming reproduction starts picture
appears PLAY
RTSP TEARDOWN
Video/audio stream ends
RTSP SETUP
31
Web Browsing KPIs
Parameters
Service coverage
Service accessibility
Web page download time
Service access time
Service retainability
Trigger points
Service access
Data transfer
Intentional termination of session
T2
T3
T0
T1
1st HTTP GET
HTTP FIN/ACK Reception of last data
packet ?Display data
1st TCP SYN
32
E-mail KPIs
Parameters
Service coverage
Service accessibility
Sending time
Receiving time
Service access time
Service retainability
Trigger points
Service access
E-mail sending
E-mail download
T2
T4
T3
T0
T1
Last data packet send TCP FIN/ACK
Last data packet received TCP FIN/ACK
IMAP FETCH Body
1st TCP SYN
SMTP 250 ACK (HELO)