Diapositiva 1

1
Italian QoS Monitoring Network impact on User
Protection
Regional Development Forum for Europe
(RDF-EUR) Broadband for Sustainable Development
20-22 April 2015, Bucharest Eng. Luca
Rea Project leader for the Italian
Resolution n.244/08/CSP deployment
2
Outline

• Project regulatory and technical aspects
• Technical aspects
• End point location (Internet eXchange Point )
• Protocol type
• QoS parameters and file size
• Next QoS measurement metrics
• Network architecture for ISP measurements
• End users measurements
• Ne.Me.Sys results
• Ne.Me.Sys speed test

3
Italian NRA (AGCOM) Resolution n. 244/08/CSP User
Protection - Main aspects -

• Aims
• ISP measurements Number of measures to help
  consumers in understanding more on broadband
  service they are going to buy. Broadband results
  obtained by ISP measurements will be used by
  each Provider instead of up to x Mbit/s
  (advertised values)
• End users measurements By means of the software
  Ne.Me.Sys (Network Measurement System) every end
  user can evaluate its own fixed line Quality of
  Service. Results could be compared with ISPs
  promised ones
• Contents
• Measurement models
• Services supplied by operators
• QoS control and consumers rights
• Supervision by an independent body (Fondazione
  Ugo Bordoni)
• Sanctions

4
Resolution n. 244/08/CSP QoS fixed-line broadband
connection - Operators duties -

• Operators must publish the named Carta dei
  servizi document containing the main principles
  that offered services must comply with. Minimum
  requirements in terms of Quality of Service are
  established according to ETSI ES 202 765 4
• Operators must transmit relevant data to AGCOM
  (Italian NRA), publish on their websites
  conditions of their services and advertise
  correct data as far as data transmission speed is
  concerned
• Provide predefined contents for fixed-line
  contracts
• communication indicating where information on
  aims and results concerning QoS can be found
• communication concerning the technical aspects of
  the QoS provided, which must be available to the
  customer before signing the contract
• technology used for the access (e.g., Dial-up
  POTS, Dial-up ISDN, ADSL, ADSL2, ADSL2, fiber
  optic) and the minimum characteristics that
  customers CPE must have in order to guarantee
  the full and correct performance of Internet
  access contract

5
Resolution n. 244/08/CSP QoS fixed-line broadband
connection Consumer rights -

• Operators bind themselves to observe values
  indicated within the contracts, so if values
  measured are worse with respect to published
  ones, he can file a detailed complaint to the
  undertaking involved
• He can claim QoS level restored
• If proper levels of QoS are not restored within
  30 days from the complaint, the customer has the
  right to recede without penalties from the
  contract.
• Each customer can formally certify the quality of
  his own fixed-line broadband access by means of
  the free software Ne.Me.Sys
• www.misurainternet.it

6
Measure Architecture
Measurement Server
Operators IP Network
Twisted pair
Operators IP Network
IXP
Central System
Operators IP Network
Twisted pair
Clients Location
Both ISP Measurements and End-user measurements
employ the same Network Measurements System,
based on a PC software agent
Twisted pair
7
IXP (Internet eXchange Point) IXP architecture
Providers

• One or more network switch, connecting the
  participating ISPs to each other.
• It is a physical infrastructure through which
  ISPs exchange Internet traffic between their
  networks (autonomous systems).
• The primary purpose of an IXP is to allow
  networks to interconnect directly, via the
  exchange, rather than through one or more 3rd
  party networks

FUB
..
1
n
2
Server FTP 1 (ISP Measurements)
LAN Peering
1 Gbit/s
Router
Up to 100 Mbit/s toward FTP1 Up to 900 Mbit/s
toward FTP2
Server FTP 2 (End users )
LAN
IXP
Switch
Through a measurement server located in IXP, it
is guaranteed that each client measures only
performances related to its own ISP Network. This
approach has been chosen since it allows for
comparable and reproducible results
8
QoS parameters involved ETSI ES 202 765-4

• List of QoS parameters defined by ETSI Standard
  ES 202 765-4
• Data transmission speed
• Data transmission rate achieved in a fixed time
  period (10 seconds) separately for downloading
  and uploading specified test files between a
  remote server and a user's computer
• Packet Delay (one way transmission time)
• half the time in milliseconds, needed for an ICMP
  Echo Request/Reply Packet (Ping Command) to a
  valid IP address
• Packet loss ratio
• the ratio between the number of not replied Ping
  commands to the total number of sent Ping
  commands
• Unsuccessful data transmission ratio
• the ratio of unsuccessful data transmissions to
  the total number of data transmission attempts in
  a specified time period

9
QoS parameters involved ETSI ES 202 765-4

• Regarding data transmission speed, the following
  four values relevant to data transmission speed
  are computed for both upload and download
  direction
• 95th quantile in kbps
• maximum bit rate during the analysis period of 10
  seconds
• average bit rate during the analysis period of 10
  seconds
• The following two values are computed relative to
  delay
• The mean value in milliseconds
• The standard deviation

EG 202 057-4 (up to 2014) ES 202 765-4 (starting at 2015)
Data speed rate protocol used FTP HTTP
Throughput 20 times FTP (GET/PUT) FTP File size is equal to 10 time the headline speed under test. The results of each test is File length divided time to download We take into account number of Byte downloaded with a time slot of 10 seconds
Delay 10 pings with a 10s distance from each other (ICMP packet length is 32 byte) 10 pings with a 10s distance from each other (ICMP packet length is 32 byte)
10
ISP measurements network architecture Network
deployed on the whole Italian Country
Bolzano
Measurement Server Neutral Acess Point (NAP)
BL
TN
UD
Trieste
PN
VR
NO
Venezia
PD
MN
Milano
PR
Measurement Client Ministry peripheral premises
Torino
FO
Bologna
Genova
LU
Main peripheral site (Ispettorato)
Ancona
AR
Firenze
Secondary peripheral site (Dipendenza)
PG
PI
Avezzano
SI
TR
GR
PE
Sulmona
RI
VT
FUB central site central system
CB
LT
Roma
FR
BN
CE
Bari
AV
PZ
SA
TA
Napoli
Cagliari
CS
CZ
Palermo
R.Calabria
MS
TP
CT
11
ISP measurements (site characteristics) www.misura
internet.it/stats.php

• Upload line attenuation is considered because it
  is less affected by frequency depending.
• The Italian average distance between customers
  and exchange is about 1,2 km and the
  corresponding average upload Line Attenuation is
  11 db
• 11 db in upload line attenuation 1,2 km between
  users and Central Office (Exchange) regards 50
  of xDSL consumers in Italy

To have a comparison between ISP performances
also Line Attenuation must be the same
12
ISP www.misurainternet.it/stats.php
Attenuation equal to 11 dB in Upload
OLO room
13
ISP mesurements Example of Probe Room
14
ISP mesurements www.misurainternet.it
5th percentile, computed on the whole national
samples, represents the value that each ISPs can
advertise. In a short time the minimum will be
95th quantile . All other values are available by
local analysis at www.misurainternet.it
15
End User measurements Ne.Me.Sys Ne.Me.Sys
(Network Measurement System)

• It is an open source software that allows to any
  consumer to certify himself his own fixed-line
  QoS
• Free download from www.misurainternet.it
• Registration is needed
• Cross-platform
• It runs under all major OSes Windows, Mac OSX,
  and Linux
• Before starting measures, Ne.Me.Sys checks CPE
  right environment
• Before of a single test users LAN is analyzed to
  assure that nobody share bandwidth it is going to
  measure. RAM occupation and CPU busy are also
  controlled.
• Measures endure 24 hours
• To obtain a significant number of samples that
  allow to well characterize the fixed line under
  test, Ne.Me.Sys executes at least 50 tests per
  hour. 50 tests (20 download, 20 upload and 10
  ping) constitute a measure.

16
Headline Speed VS Actual Speed Up to now, more
than 31000 users has completed fixed-line
characterization
Certified distribution
The average Actual Speed is ever half with
respect to headline one. In many cases, it is due
to the distance between users location and the
central office
17
Ne.Me.Sys Statistics
Registered users User with at least one measure Emitted certificates In advance certificates
203.938 58.581 39.757 21.733
Emitted certificates
18
Ne.Me.Sys Statistics
Registered users
19
Resolution n. 244/08/CSP QoS fixed-line broadband
connection MisuraInternet Speed Test-

• To give to users an easier instrument to estimate
  QoS, a short version of Ne.Me.Sys has been
  realized. Such a version is named MisuraInternet
  Speed Test, and it allows users to have an
  immediately result.
• Obviously results do not have an official value
  from receding contract point of view, anyway
  crowdsourcing approach allows an ex ante
  information to evaluate the single ISP
  performance according to coverage
• Sample collected are one the pillar of national
  statistic with the aim of monitoring the growth
  and evolution of broadband

20
MisuraInternet Speed Test Up to now, more than
399160 users has used MisuraInternet Speed Test
21
Conclusions The Italian Experience

• Next Challenges
• Extend the monitoring/evaluating system to mobile
  connections
• Create a detailed National Actual Speed Map to
  give right information to users
• Exactly identify coverage areas and suggest
  solution for broadband connectivity according to
  2020 European Digital Agenda Target (i.e. LTE,
  LSA Carrier Grade, FTTx)
• Stimulate competition between ISPs

This map is obtained by Telecom Italian Wholesale
portal. It is an approximations because of each
local municipality perimeter has been colored
according to the ISP presence and technology
22
Italian QoS Monitoring Network impact on User
Protecion
Next Meeting Villa Griffone, Bologna
Italy The head office of Guglielmo Marconi
Foundation is in Villa Griffone, the place where
the Bolognese inventor began to develop the
system of wireless telegraphy which then spread
throughout the world. Villa Griffone is located
in Pontecchio Marconi, about 15 km from Bologna,
Italy. The garden and the rooms of the villa are
home to the Marconi Museum which is dedicated to
the origins and development of radio
communications. Villa Griffone is a National
Monument Thanks for your attention
Luca Rea lrea_at_fub.it