SIP Performance Metrics

1
SIP Performance Metrics

• 66th IETF Montreal
• Daryl Malas

2
Motivation

• With widespread implementation of SIP the
  following problems have surfaced
• No standard performance terms
• No standard performance metrics
• No standard methods
• No standards between vendors and operators
• Current reliance on PSTN metrics
• Operator and vendor problem!!!

3
Industry Acknowledgment Relevance

• SIPPING WG develops standard to benchmark
  end-to-end SIP application performance
• BMWG develops standard to benchmark SIP
  networking device performance
• SPEC to develop industry-available test code for
  SIP benchmarking in accordance with IETFs BMWG
  and SIPPING standards.
• ITU-T (Study Group 12) focused on detailed
  failure/success scenarios and end-to-end
  benchmark guidance.

4
SIPPING vs. BMWG

• -----Original Message-----From Poretsky, Scott
  Sent Thursday, June 22, 2006 800 PMTo
  'Malas, Daryl' acmorton_at_att.com
  gonzalo.camarillo_at_ericsson.com
  mary.barnes_at_nortel.comCc vkg_at_lucent.com
  Poretsky, ScottSubject RE (BMWG/SippingWG) SIP
  performance metricsYes Daryl.  I absolutely
  agree.  The item posted to BMWG focuses on single
  DUT benchmarking of SIP performance.  Your work
  in SIPPING is focused on end-to-end application
  benchmarking.  It would be great (and I would
  even say a requirement) that the Terminologies
  for these two work items remain consistent with
  each other.   

5
Proposed Metrics (Solution)

• Create standard metrics with consistent terms and
  methods of calculations
• The metrics
• Session Request Delay
• Measures time from the initiation request to an
  acknowledgment
• Session Completion Delay
• Measures time from a completion request to an
  acknowledgment
• Average Hops per INVITE
• Measures the average hops an INVITE takes to
  traverse the network
• Average Hops per Session
• Measures the average hops of origination
  messages, associated with a session, taken to
  traverse the network
• Session Duration Time
• Measures time from the initiation message to the
  completion message
• Session Establishment Rate
• Ratio of number of initiation requests to
  established sessions

6
Metrics Contd

• Session Defects
• Identifies the number of persistent failures
  defects in dialog processing
• Ineffective Session Attempts
• Identify attempts described as ineffective due to
  failure or congestion
• Session Disconnect Failures
• Identify issues causing sessions to fail after
  they were established
• Session Completion Rate
• Measures the number of sessions completed without
  failure due to timeouts
• Session Success Rate
• Identifies the number of successful calls by
  removing the percentage of Ineffective Session
  Attempts and Session Disconnect Failures

7
Feedback and Next Steps

• Interest in making a working group item?
• Should end-to-end media performance metrics be
  included?
• Should terminology be extracted and included only
  in BMWG terminology draft?
• Additional metrics?
• Reduce number of metrics?
• SIP MIB incorporation?