Providing testability for ITU Recommendations

1
Providing testability for ITU Recommendations
ITU-T Workshop onNew challenges for
Telecommunication Security Standardizations"
Geneva, 9(pm)-10 February 2009

• Ostap Monkewich,
• OMCI

2
Theme

• What is missing from your Recommendations that
  is needed for testing?

3
Why do we need to test?

• To show the customer that the product meets the
  requirements
• To show that the product is likely to
  interoperate with other vendors
• To demonstrate that Recommendations were
  implemented inside the product
• To improve the quality of ITU-T Recommendations

4
Need high-quality test results

• Repeatable by any competent test laboratory
• Can be used to compare with test results obtained
  for similar products
• Recognized by all and accepted in all
  geographical market regions

5
What kind of testing?

• Conformance Testing
• verifies if the implementation does what the
  Recommendation says it is supposed to do

• Interoperability Testing
• checks if two implementation communicate at
  functionalities level

• Functionalities
• Every pair

6
Frequently Asked Question

• Why not do only interoperability testing?
• Answer We dont know what to do when two
  implementations do not interoperate
• If we change something are we closer to the
  Recommendation or farther from it?
• Non-conforming changes destroy interoperability
  with other vendors
• Serious problems will not be discovered when
  observing functionalities only
• example turning lights on and off in a new house
  is a good interoperability or functionality
  test. The house may burn down at a later time
  because the wiring did not conform to the standard

7
Sources of Interoperability Problems

• Recommendations
• Errors in Recommendations
• Ambiguities in natural language
• Unverified or invalid behaviours described
• Implementers different interpretations of text
• Requirements in text, more than one meaning
• No standardized questionnaire for supplier
• Incompatible Recommendations/implementations
• Different choices of options
• Device incompatibilities
• Different host system configurations

8
In addition to Base Recommendation

• Requirements Clause
• Extracted from text, need no interpretation
• Implementation Conformance Statement (ICS)
  Proforma
• Supplier declares what pieces were implemented
  from Recommendation
• Implementation eXtra Information for Testing
  (IXIT) Proforma
• Identifies non-standardized items, how
  architecture, interfaces are packaged
• Test Suite Structure
• Logically groups test cases
• Test Purposes
• one test purpose/verdict per test case
• Abstract Test Suite
• Set of tests that covers the Recommendation

9
Recommendations to support testing
10
Requirements NGN Draft Rec. Y.2702
Sections 7.2.3 and 8.2.1 NGN User
authentication for service access and network
attachment

• (R-40) Each user/subscriber associated with an
  application service subscription is required to
  be uniquely addressable
• (R-41) It is required that it be possible for the
  end user to access a service simultaneously
  multiple times and/or from multiple devices.
• (R-42) It is required that it be possible to
  support multiple subscription profiles for an
  individual end-user.
• (O-1) It is an option that network access points
  supporting NGN TE and TE-BE support capabilities
  to allow the end user to uniquely identify the
  NGN provider.
• (O-2) It is an option that network access points
  supporting NGN TE and TE-BE support
  capabilities to allow the user to authenticate
  and authorize the NGN provider.

11
Implementation Conformance Statement (ICS)
Proforma
Appendix IV to NGN Draft Rec. Y.2702 on Identity
management (IdM)
12
Examples of Test Purposes inSIP Security Testing

• Ensure that the IUT, after sending the initial
  REGISTER request to the Registrar, ignores
  Registrar OK response by sending a second
  REGISTER request
• Ensure that the IUT re-sends the initial REGISTER
  request on receipt from the Registrar of a 401
  Unauthorized response in which WWW-Authenticate
  header does not contain the nonce field

13
Implementation Conformance Statement (ICS)

• ICS ICS Proforma with answers
• A list of requirements and options claimed to
  have been implemented
• Used for
• Shopping for products to match for
  interoperability
• selecting Test Cases from test suite for test
  execution

14
Test Suite Structure
15
Test Case Structure
16
Conclusion

• First, Conformance testing then Interoperability
  testing
• High-quality Recommendations are needed
• Base Recommendations require additional parts to
  produce widely accepted test results
• Methodology Recommendations for testing are
  available attached charts

17
Additional Slides
All you need to develop high-quality Recommendatio
ns and Test Specifications
18
Conformance and Interoperability Testing
Methodology Recommendations

• X.290 - General Concepts
• X.291 - Abstract Test Suite Specification
• X.292 - (Superseded by Z.160/170 series)
• X.293 - Test Realization
• X.294 - Requirements on Test Laboratories and
  Clients
• X.295 - Protocol Profile Test Specification
• X.296 - Implementation Conformance Statements
• Supplement 1 to X.290 series - Generic approach
  to interoperability testing
• Supplement 2 to X.290 series - Interoperability
  testing framework and methodology

19
Testing and Test Control Notation (TTCN-3)

• Z.161 (Z.140 Revised) Core Language
• Z.162 (Z.141 Revised) Tabular Format
• Z.163 (Z.142 Revised) Graphical Format
• Z.164 (Z.143 Revised) Operational semantics
• Z.165 (Z.144 Revised) Runtime interface
• Z.166 (Z.145 Revised) Control interface
• Z.167 (New) Using ASN.1 with TTCN-3
• Z.168 (New) The IDL to TTCN-3 mapping
• Z.169 (New) Using XML Schema with TTCN-3
• Z.170 (New) TTCN-3 documentation tags

20
Specification and Description Language

• Z.100 Overview of SDL-2008
• Z.101 Basic SDL-2008
• Z.102 Comprehensive SDL-2008
• Z.103 Shorthand notation and annotation in
  SDL-2008
• Z.104 Data and action language in SDL-2008
• Z.105 SDL-2008 combined with ASN.1 modules
• Z.106 Common Interchange Format for SDL-2008

21
Abstract Syntax Notation One (ASN.1)

• X.680 (07/02)  Information technology - Abstract
  Syntax Notation One (ASN.1) Specification of
  basic notation
• X.680-X.693 (07/02) Information Technology -
  Abstract Syntax Notation One (ASN.1) ASN.1
  encoding rules
• X.681 (07/02) Information technology - Abstract
  Syntax Notation One (ASN.1) Information object
  specification
• X.682 (07/02) Information technology - Abstract
  Syntax Notation One (ASN.1) Constraint
  specification
• X.683 (07/02) Information technology - Abstract
  Syntax Notation One (ASN.1) Parameterization of
  ASN.1 specifications

22
Abstract Syntax Notation One (ASN.1)

• X.690 (07/02)Information technology - ASN.1
  encoding rules Specification of Basic Encoding
  Rules (BER), Canonical Encoding Rules (CER) and
  Distinguished Encoding Rules (DER)
• X.691 (07/02) Information technology - ASN.1
  encoding rules Specification of Packed Encoding
  Rules (PER)
• X.692 (03/02)  Information technology - ASN.1
  encoding rules Specification of Encoding Control
  Notation (ECN)  
• X.693 (12/01) Information technology - ASN.1
  encoding rules XML encoding rules

23
Supporting Recommendations

• Z.120 Message Sequence Chart (MSC)
• Z.150 User Requirements Notation - Language
  Requirements and Framework
• Z.151 User Requirements Notation - Language
  Definition

• Z.110 Application of Formal Description
  Techniques
• Z.450 Quality aspects of protocol-related
  Recommendations