ETSI MTS Presentation

1
An Approach to Testing Distributed Systems
Ina Schieferdecker, Mang Li, Stephan Pietsch,
Theofanis Vassiliou-Gioles, Andreas Hoffmann
GMD FOKUS, Berlin
2
Contents

• Introduction
• Testing of Distributed Systems
• Test Case Development
• Test Validation TSsim
• Test Implementation TCgate
• Test Execution TTman
• An Example TINA Retailer Reference Point
• Conclusions

3
Introduction

• Distributed systems are complex, heteregeneous,
  and concurrent systems
• Validated properties of the system specification
  might not be fulfilled by a system
  implementation
• CTMF is a well-established and widely used test
  method for conformance and interoperability
  testing of OSI systems (including TTCN as a test
  notation)
• Goal
• Application of TTCN to distributed systems
• Automated CORBA based distributed test execution

4
Local and Distributed Testing
5
The Overall Approach
6
Test Case Development

• Declarations part
• TTCN types
• Translation from ODL types is supported by
  mapping rules.
• Mapping rules can be used for manual or
  semi-automatic test development.
• Mapping rules support generic encoding/decoding.
• Others
• Operations, parameters, constants, variables,
  timers.

7
Test Case Development (contd)

• Dynamic part, constraints part
• Currently manually developed, based on
  interpretation of textual description.
• Alternative semi-automatic, based on SDL
  specificationI.e. with TTCNLink, AutoLink or
  TGV.

8
ODL-to-TTCN Mapping

• Mapping for an ODL interface declaration
• Interface inheritance is flattened.
• An interface is comparable to a SAP.
• Requested and provided interfaces are distinct ?
  two PCO types
• PCOtoSUTServer
• PCOtoSUTClient
• Multiple PCOs derived from the same interface
  declaration can be used in a test (multi-server
  or multi-client).

9
ODL-to-TTCN Mapping (contd)

• Mapping for a typical ODL operation declaration
• Operations are mapped to ASPs.
• Three ASP types are derived from an operation
  declaration
• A request-related ASP type.
• A reply-related ASP type.
• A raise-related (exception-related) ASP type.

10
Example Test for Service Access Session
Retailer Reference Point
11
Example IDL Specification
module TINARetRetailerInitial interface
i_RetailerInitial void requestNamedAccess ( in
TINACommonTypest_UserId userId, in
TINACommonTypest_UserProperties
userProperties, out Object namedAccessIR, out
TINAAccessCommonTypest_AccessSessionSecretId
asSecretId, out TINAAccessCommonTypest_AccessSes
sionId asId )
12
Example Simplified TTCN Test Case
13
Example Test Case for Service Access Session
14
The Test Tool Chain
15
Motivation for a TTCN Simulator

• Testing the test problem
• Validation of test suites is essential
• Complexity of Concurrent TTCN test cases
• Analyser tools provided by TTCN editors provide
  only syntax and static semantic checks
• Manual dynamic semantic check by experts?
  inefficient and error-prone

TSsim
16
Test Suite Simulator TSsim

• Basis for test suite validation
• Generic test suite simulator
• Supports Concurrent TTCN
• Interactive step by step execution of test cases
  (PDUs from the IUTs, Timeouts)
• Verification of dynamic aspects (e.g. test
  coordination procedures, timer settings)
• Validation of logical correctness, deadlocks

TSsim
17
Test Suite Simulator GUI
TSsim
18
Validation Process with TSsim
TSsim
19
Error Detection with TSsim

• Timer Errors? deadlock, if no receive event is
  evaluated to true
• Test Component Coordination Errors? timeout, if
  a test component waits for a CM
• Indentation Errors? unexpected dynamic behavior
• Constraint Errors? non-matching of correct PDUs
  or CMs

TSsim
20
The Test Tool Chain
TCgate
21
Motivation for a TTCN/CORBA GW

• Acceptance of TTCN
• Support of parallel test components via
  Concurrent TTCN
• Test access to CORBA-based systems.
• Uniform interface to TTCN-based test systems
• Separated from TTCN-based test systems ?TTCN
  compiler-independent.
• Problem TTCN OSI paradigm vs. CORBA OO paradigm

TCgate
22
TTCN/CORBA GW TCgate

• Architecture of the gateway
• Supports testing of interfaces provided by the
  SUT.
• Supports emulation of interfaces required by the
  SUT.
• Is a generic request-level bridge.
• Uses dynamic interfaces of ORB ?
  type-independent.
• Is a CORBA-compliant application.
• Uses public interfaces of ORB ? easily portable.

TCgate
23
Test Interface for CORBA Based Systems
TCgate
24
TCGate Main

• interface GatewayMain
• GWStatus GWInit()
• void GWDestroy()
• GWStatus GWConfigure(...)
• GWStatus GWAddConnection(...)
• GWStatus GWSend(in string pcoID, in GWMessage
  msg)
• GWMsgFlag GWReceive(out string pcoID, out
  GWMessage msg)
• GWStatus here_is_a_CM(in GWMessage msg)

TCgate
25
Test Suite Adaption to TCgate

• Is based on GCI interface.
• Has generic encoding/decoding for all IDL types.
• Is reusable for other TINA and CORBA
  applications.

TCgate
26
Example Test for Service Access Session
27
Development of the Example Tester
TCgate
28
Example Test Execution
1. Test Request from ETS to IUT
2. IUT Parameter Request to ETS
3. ETS Parameter Reply to IUT
4. IUT Test Reply to ETS
5. Test Result Verdict Assignment gt PASS /
FAIL
TCgate
29
The Test Tool Chain
TTman
30
Test Sychronizations Protocol 1

• Defines Procedures for
• Test Setup,
• Setting Up Configuration
• Distributing Parameters, etc.
• Test Execution
• Start, Stop and Cancel
• Routing of Coordination Messages
• Test Reporting

TTman
31
TSP1 Architecture
TTman
32
The Test Manager TTman

• Portable Library for Implementing
• The System Supervisor
• The Front End
• State Driven TCL/TK GUI at the CMI

TTman
33
Test Execution

• Generation of MSC logs
• Uses a library supplied by Telelogic.
• Display selected test events in an MSC editor.
• Benefits visualization, problem location.

34
Test Execution (contd)
35
Motivation for Test Scripting

• Distributed Testing requires detailed
  Specification of Test Case Execution

• ? Test Session Specification Language TSSL
• TSSL Provides
• Dynamic Test Case Selection based on
• Test Case Verdicts
• Group Verdicts
• Calculated Verdicts

36
TSSL Features

• Declarations part dynamic part
• INTEGER, REAL, BOOLEAN, STRING standard
  datatypes
• Session global and test group local variables
• GROUP is an object with methods exec and eval and
  with attributes ref, list and verdict
• Group hierarchies are supported
• loop (while) and decision constructs (if)
• Functions count_tc, count_verdict,
  count_verdict_rel
• Exception handling

37
TSSL Example (static part)

• basic_capabilities_session
• import "SIG_USER"
• GROUP outgoing, incoming, clearing
• TESTCASE out_setup, out_callproc, out_connack,
  in_aal, in_bhl
• INTEGER tries
• out_setup.ref "SIG_USER/GENERAL/OUTGOING/U0_U1
  _001"
• out_callproc.ref "SIG_USER/GENERAL/OUTGOING/U1
  _U3_002"
• out_connack.ref "SIG_USER/GENERAL/OUTGOING/U3_
  U10_003"
• .

38
TSSL Example (dynamic part)

• outgoing.exec
• if(exec(out_setup) exec(out_callproc))
• exec(out_connack) verdict PASS
• else verdict FAIL break
• clearing.eval
• if(count_verdict_rel(clearing, FAIL) gt 0.25)
  verdict FAIL break
• if(exec(outgoing) PASS)
• tries 1
• while(!incoming.verdict tries lt 10)
• ...

39
Use of TSSL
TTman
40
Conclusions

• Testing of CORBA based systems
• ETS Generator for transformation of abstract test
  suites into executable ones, validation with
  TSsim
• TCgate and TTman for automated distributed test
  execution
• Testing within the development methodology for
  distributed systems
• Future work
• Enhanced coordination features for parallel test
  components
• Investigation of real-time aspects of the
  TTCN/CORBA Gateway
• TTCN extensions to better address the TINA object
  model