Towards ServiceOriented Testing of Web Services

1
Towards Service-Oriented Testing of Web Services

• Hong Zhu
• Department of Computing, Oxford Brookes
  University
• Oxford OX33 1HX, UK
• Yufeng Zhang
• Dept of Computer Sci, National Univ. of Defense
  Tech., Changsha, China, Email yuffonzhang_at_163.com

2
Overview

• Motivation
• The impact of WS on software testing
• The requirements on supports to testing WS
• Proposed framework
• Prototype implementation
• Case studies
• Conclusion

3
Characteristics of Web Services

• The components (services) of WS applications
• Autonomous control their own resources and their
  own behaviours
• Active execution not triggered by message, and
• Persistent computational entities that last long
  time
• Interactions between services
• Social ability discover and establish
  interaction at runtime
• Collaboration as opposite to control, may refuse
  service, follow a complicated protocol, etc.

4
WS technique stack

• Basic standards
• WSDL service description and publication
• UDDI for service registration and retrieval
• SOAP for service invocation and delivery
• More advanced standards for collaborations
  between service providers and requesters.
• BPEL4WS business process and workflow models.
• OWL-S ontology for the description of semantics
  of services

5
Testing developers own services

• Service has similarity to test software
  components
• Many existing work on software component testing
  can be applied or adapted
• Requires special considerations
• The stateless feature of HTTP protocol
• XML encoding of the data passing between services
  as in SOAP standard
• Confirmation to the published descriptions
• WSDL for the syntax of the services
• workflow specification in BPEL4WS
• semantic specification in e.g. OWL-S.

6
Testing developers own services (continue)

• Dealing with requesters abnormal behaviours
• The requesters are autonomous, their behaviours
  may be unexpected
• Need to ensure that the service handles abnormal
  behaviours properly
• Dealing with unexpected usages/loads
• As all web-based applications, load balance is
  essential.
• The usage of a WS may not be available during the
  design and implementation of the system.
• Dealing with incomplete systems
• A service may have to rely on other services,
  thus hard to separate the testing of the own
  services from the integration testing, especially
  when it involves complicated workflows.
• In the worst case, when dynamically bound to the
  other services

7
Testing of others services in composition

• Some similarity to component integration
• However, the differences are dominant
• Problems in the application of existing
  integration testing techniques
• Lack of software artifacts
• Lack of control over test executions
• Lack of means of observation on system behaviour

8
Lack of software artifacts

• The problem
• No design documents, No source code, No
  executable code
• The impacts
• For statically bound services,
• Techniques that automatically derive stubs from
  source code are not applicable
• Automatic instrumentation of original source code
  or executable code is not applicable
• For dynamic bound services,
• Human involvement in the integration becomes
  impossible.
• Possible solutions
• (a) Derive test harness from WS descriptions
• (b) The service provider to make the test stubs
  and drivers available for integration.

9
Lack of control over test executions

• Problem
• Services are typically located on a computer on
  the Internet that testers have no control over
  its execution.
• Impact
• An invocation of the service as a test must be
  distinguished from a real request of the service.
  
• System may be need to be restarted or put into a
  certain state to test it.
• The situation could become much more complicated
  when a WS is simultaneously tested by many
  service requesters.
• Possible solution
• The service provider must provide a mechanism and
  a service that enable service requesters to
  control the testing executions of the service.

Currently, there is no support to such mechanisms
in W3C standards of WS.
10
Lack of means of observation

• The problem
• A tester cannot observe the internal behaviours
  of the services
• The Impacts
• No way to measure test coverage
• No way to ensure internal state is correct
• Possible solutions
• The service provider provides a mechanism and the
  services to the outside tester to observe its
  softwares internal behaviour in order to achieve
  the test adequacy that a service requester
  requires.
• The service provider opens its document, source
  code as well as other software artifacts that are
  necessary for testing to some trusted test
  service providers.

11
The proposed approach

• A WS should be accompanied by a testing service.
• functional services the services of the original
  functionality
• testing services the services to enable test the
  functional services
• Testing services can be either provided by the
  same vendor of the functional services, or by a
  third party.

12
Architecture of service oriented testing
13
A Typical Scenario Car Insurance Broker
GUI Interface
CIBs service requester
Bank Bs F-Services
CIBs F-Services
WS Registry
Insurance A2s F-Services
Insurance Ans F-Services
Insurance A1s F-Services
14
How does it work?

• Suppose the car insurance broker want to search
  for web services of insurers and test the web
  service before making quote for its customers.

Testing the integration of two services
Information about the car and the user
Car Insurance Broker CIB
Insurer Web Service IS
Insurance quotes
customer
15
Matchmaker
Car Insurance
Test Broker

TB
Test Broker
Broker CIB
0. Intended composition
of services
Insurer Web
Service IS
(
F
-
Service
)
Testing Service

TG
(
Test case
Generator
)
Insurer Web
Service

IS
.
(
T
-
Service
)
Testing Service

TE
(
Test Executor
)
16
Key Issues to Automate Test Services

• How a testing service should be described,
  published and registered at WS registry
• How a testing service can be searched and
  retrieved automatically even for testing
  dynamically bound services
• How a testing service can be invoked by both a
  human tester and a program to dynamically
  discover a service and then test it before bind
  to it.
• How testing results can be summarized and
  reported in the forms that are suitable for both
  human beings to read and machine to understand.

These issues can be resolved by the utilization
of a software testing ontology (Zhu Huo 2003,
2005).
17
STOWS Software Testing Ontology for WS

• Ontology defines the basic terms and relations
  comprising the vocabulary of a topic area as well
  as the rules for combining them to define
  extensions to the vocabulary
• STOWS is base on an ontology of software testing
  originally developed for agent oriented software
  testing (Zhu Huo 2003, 2005).
• The concepts of software testing are divided into
  two groups.
• Knowledge about software testing are also
  represented as relations between concepts

18
STOWS (1) Basic concepts

• Tester a particular party who carries out a
  testing activity.
• Activity consists of actions performed in
  testing process, including test planning, test
  case generation, test execution, result
  validation, adequacy measurement and test report
  generation, etc.
• Artefact the files, data, program code and
  documents etc. inovlved in testing activities. An
  Artefact possesses an attribute Location
  expressed by a URL or a URI.
• Method the method used to perform a test
  activity. Test methods can be classified in a
  number of different ways.
• Context the context in which testing activities
  may occur in software development stages to
  achieve various testing purposes. Testing
  contexts typically include unit testing,
  integration testing, system testing, regression
  testing, etc.
• Environment. The testing environment is the
  hardware and software configurations in which a
  testing is to be performed.

19
STOWS (2) Compound concepts

• Capability describes what a tester can do

• the activities that a tester can perform
• the context to perform the activity
• the testing method used
• the environment to perform the testing
• the required resources (i.e. the input)
• the output that the tester can generate

20

• Task describes what testing service is requested

• A testing activity to be performed
• How the activity is to be performed
• the context
• the testing method to be used
• the environment in which the activity must be
  carried out
• the available resources
• the expected outcomes

21
STOWS (3) Relations between concepts

• Relationships between concepts are a very
  important part of the knowledge of software
  testing
• Subsumption relation between testing methods
• Compatibility between artefacts formats
• Enhancement relation between environments
• Inclusion relation between test activities
• Temporal ordering between test activities
• How such knowledge is used
• Instances of basic relations are stored in a
  knowledge-base as basic facts
• Used by the testing broker to search for test
  services through compound relations

22
Compound relations

• MorePowerful relation between two capabilities.
• MorePowerful(c1, c2) means that a tester has
  capability c1 implies that the tester can do all
  the tasks that can be done by a tester who has
  capability c2.
• Contains relation between two tasks.
• Contains(t1, t2) means that accomplishing task t1
  implies accomplishing t2.
• Matches relation between a capability and a
  task.
• Match(c, t) means that a tester with capability c
  can fulfil the task t.

23
Prototype Implementation

• STOWS is represented in OWL-S
• Basic concepts as XML data definition
• Compound concepts defined as service profile
• UDDI /OWL-S registry server (as the test broker)
• Using OWL-S/UDDI Matchmaker
• The environment
• Windows XP,
• Intel Core Duo CPU 2.16GHz,
• Jdk 1.5, Tomcat 5.5 and Mysql 5.0.

24
Case Study

• An automated software testing tool CASCAT is
  wrapped into a test service
• Registered
• Capability is described in the ontology
  represented in OWL-S
• Searchable
• It can be searched when the testing task matches
  its capability
• Invoked through the internet
• As a web services to generation test cases based
  on algebraic specification
• A web service and its corresponding test service
  are implemented
• Both registered
• Testing of the WS can be invoked through the
  corresponding T-Service

25
Conclusion

• Challenges to testing web services applications
• Testing a web service as developers own software
• Integration testing at development and at
  run-time
• No support in current WS standard stack
• A service oriented approach is proposed
• Architecture fits well into service oriented
  architecture
• Supported by software testing ontology
• Feasibility of the approach tested via a case
  study.

26
Advantages

• Automated process to meet the requirements of
  on-the-fly service integration testing
• Automation without human involvement
• Testing without interference to providing normal
  functional services
• Testing without affect the real world state
• Security and IPR can be managed through a
  certification and authentication mechanism for
  third party specialised testing services
• Business opportunities for testing tool vendors
  and software testing companies to provide testing
  services online as web services

27
Remaining challenges and future work

• Technical challenges
• To develop a complete ontology of software
  testing (e.g. the formats of many different
  representations of testing related artefacts)
• To implement the test brokers efficiently
• To device the mechanism of certification and
  authentication for testing services
• Social challenges
• For the above approach to be practically useful,
  it must be adopted by web service developers,
  testing tool vendors and software testing
  companies
• Need standards, such as a standard of software
  testing ontology

28
References

• Zhang, Y. and Zhu, H., Ontology for Service
  Oriented Testing of Web Services, Proc. of The
  Fourth IEEE International Symposium on
  Service-Oriented System Engineering (SOSE 2008) ,
  Dec. 18-19, 2008, Taiwan. In press.
• Zhu, H., A Framework for Service-Oriented Testing
  of Web Services, Proc. of COMPSAC06, Sept. 2006,
  pp679-691.
• Zhu, H. and Huo, Q., Developing A Software
  Testing Ontology in UML for A Software Growth
  Environment of Web-Based Applications, Chapter
  IX Software Evolution with UML and XML, Hongji
  Yang (ed.). IDEA Group Inc. 2005, pp263-295.
• Zhu, H. Cooperative Agent Approach to Quality
  Assurance and Testing Web Software, Proc. of
  QATWBA04/COMPSAC04, Sept. 2004, IEEE CS, Hong
  Kong,pp110-113.
• Zhu, H., Huo, Q. and Greenwood, S., A Multi-Agent
  Software Environment for Testing Web-based
  Applications, Proc. of COMPSAC03, 2003,
  pp210-215.