An Automated Test Strategy Based on UML Diagrams

1
An Automated Test Strategy Based on UML Diagrams
2
Objectives

• Use for test planning the same UML diagrams
  developed (e.g., by Rational Rose) for design and
  analysis
• Tool-supported generation and maintenance of an
  updated and strategic test plan, as the design
  evolves.

3
Cow_Suite approach

• The processing of the Cow_Suite approach starts
  by analysing the project description, and
  proceeds in parallel with the design and analysis
  refinement steps.
• The reference documentation is derived by
  collecting together and automatically organizing
  the diagrams developed during the design phase
  without additional manual effort of formalization
• the internal description of the project
  model, given by Rational Rose in a .mdl file

4
Cow_Suite Scheme
Test Procedures list
5
Nodes and Arcs identification (Activity 1)

• The UML design is analyzed for deriving two
  different graph The Main Graph and the Design
  Graph. These represent two different points
  typologies of test.
• For deriving the two different graphs is
  necessary to analyze
• the Use Case View
• UCs, SDs, Actors represent the nodes of the Main
  Graph
• Their relations represent the arcs of the Main
  Graph
• the Logical View
• The UC realizations and their associated SDs
  represent the nodes of the Main Graph, the
  remaining design elements are the nodes of the
  Design Graph
• The relations and the dependences are the arcs of
  the two graphs

6
Use Case View

• The Use Case View describes the set of UCs (and
  scenarios) that represents the significant
  functionalities and the interaction of the system
  with the external world.
• The tree derived represents a detailed
  documentation of how features/system requirement
  are realized in the Use Case View.
• The path from a high level Use Case to a leaf
  (UCs or SDs) describes the refinement of the
  associated system feature
• The Test Cases derived from the SDs of the Use
  Case View are specifically designed for
• System integration Test
• Subsystem integration Test

7
Use Case View vs Logical View

• The Logical View describes how the system
  functionalities are provided in terms of static
  structure and dynamic collaborations of the
  objects. This information is used for completing
  the graphs and in particular fro the Main Graph
• The Use Case Realizations link the information
  of the Use Case View and the Logical View. For
  each UC its UC realization
• Traces the evolution of the UC in the Design
  Model
• Details the sub-flow of a UC with Sequence
  (Collaboration) Diagrams
• Specifies the Classes and the relationship that
  participate in the realization of the UC (Class
  Diagrams)
• Optionally holds the package Design Link which
  holds the list of the packages of the design
  model that effectively implement the UC

8
Trees Derivation (Activity 2)

• The Main Graph and the Design Graph are
  automatically organized by Cow_Suite tool in
  different trees by using a DFS_mod algorithm

The case study is Course Registration System from
RUP documentation
9
Choice of a tree (Activity 3)

• The users can get a complete view of the status
  of the functionalities specification and a
  detailed documentation
• UCs and their realization
• SDs associated to each specification level
• The links between Use Case and Design Model
• Reused nodes
• Not linked model elements

10
Design Link Package

• Design Link package for every use case
  realization it collects the package/s that
  represent the implementation of the relative UC.

It is an explicit link between the Main Trees and
the Design trees
11
Design Tree
Design Tree is derived directly from the Design
Model and represents the organization of the
packages in a hierarchy.

• Analyzing this tree it is possible to
• Observe the dependence relationship between
  packages
• Know which are the reused packages and where the
  reuse happens
• Connect to the main tree the low level SDs not
  explicitly linked in the .mdl file

By selecting the SDs of the Design Tree and
applying to them the Cow_Suite test derivation
strategies, a list of low level Test Cases can be
derived
12
Weights assignment (Activity 4)

• Systems are naturally composed of different parts
  realizing one or more functionalities, each with
  different importance for the overall system
  performance.
• The effort devoted to the test of different
  functionalities must be planned and scheduled
  considering their relative importance
  (risk-based testing, reliability guided testing,
  etc).
• Following level by level the tree structure of
  the UCs and SDs, the strategy allows the people
  in charge of the development of one or more
  functionalities to simply annotate the
  corresponding node in the tree with a number
  (weight).

13
Deduce the Critical Profile

• The importance, represented by number (weight)
  belonging to 0,1 interval, can be assigned in
  two different manner
• Interacting with Cow_Suite Tool Following level
  by level the tree structure of the UCs and SDs,
  the user annotates the corresponding node in the
  tree with its weights
• Directly in the .mdl file (UCs only) During the
  development of the Use Case Model
  architects/designers assign a value, belonging to
  1,9 interval, to each Use Case.
• The value is memorized in the use case
  documentation tab with the following structure
  WGH value
• (For SDs or when no value is expressed for UCs
  the default value is 5)

14
Weighted tree derivation

• The tool
• reads the assigned values the .mdl file (if they
  exist)
• sums the values of the node at the same level in
  the tree
• normalizes the weights among the nodes at the
  same level so that their sum is 1.

The user can always modify the assigned weights
15
Integration Testing vs Integration Stage

• To realize a complete system functionality, it is
  generally necessary to execute several actions
  realizing lower level functionalities
• A system functionality is realized by the
  different components interaction.
• Integration Testing testing interactions between
  system components (processes, packages,
  subsystems ) at an appropriate level of
  abstraction.
• Integration Stage is useful because generally
  the functionalities are not specified at the same
  level of detail and with the same number of UCs
  and SDs

16
Integration Stage definition

• The first integration stage is represented by the
  main UC and the SDs (if any), which are children
  of this node (hence they are at level 2 of the
  tree).
• The i-th integration stage is represented by the
  UCs positioned in at i-th level of the tree and
  every SDs, children of these nodes, situated at
  i1-th level.
• NOTE the SDs at level i1 represent the
  interaction between the different components
  that realize the functionalities described in the
  UCs at i-th level.

17
Integration Stage selection (Activity 5)

• The Cow_Suite user selects the integration stage
  at which the test is conducted.
• The nodes that are considered by Cow_Suite tool
  for the subsequent evaluations are all and only
  those belonging to the selected integration stage
  plus all the tree leaves existing at higher
  levels.

18
Test Strategy selection (Activity 6)

• Two different situations are considered in test
  cases planning
• 1 Cowtest_ing with fixed number of tests
• A certain budget is available, which is
  translated in terms of number of tests, NT, to
  execute.
• In this case the tool select the most adequate
  selection of NT tests from among the (many)
  various test cases that could be potentially
  conceived.

19
Test Strategy selection (Activity 6)

• 2 Cowtest_ing with fixed functional coverage
• A certain percentage of functionalities must be
  covered.
• In this case by the tool is possible to define in
  advance which are the functionalities to be
  covered and the minimum number of tests to
  execute.

Functional Coverage 90
20
Select the Test Strategy
21
Weighted Tree Derivation (Activity 7)

• The weights assigned to each node are used to
  define a relative importance factor, in terms of
  how risky is that node and how much effort should
  be put to test it.
• Considering every node, from the root down to the
  elements belonging to the integration stage
  considered, its final weight is computed as the
  product of all the nodes weights on the complete
  path from the root to this node.
• The final resulting weight is an element of
  discrimination for choosing amongst the tests to
  execute.

22
Cow_Suite Scheme
23
Sequence Diagrams Selection (Activity 8)

• For each fixed integration stage, a weighted
  subtree is derived according to the choosen test
  criterion. Only the relative SDs are considered
  for Test Case generation.

24
Test Case Derivation (Activity 9)

• The tool uses the UIT method, Use Interaction
  Test, to derive the Test Cases.
• The method is mainly based on the SDs
  (particularly objects, messages and parameters)

25
Use Interaction Test methodology

• A method to systematically derive Integration
  Test suites from UML diagrams.
• Inspired at large by the Category Partition
  method
• it defines classes of functional equivalence by
  selecting significant values (choices) of
  relevant categories (parameters or data)
• Incremental test strategy (over the Use Case
  organizational structure)

26
9.1 Highlight Test Units

• Horizontal axis shows a set of objects, Test
  Units, that interact through messages.
• Those are system units separately testable for
  exercising a possible use of system.
• Example
• DecisionModel, Decision, GoalModel, Goal.

27
9.2 Identify Interactions Categories

• Vertical axis shows
• the exchanged messages, Interactions Categories,
  involved in object interactions.
• We list all the messages entering in the selected
  Test Unit.

Message is a communication where the sender
object invokes an action, a service, belonging
to the receiver object which will perform it.
28
9.3 Identify Settings Categories

• Vertical axis shows
• their parameters and relevant inputs, Settings
  Categories.

Example DecisionModel Settings
_decisions Interaction DecisionModel() getDecisi
ons() Decision Interactions
Decision(nameString,priorityint) GetName(),
GetPriority() ............
29
9.3 Additional Information Class Diagram
Complementary analysis of the Class Diagram
description (mainly used for searching the
Settings Categories). These are attributes (or a
subset of them) of a class (and the corresponding
SDs object) that affect object interactions and
are represented by input parameters used in
messages or data structures.
30
9.4 Define Messages_Sequences
For each SD where the studied Test Unit is
involved, observing the vertical temporal order
of the messages along its lifeline the tool
detects a set of Messages_Sequences created
considering each message entering in the Test
Unit plus, if any, all the messages (not yet
considered) belonging to its activation bounded
from focus of control region.
A Messages_Sequence describes interactions
between the selected Test Unit and the other
objects necessary to realize a specific system
functionality.
31
The implemented UIT UIT_sd

• From an industrial point of view UIT is too
  expensive because
• generates many Test Cases of a small
  granularity
• needs lot of ad-hoc stubs

• UIT_sd, for each selected SD
• Messages_Sequence is defined considering, in
  temporal order, each message with no predecessor
  association plus, if any, all the messages
  belonging to the nested activation (focus of
  control region). A Messages_Sequence represents a
  behavior to be tested and describes the
  interactions among objects necessary to realize
  the corresponding functionality.

32
9.4 Define Messages_Sequences in UIT_SD

• Example
• Messages_Sequence 1
• DecisionModel()
• Decision (name, priority)
• Messages_Sequence 2
• GoalModel()
• Goal (name, priority)

33
9.5 Test Cases Derivation

• For each traceable Messages_Sequence a Test Case
  is generated. It contains the list of all
  Settings and Interactions Categories involved in
  the Messages_Sequence and their values.

34
Number of Test Procedures

• The final weight of every SDs is used to
  automatically derive the number of Test
  Procedures associated to each Test Case.

35
9.5.1 Analyze possible subcases

• The messages involved in a Test Case (TC) may
  contain some feasibility conditions
• usually described in messages notes or
  specifications tab
• formally expressed using the OCL notation

TC is divided in different subcases corresponding
to the diverse choices values.
36
Test Procedures Specification (Activity 10)

• Test Specification
• For each identified category, we consider all its
  possible values and constraints (choices).

37
10.1 Interactions Categories Specification

• Interactions
• Categories objects interactions, exchanged
  messages.
• The tool implements two possible choices for the
  user
• the default values,
• the user values

38
State Diagram
If a State Diagram is linked to a specific Test
Unit (class), further information can be found
about the relevant configurations of
InteractionCategories
State diagram of the Critic class
39
10.2 Settings Categories Specification

• Settings
• Categories
• parameters and inputs relevant for the Test Unit.
  
• The tool implements two possible choices for the
  user
• the default values,
• the user values.

40
10.3 Constraints on Choices

• Some choices combinations could result
  contradictory or meaningless. Following the
  Category Partition methodology, constraints must
  be added to the Setting Categories choices.
• Constraint are specified using
• Properties, that can be assigned to certain
  choices to check the compatibility with other
  choices.
• IF Selector, is a conjunction of properties
  previously assigned to other choices.

41
Example

• SettingsCategories
• pattern size
• empty Property Empty
• single character Property Not Empty
• many character Property Not Empty
• quoting
• pattern is quoted Property quoted
• pattern is not quoted if Not Empty
• pattern is improperly quoted if Not Empty

42
Settings Categories Specification Constraints

• Test Specification
• Decision(_name, _priority)
• _name
• Naming if prioritylt5
• Class Selection if priority0
• Storage
• Inheritance
• Modularity..
• _priority
• 0 Property priority0
• 1
• 2
• 3
• 7 Property not prioritylt5

43
Settings Categories Specification Constraints

• Test Procedure
• DecisionModel()
• Opening an existing document
• Decision( _name, _priority)
• _name
• Naming if prioritylt5
• _priority
• 7 Property not prioritylt5

Test Procedure DecisionModel() Opening an
existing document Decision( _name,
_priority) _name Naming if prioritylt5 _prior
ity 4
X
ok
44
Test Procedures Generation (Activity 11)

• A Test Procedure is automatically generated from
  a Test Specification, for each possible
  combination of choices of every category involved
  in a Messages_Sequence.

45
Test Procedures an example

• Test Specification
• DecisionModel
• Settings
• _decisions
• Naming
• Storage
• Relationships..
• Interactions
• getDecisions
• Opening a new file
• Opening a saved file
• After a modification and before saving
• After a modification and after saving
• DecisionModel()
• Class constructor

Test Procedure getDecisions() getName()
Opening a saved file _decisions
Naming Note Such a Test Case must be repeated
for all possible values of _decisions
46
Test Suite artifact (Activity 12)

• The set of all generated Test Procedures is named
  Test Suite

47
Note

• The Test Procedures, as derived by the tool, are
  not directly executable (? Test Scripts). The
  derived Test Procedures have to be executed and
  logged by a test driver.
• CowSuite goal is
• giving automatic and systematic support to
  construct a meaningful Test Suite.
• It is not to directly execute the tests.

48
Remarks

• The degree of detail of the Test Suite depends
  directly on
• the design detail level
• granularity of the inserted values (selection of
  the constraints and choices of the UIT
  categories).
• Currently the final result is a text document
  containing the Test Suite.
• In future different format (e.g. XML), that can
  be accepted in input by the adopted test driver.

49
Questions?
50
Procedure for tree derivation

• The strategy followed for the tree derivation is
  a modified depth-first search
• DFS_Mod (G)
• For each vertex v?VG
• do coloru ?WHITE
• ?u?NIL
• time?0
• For each vertex u ?ActorsG
•                                     do
  DEF-Visit_Mod (u)
• For each vertex u ?VG\ActorsG
• do if coloru WHITE
• then DEF-Visit_Mod (u)

51
Procedure for tree derivation

•  DEF-Visit_Mod (u)
• Coloru ? GRAY
• Du ? time ?time 1
• For each v ?Adju
• If v ?ActorG then
• v? NewNodeGeneration(v)
•              ?v?u
• else
•     Do if colorv WHITE
•           then ?v?u
•            DFS-Visit_Mod(v)
•                 else
•                v? NewNodeGeneration(v)
•                  ?v?u
•              if v?Predecessorsu,v then
•              TreeDuplication v,v
•  coloru ? BLACK
•  fu ? time ?time 1

52
Conclusion
Systematic and automated support for test
planning and test case generation, based on the
same UML diagrams developed for design and
analysis
OK
Tool integrated with the design tool (e.g.
Rational Rose)
OK