Test Purpose Concretization through Symbolic Action Refinement

1
Test Purpose Concretization through Symbolic
Action Refinement
Ecole Centrale Supelec
CEA-LIST Pascale Le Gall Assia Touil
Christophe Gaston Alain Faivre
2
Model based testing process
Model
conforms
SUT
Test cases extracted from
Messages exchanged through a common interface
Conformance relation is based on the shared
interface hypothesis
3
Model based testing process
ATM model
pin?4356
ATM implementation
digit?4.digit?3.digit?5.digit?6
Action refinement relation between abstract
actions and sequences of concrete ones process
algebra
(pin?4356 , digit?4.digit?3.digit?5.digit?6)
4

• Input/Output Symbolic Transition Systems
• Symbolic Execution based conformance
• testing
• Action refinement in symbolic conformance testing
• Test purpose concretization

5

• Input/Output Symbolic Transition Systems
• (IOSTS)

6
IOSTS example
Btrue drink!coffee
mlt2 screen!more money
drink?B
m?2 screen!a drink?
mm-2
money?x m mx
Bfalse drink!tea
7
IOSTS Semantics paths
A path is a sequence of consecutive transitions
of an IOSTS. Here is a path p
money?x
mmx
m2
screen!a drink?
q1
q0
q2
m m-2
A possible run of p is
(i0 x?0, money?3, i1 x?3)(i1 x?3 , screen!a
drink?, i2 x?3 ) m?0
m?3 m?3 m?1
Corresponding trace money?3 .screen!a
drink?
Quiescence enrichment since
money?3.screen!a drink? cannot be continued in
money?3.screen!a drink?.out then
money?3.screen!a drink?.?!.?!.... are traces
of p
8

• Symbolic Execution based conformance
• testing

9
Symbolic Execution (SE) Context and Principles

• SE introduced by King in 1975 to characterize
  and
• explore feasible paths of programs
• Adapted to IOSTS
• Main idea
• To execute IOSTS not for real input values but
  for
• symbolic input parameters
• To characterize constraints on state variables
  
• associated to executions

10
Symbolic Execution (SE) Example
11
Test purpose intuition

• Test purposes are first class citizen
  behaviors to be tested
• Symbolic Execution Tree
• description of all expected behaviors
• A symbolic path of SE tree
• an intentional class of similar concrete
  behaviors
• Test purpose a finite path of the SE tree

12
Test purpose example (1)
screen!  " a drink? "
screen!  " more money "
drink!  " tea "
drink!  " coffee  "
13
Test purpose example (2)
mmx
mm-2

14
Conformance testing algorithm conformance
relation

• ioco conformance relation Tret 96
• SUT ioco G iff
• For any trace tr of TR(G)? TR(SUT),
• If there exists out in c!t ? t concrete
  data ??!
• such that tr.out ? TR(SUT),
• then tr.out ? TR(G)

15
Conformance testing algorithm principles
observes SUT reactions and either computes new
inputs or assigns a verdict
uses TP to compute inputs for SUT

SUT
screen! more money
money?1
16
To resume the 4 verdicts

• Fail
• The test case is neither a trace of TP nor a
  trace of the SE Tree
• Inconc
• The test case is a trace of the SE Tree, but not
  a trace of TP
• Pass
• The test case is a trace of TP, and not a trace
  of any other path of the SE Tree
• WeakPass
• The test case is a trace of TP, and a trace of at
  least one other path of SE Tree

17

• Action refinement in symbolic conformance
  testing

18
Action refinement in Labelled Transition Systems
Van der Bijl and all 05
A Abstract model
r (a,s)
iocor-conform
C concrete model C Aa ? s
ioco-conform
SUT
Result SUT ioco C iff SUT iocor A
19
Action refinement with symbolic data issues (1)

• Example of inputs in LTS (enumerated data)
  pin?4356
• Example of symbolic input pin?x
• pin?x for x 4356 ? digit?4 . digit ?3 .
  digit?5 . digit ?6
• pin?x for x 3271 ? digit?3 . digit ?2 .
  digit?7 . digit ?1
• ? Refinement denotes several concrete input
  sequences depending on assignments of x

20
Action refinement with symbolic data issues (2)

• Example of inputs in LTS money?2
• Example of symbolic input money?x
• money?x for x 2
• ? coin?2
• ? coin?1 . coin ?1
• ? Several concrete input sequences may correspond
  to a common assignment of x

21
Action refinement with symbolic data issues (2)

• LTS case (enumerated data)
• An abstract action c?v
• is refined by a concrete sequence c1?v1.cn?vn
• Symbolic case
• An abstract action c?x
• is refined by a symbolic automaton (IOSTS)
  characterizing all
• traces that may refine c?x

22
Action refinement with symbolic data refinement
pair
money??
refined by
(money,(G,s,?)) is a refinement pair
23
Refinement function

• Suppose
• Ga is an IOSTS introducing an abstract channel c
• (c,(G,s,?)) is a refinement pair
• Refinement function is a syntactical mechanism
  which builds
• a concrete IOSTS Gc by replacing any abstract
  input c?x by
• a refining IOSTS built accordingly to the
  refinement pair

24
Refinement function example
Btrue drink!coffee
mlt2 screen!more money
drink?B
m?2 screen!a drink?
mm-2
money?x m mx
Bfalse drink!tea
25
Refinement function example
transition tr
for the refinement pair (money,(G,s,?)) with
q1
q0
true money?x m mx
e
G
coin?y ? y
is refined as
s
coin?y ? ?y
mmx
true
26
Refinement function example
Btrue drink!coffee
mlt2 screen!more money
drink?B
m?2 screen!a drink?
mm-2
money?x m mx
Bfalse drink!tea
27
Refinement and testing processes
Test purposes extracted from
Gc
conforms
behaves consistently to
SUT
28
Refinement and testing processes
depends on conformance up to refinement
Test purposes corresponding to requirements are
those extracted from
Ga
well founded?
Concretized test purposes
Test purposes extracted from
Gc
conforms
behaves consistently to
SUT
29
IocoR conformance relation trace abstraction (1)
R (money,(G,s,?)) with
s coin?2 . coin?2
money?4
Abs(s,R)
30
IocoR conformance relation trace abstraction (2)

• Abs may be extended to trace of SUT by
  leaving unchanged actions which does not involve
  channels to be refined
• s a1.. am. b1bn .c1 .. ck is a trace of
  SUT
• Abs(s,R) a1.. am.d.c1 .. ck is a trace
  on INT(Ga)
• with ai and ci built over channels not
  occurring in G
• and b1 bn a trace of G abstracted in d

31
IocoR conformance relation definition and theorem

• SUT iocoR Ga
• s of SUT can be abstracted in sa ?TR(Ga)
• s.out ?TR(SUT) ? sa.out ?TR(Ga)
• s of SUT can be extended in sand s can be
  abstracted in sa?TR(Ga)
• ? s.out ?TR(SUT)
• Theorem
• SUT ioco Gc ? SUT iocoR Ga

32

• Test purpose concretization

33
Refinement and testing process
screen!  " a drink? "
screen!  " more money "
drink!  " tea "
drink!  " coffee  "
34
Refinement and testing process
mmx
mm-2
coin?y x xy

true ?
35
Refinement and testing process
screen!  " a drink? "
screen!  " a drink? "
drink!  " coffee  "
drink!  " coffee  "
accept
accept
36
Conclusion

• A framework to take benefits of action refinement
  in conformance testing TestCom08
• Refinement mechanisms are defined symbolically a
  refinement pair associates an abstract input
  action and a set of concrete ones (represented as
  an IOSTS)
• Sets of refinement pairs may be considered
• Methodology of test purpose concretization
• Perspectives
• Output refinements and outputs in refining IOSTS
• Other refinements than action refinement
  APSEC05, TAP07