Strukturinis Testavimas

1
Strukturinis Testavimas
2
Testavimo procesas

• Komponentu testavimas
• individualiu programos komponentu testavimas,
• iprastai atsakomybe u komponentu testavima tenka
  komponentu kurejams, iskyrus
  
  kritines sistemas,
• testai yra gaunami pagal kureju patirti.
• Integravimo testavimas
• komponentu, sujungtu i atskiras grupes,
  testavimas, sukuriant sistemas ar posistemes,
• atsakomybe tenka nepriklausomoms testavimo
  komandoms,
• testai remiasi sistemos specifikacija.

3
Testavimo fazes
4
Introduction to Testing
5
First a riddle about testingby Brian Marick

• A mathematician, a physicist, and an engineer are
  told All odd numbers are prime.
• The mathematician says, Thats silly, nine is a
  non-prime odd number.
• The physicist says, Lets see, 3 is prime, 5 is
  prime, 7 is prime -- looks like its true.
• The engineer says, lets see, 3 is prime, 5 is
  prime, 7 is prime, 9 is prime, 11 is prime --
  looks like its true.

6
Software testing

• Historically, was not popular
• with managers
• with testers
• with developers
• with students
• testing and many software innovations evolved out
  of the software crisis

7
Software Failure rate (ideal)
8
Software Failure rate (real)
How does AO OO relate to this figure?
9
The Cost of Change
When should testing be done?
10
An error found after release costs four times (W.
Perry)

• 1st cost developing program erroneously
• 2nd cost system has to be tested to detect the
  error
• 3rd cost wrong specs/code removed, correct
  specs/code added
• 4th cost system must be retested!

11
The software crisis

• By the 1980s, quality in software became a
  goal SEI was born
• software engineering became popular
• the life cycle was studied
• software developers and testers began to work
  together
• by the 1990s, testing tools became available

12
What is software testing

• The process of executing computer software in
  order to determine whether the results it
  produces are correct, Glass 79
• The process of executing a program with the
  intent of finding errors, Myers 79
• Program testing can be used to show the presence
  of bugs, but never their absence, Dijkstra 72

13
What is software testing (cont)

• The aim is not to discover errors but to provide
  convincing evidence that there are none, or to
  show that particular classes of faults are not
  present, Hennell 84
• Testing is the measure of software quality,
  Hetzel 85

14
What is software testing (cont)

• The process of operating a system or component
  under specified conditions, observing or
  recording the results, and making an evaluation
  of some aspect of the system or component.
• IEEE/ANSI, 1990

15
Testing is a state of mind

• If our goal is to show the absence of errors, we
  will find very few of them
• If our goal is to show the presence of errors,
  we will discover a large number of them

Myers, 1979
16
Time spent on testing

• 50 Brooks/Myers, 1970s
• 80 Arthur Andersons Director of testing in
  North America, 1990s

17
Tester-to-developer ratios

• 15-10 Mainframes
• i.e.,1 tester for every 5 to 10 developers
• 23 Microsoft, 1992
• 21 Lotus (for 1-2-3 Windows)
• 12 Average of 4 major companies,1992 Microsoft,
  Borland, WordPerfect, Novell

18
Difficulties in testing software

• poorly expressed requirements
• informal design techniques
• nothing executable until coding stage
• Huge input set consider testing software that
  categorises an exam grade 101 inputs
• consider testing software that categorises two
  exam grades 101101 inputs!

19
Difficulties in testing software (cont)

• Exhaustive software testing is intractable
• Even if all possible inputs could be identified,
  the problem of identifying non-halting cases is
  undecidable
• Weyuker (1979) has shown that there is no
  algorithm that can determine if a given
  statement, branch or path will be exercised!
• well look at this difficulty in more detail
  after we understand graphs

20
Strukturinis testavimas

• Dar vadinamas baltos dees testu.
• Testiniai atvejai gaunami i programos
  strukturos. inios apie programa naudojamos
  nustatyti papildomus testinius atvejus.
• Tikslas yra ibandyti visus programos operatorius
  (ne visas keliu kombinacijas).

21
Baltos -dees testavimas
Komponento kodas
22
Keliu testavimas

• Keliu testavimo tikslas yra isitikinti, ar
  testiniu atveju rinkinys yra toks, kad kiekvienas
  kelias per programa yra ivykdytas bent karta.
• Keliu tikrinime esminis yra programos skaiciavimu
  (flow) grafas, kuris parodo programos mazgus
  priimancius sprendimus ir lankus, rodancius
  skaiciavimu valdyma.
• Salyginiai operatoriai yra skaiciavimu grafo
  mazgai.

23
control flow graph

• Directed graph G(V, E)
• V is set of vertices
• E is set of edges, E VXV
• The granularity of the vertices can be an
  operation, a statement or a basic block
• The edges are directed direction indicates flow
  of control from one vertex to another

24
Programos skaiciavimu grafas

• Aprao programos skaiciavimu valdyma. Kiekviena
  ataka yra parodyta kaip atskiras kelias ir
  ciklai parodyti kaip rodykles griimo i ciklo
  salygos mazga.
• Yra naudojamas kaip pagrindas ciklomatiniam
  (cyclomatic ) sudetingumui skaiciuoti
• ciklomatinis sudetingumas briaunu skaicius
  mazgu skaicius 2

25
Ciklomatinis sudetingumas

• Testu skaicius, kad patikrinti visas valdymo
  instrukcijas prilygsta ciklomatiniam
  sudetingumui.
• Ciklomatinis sudetingumas lygus salygu skaiciui
  programoje.
• Naudingas, taciau reikia naudoti atsargiai, nes
  neadekvatus testavimui.
• Skaiciavimu vykdymas visais keliais neatitinka
  visu keliu kombinaciju vykdymo.

26
Binary search flow graph
27
Nepriklausomi keliai

• 1,2,3,8,9
• 1,2,3,4,6,7,2
• 1,2,3,4,5,7,2
• 1,2,3,4,6,7,2,8,9
• testiniai atvejai turi buti gauti taip, kad visi
  i iu keliu butu ivykdyti
• dinaminis programos analizatorius gali buti
  panaudotas patikrinti, ar tie keliai buvo
  ivykdyti.

28
Operatoriai, akos, keliai

• Example Proc(x) (1)
• If x gt 17 then (2)
• x x - 17 (3)
• If x 13 then (4)
• x 0 (5)
• End (6)
• Testiniai duomenys x 30- visi operatoriai
• Testiniai duomenys x 30, x 17- visos akos
• Testiniai duomenys x 30, x 17, x 13, x
  21- visi keliai
• 1,2,3,4,5,6 1,2,4,61,2,4,5,61,2,3,4,6

29
basic block (defn)

• sequence of statements such that the only
  entrance to the block is through the first
  statement and the only exit from the block is
  through the last statement

30
Lets consider path testing

• Construct test cases to exercise all paths
  through a program.
• Called path coverage.

31
Finding the square root of an inputted value an
example
start read number root square_root(number)
print root end
one path!
32
Finding the square root
start read number if number gt 0 root
square_root(number) print root else
print error message endif end
1
t
f
1
2
2
3
3
how many paths?
4
4
33
Finding the square root
1
start read number while number ! 0
if number gt 0 root square_root(number)
print root else print
error message endif read number
endwhile end
1
2
2
3
3
4
4
5
5
how many paths?
6
6
7
7
34
How many paths?
1
a
ah
2
b
abcegh
3
g
d
c
abcegbcegh
4
5
h
f
e
abcegbcegbcegh
6
etc.
7
35
Exam processing example

• consider a program to process one exam result for
  10 students
• categorise the result as A, B, C, D, F
• How many paths through the program?

36
Find the number of paths for 10 inputs
37
White Box Testing

• Aim to test every path through the program
• This should ensure 100 correct programs?
• How many paths in the following program?
• 100 lines of C code, starts with var decls.
• 2 nested loops, executing between 1 20 times
• Inside inner loop, four if-then-else statements
• How long to test at 1ms per test?

38
Rationale for White Box Testing

• Errors tend to occur in code written to handle
  special cases
• Our assumptions about which parts of our programs
  are executed most often are frequently wrong
• Typos can occur in out-of-the-way parts of the
  program as easily as in the main control flow.

39
Condition Testing

• Branch Testing (Myers)
• Write test cases so that the true and false
  branches of every condition are executed at least
  once.
• Domain Testing (White and Cohen)
• For relational comparisons (e.g. x lt y)
• 3 test cases x lt y, x y, x gt y
• Choose neighbouring x and y

40
Loop Testing

• Beizer proposed this approach to testing loops
• Different types of loop
• simple
• nested
• concatenated
• unstructured

41
Testing Simple Loops

• Create a test case for the following situations
• the loop is never executed
• the loop is executed once
• the loop is executed twice
• the loop is executed m times (m lt n)
• the loop is executed n - 1 times
• the loop is executed n times
• the loop is executed n 1 times

42
Simple Loop Example

• empRec read(taxFile)
• totalTax 0
• while not eof(taxFile) do
• begin
• totalTax totalTax empRec.tax
• empRec read(taxFile)
• end

43
Testing Nested Loops

• Simple loop testing for inner-most loop, with
  outer loops held at minimum iterations
• Add other tests for excluded or out-of-range
  values
• Work outwards, keeping outer loops at minimum
  iterations and inner loops at typical numbers
  of iterations
• Continue until all loops have been tested

44
Testing Other Kinds of Loop

• Concatenated loops
• test as for simple loops if independent
• test as for nested loops if dependent
• Unstructured loops
• Dont start from here!!!

45
Test Coverage

• Refers to the proportion of the potential paths
  through a program that are covered by a given
  test set.
• Ideally, we want to maximise test coverage while
  minimising the resources used during testing
• McCabe proposed Basis Paths as solution

46
Basis Path Testing (1)

• Depends upon view of a program
• or design as a flow graph

initialise counter read first record while not
eof do process record if okay then
increment counter else report error
endif read next record endwhile
47
Basis Path Testing (2)

• A basis set of paths through a program executes
  each instruction in that program at least once
• An independent path in a basis set is one which
  differs from other paths in the set in at least
  one way

48
Independence of Paths

• Example paths
• 1,2,3,11
• 1,2,3,4,5,7,9,10,3,11
• 1,2,3,4,5,7,9,10,3,11
• 1,2,3,4,6,8,9,10,3,11

49
Cyclomatic Complexity

• The number of independent paths through a graph
  is called the cyclomatic complexity of the graph
• v(G) e(G) - n(G) 1 (or sometimes 2)
• e(G) number of edges in G
• n(G) number of nodes in G
• Gives an upper limit for number of test cases

50
Cyclomatic Complexity Example

• e(G) 12
• n(G) 11
• v(G) 12 - 11 1
• 2

51
Basis Path Testing Cont.

• Once you have determined the cyclomatic
  complexity, v(G) test cases can be generated.
• Determine basis set of independent paths
• Prepare a test case that will cause execution of
  each such path
• Some basis paths can only be tested in
  conjunction with others