EuroSTAR%202006%20Making%20the%20Most%20of%20Test%20Design

1
EuroSTAR 2006Making the Most of Test Design

• Torbjörn Ryber
• Keep IT Simple

2
(No Transcript)
3
(No Transcript)
4
Testers are Complainers

• Testers are often seen as complainers.
  Unfortunately this is too often the truth
• We demand to participate early in the project.
  Then we complain about the quality of
  requirements
• We can do better!

5
What Can we NOT Change?

• The way requirements are handled
• How the project is run
• What development process is used (or not)
• The deadline (at least most of the time)
• So leave it alone!

6
What Can we Change? Ourselves!

• Our knowledge of testing
• Our knowledge of development
• How we adapt to a situation
• The end result
• Do it!

7
Some Wisdom

• Give me the power to change what I can change,
  peace of mind to accept what I cannot and skill
  enough to tell one from the other

8
What does a tester have to know?

• Practical considerations executing tests
• Asking the right questions test design
• Quality assurance the models you create

9
Executing tests
10
How do I ask questions?

• Graphical User Interface
• Easy to ask
• Hard to test
• Harder to automate
• Read a file
• Interface may already exist
• Create your own interface
• Build your own
• Ask the developer

?
11
How do I see the answers?
?

• Grapical User Interface
• Instant answer
• Harder to automate
• Reports
• Maybe not enough!
• File
• May already exist
• Easy to check
• Create your own
• SQL-query
• Ask developer demand testability

12
Know Your Oracles

• Manually from requirements
• Older tested/verified version of same system
• Current other systems where we can assume data is
  correct
• Formulas in MS Excel or other tools
• Heuristics

13
How do I test this on time?
14
Strategy parallel testing GUI and Rules

• Demo Raktprototyp

15
Creative planning solved it!
Alt1 time to short for development and testing
Alt2 longer time period for both development AND
testing
16
Philosophy and Testing

• Every genuine test of a theory is an attempt to
  falsify it, or to refute it. Testability is
  falsifiability
• (Karl Popper Conjectures and Refutations p
  48)
• A requirement is testable if it possible to show
  that it is not true. (If you cant its a
  Barnum statement)
• More difficult tests give more information
• These tests demand better test design

17
Test Design asking the right questions

1. Analyse, create model
2. Cover the model with (basic) test cases
3. Add test data for deeper tests
4. Finally the whole system and some creative
   thinking

18
Good Quality Assurance

• Good Test Design finds more defects than test
  execution!
• Test design is verification of the requirements
• Test execution is validation of the actual code

19
Model examples
H
20
All Science is Based on Models

• All models are wrong but some are useful. (Gerald
  Weinberg)
• A good model is easy to understand
• A good model is a true simplification of reality
• Too little information makes the model incomplete
• Too much information makes the model hard to
  understand
• Quality Assurance of the models are of utmost
  importance

21
Test Design Techniques

• All techniques use some kind of model
• Focus on
• Data
• Flows
• Logic
• Combinatorics
• Overall tests

22
Data is a part of all techniques

• Equivalence partitioning
• Numeric
• Other types text, lists, time dependent,
  relative
• Boundary Values

23
Flows

• Variations
• Business processes
• Use Case Flows
• Control and Data Flows
• Design Technique
• Create Flow Graph
• Cover branches, decisions, paths etc

24
Event-Driven

• Real-time systems (telephone, dialogue, ATM)
• Design Technique
• Create State Graph/table
• Cover nodes, links, paths

25
Dialogues
Concurrent users? Robustness? Time-Outs?
26
Insurance policy states
27
Driving a train
28
Logic

• Variations
• Set of Rules
• Conditions
• Formulas
• Design techniques
• Decision table
• Decision Tree

29
Combinatoric Explosion

• Common Problem too many combinations
• Design Technique
• Elementary Comparison (MCDC)
• Pairwise Testing allpairs.exe

30
Advanced testing the whole picture

• Syntax standard over the whole system?
• Data Cycle covered in CRUD?
• Time Cycle regular events that need to be
  tested (yearly, monthly etc)
• Soap Opera
• Non-functional what should WE do?

31
How do we get the respect we (think we) deserve?

• By adding value - And showing that we add value
• Testers light the way
• - James Bach
• Effective and efficient test design adds value in
  all phases
• Whatever there is we will test it and inform of
  our findings

32
Testing is Like Martial Arts
If you believe you already know everything you
will never learn! You must open your mind.
(Tomita Sensei 8th Dan)
33
Your Success Depends on YOU!

• Enhance your competence
• Learn how to model
• Learn test design techniques
• Read, take classes, be creative
• Be flexible adapt to the situation
• Testing is advanced stuff you need skill to
  succeed
• Change yourself not others

34
Recommended books

• Testdesign för programvara (Swedish)
• Ryber, Torbjörn
• A Practitioners Guide to Test Design
• Copeland, Lee
• Lessons Learned in SW Testing
• Kaner, Bach, Pettichord
• How to Break Software
• Whittaker, James
• Exploring requirements
• Weinberg, Gerald
• Lateral Thinking
• De Bono, Edward
• Mail torbjorn_at_ryber.se