Improving Software Testing by Observing Process

1
Improving Software Testing by Observing Process

• -Ossi Taipale
• -Kari Smolander
• Lappeenranta University of Technology, Finland

Presented by Albert Saryan and Karo Mazidzhyan
2
Breakdown

• Introduction
• Related Research
• Research Process
• Analysis Results
• Process Improvement Propositions
• Conclusion

3
Introduction

• The objective of this study was to understand how
  software testing is conducted by observation.
• From observations propose improvement to the
  testing process.
• Improvements by reducing development and testing
  costs, and improving quality.

4
Software Costs and Quality

• Software Engineering strives to reduce
  development costs and improving quality.
• Software Process Improvements (SPI) are the means
  to reaching these goals.
• Commitment to SPIs by all from all
  organizational levels is key to success
• Quality can be tested into products or developed
  and built into products

5
Software Costs and Quality Cont.

• External events such as deadlines affect software
  quality.
• The cost of software testing is high, therefore
  SPIs are necessary to reduce cost.

6
Related Research

• Involvement of testing during development occurs
  when testers develop test for developers to
  analyze
• The complexity of testing increases as a function
  of the complexity of the systems under testing.
• Testing strategy defines the contents of testing.

7
Related Research Cont.

• Communication and interaction between development
  and testing processes requires cooperation and
  coordination.
• The use of software components are increasing
  rapidly
• Design outsourcing and distributed development
  increase the use of components.
• Cost of Quality is Free, but being late with
  products may be more costly than fixing faults.

8
Research Process

• This study consisted of Organizational Units (OU)
  which develop and test technical software for
  automation or telecommunication in Finland.
• Initial Sample included 26 OUs, from which 5
  were used as case studies.
• Cases were chosen to show polar types

9
Research Process Cont.

• Data for the research was collected by a series
  interviews.
• Each interview had a different theme in mind and
  possibly a different interviewee in mind.
• The interviews took place during five rounds,
  based on the theme.

10
Research Process Cont.

• Interview Rounds
• Development and Testing Managers were asked to
  define their testing process .
• Managers of Testing were asked to define their
  testing process in depth.
• Testers were interviewed.
• Systems Analysts interviewed.

11
Research Process Cont.
Case Breakdowns
12
Research Process Cont.

• Data Analysis
• Information gathered from these interviews were
  then categorized.
• Categories were then analyzed to see how they
  were connected.
• The categories were then used to identify factors
  which affected testing.

13
Analysis Results

• Description of Cases
• Case A - Developed and tested Manufacturing
  Execution Systems
• Turnover 50 product, 50 service
• Services included systems integration and
  customization
• Testing against requirements was a challenge
  because customers had special in-house
  requirements standards
• Developers and testers worked physically close to
  each other
• Time allocated to testing was consistent,
  although over time it has been reduced
• Use of components low, hinders testing

14
Analysis Results Cont.

• Case B - Tested in house products and provided
  testing services for external customers
• Turnover 75 service, 25 product
• Majority of requirements specifications were
  based on standards.
• Delays in development allowed for fewer time
  allocated for testing
• Communication flexible, developers talked
  face-to-face with testers
• Use of components high, testability of components
  must be considered

15
Analysis Results Cont.

• Case C Customized Software Development
• Turnover 2/3 service, 1/3 product
• Testers were involved early, involved in
  development process
• Testers often had issues due to lack of
  advisement from developers
• Delays in development does not often result in
  reduction of testing time
• Developers and testers communicate face-to-face
• Use of components low
• Testing of software components seen as difficult
  b/c of different implementation.

16
Analysis Results Cont.

• Case D - Electronics
• Turnover 100 product
• High product orientation required high quality
  because recalls are very expensive
• Avoided testing of unfinished product
• Testing tasks clear and well documented
• Testing involved in development late but planning
  of testing automation provided information on
  upcoming tests
• Communication between developers and tester
  planned, formal and transparent
• Use of components high
• Components tested initially by suppliers then
  again at system testing.

17
Analysis Results Cont.

• Case E Software Testing Services
• Turnover 100 service
• Working as an external testing organization
  required the adaptation of the process of the
  customer
• Early involvement of testing was necessary for
  the testing company to increase the testability
  of the software
• Sometimes testing was involved late
• Budgets for testing affected testing time
• Communication was handle through a contact
  person, but was active and clear
• Use of components depends on customer
• As an external testing organization it was hard
  to receive information about customers purchased
  components.

18
Analysis Results Cont.
19
Analysis Results Cont.

• Cause and effect

20
Analysis Result Cont.

• Cause and Effect
• Business Orientation
• Directly associated with use of components and
  testing schedules
• Business Model value adding process
• Purely Service Oriented
• System integration
• Customizing
• Consulting
• Customers directly affect development and testing
  process
• Purely Product Oriented
• Product Development
• Marketing
• Customers do not directly affect development and
  testing process

21
Analysis Results Cont.

• Process Improvement Propositions
• Testing ought to be adjusted to business
  orientation
• Product oriented should adopt formal planned
  testing process
• Service oriented should adopt a flexible testing
  process
• Enhanced Testability of software
• Consider testability when selecting components
• Review testing process of suppliers

22
Analysis Results Cont.

• Effective Communication and interaction between
  development and testing
• Early involvement of testing and planning of
  testing
• Use of risk based testing

23
Conclusion

• Proposals by observing best practices using
  grounded theory
• Better documentation improved testability of
  software and components
• Efficient communication between development and
  testing improved quality
• When time is a major issue, risk based testing is
  the best solution
• Business orientation affects
• Use of components
• Amount and quality of communication
• Allocated testing time and the planning of testing

24
Questions?