DataDriven Automation Testing

1

• Data-Driven Automation Testing

• - An Overview

• Pavan Prakash

• Senior QA

• Butler International

2
Table of contents

• Cost-Effective Automated Testing
• Record/Playback Myth
• Viable Automated Testing Methodologies
• Managing Resistance to change
• Staffing Requirements
• Conclusion

3
Cost-Effective Automated Testing

• Automation testing more expensive and it doesnt
  replace the complete manual testing. Instead, it
  reduces the effort of manual testing by
  automating the repeatable test cases.
• It doesnt make sense in automating some test
  cases. Over complex tests are more trouble than
  they are worth to automate. Concentrate mainly on
  the straight forward test cases and leave the
  complex cases to manual testing.
• Automate only the repeated test cases. One time
  running test cases are not worth automating.
  Besides, it is useless to automate one time
  running test cases. Which will be of no use in
  the future.
• Avoid always record/playback approach of
  automation. It consumes most of time and cost to
  maintain the scripts. Most of the concerns who
  started with record/playback methodology of
  scripts closed automation and started continuing
  manual testing.

• contd...

4

• Adopt Data-Driven approach for testing. In this
  methodology, only inputs and expected results
  needs to be updated mostly in maintenance. Its
  ease in maintaining the automation scripts.

5
Record/Playback Myth

• The script resulting from this method contains
  hard-coded values.
• The cost of maintaining the scripts are
  astronomical and unacceptable.
• The scripts are not reliable and robust enough.
  They may fail even if the application has not
  been changed, in the replay.
• Its time consuming process. For ex. If a tester
  enters wrong data, he needs to re-record again.
• The scripts must be re-recorded even if the
  application changes.
• While recording test cases, if the application
  has a bug, the tester could not continue
  recording until the bug is fixed. Then why we
  need automation testing for ?
• Note Normally after 2 to 3 months of this
  nonsense, the testers put these scripts in
  recycle bin and get backs to manual testing again.

6
Viable Automated Testing Methodologies

• There are two best automation testing
  methodologies. They are
• Functional Decomposition Method
• Test Plan / Key-Word Driven Method
• Functional Decomposition Method
• The main concept behind the "Functional
  Decomposition" script development methodology is
  to reduce all test cases to their most
  fundamental tasks, and write User-Defined
  Functions, Business Function Scripts, and
  "Sub-routine" or "Utility" Scripts which perform
  these tasks independently of one another. In
  general, these fundamental areas include
• Navigation (e.g.
  Navigating from Login page")
• Specific Business Function (e.g. Submit an order
  or Payment")
• Data Verification (e.g. "Verify
  Payment Updates Current Balance")
• Return Navigation (e.g. Logout")

• contd...

7

• In order to achieve this, we need to separate
  data from function. A hierarchical is employed,
  using a structured or modular design.
• Driver Scripts Perform Initialization (if req.),
  then calls the test case in desired order.
• Test case Scripts Perform the application test
  case logic using business functions.
• Business Functions Perform the business
  functions with the application.
• Subroutine scripts Perform application specific
  tasks, which in turn called from 2 or more
  business function scripts.
• User-Defined functions These can be called as
  Utility functions which in turn is general and is
  not functionality related.
• The "Business Function" and "Subroutine" scripts
  invoke "User Defined Functions" to perform
  navigation. The "Test Case" script would call
  these two scripts, and the Driver Script would
  call this "Test Case" script some number of times
  required to perform all the required Test Cases
  of this kind. In each case, the only thing that
  changes are the data contained in the files that
  are read and processed by the "Business Function"
  and "Subroutine" scripts.

• contd...

8

• Advantages
• Utilizing a modular design, and using files or
  records to both input and verify data, reduces
  redundancy and duplication of effort in creating
  automated test scripts.
• Scripts may be developed while application
  development is still in progress. If
  functionality changes, only the specific
  "Business Function" script needs to be updated.
• Since scripts are written to perform and test
  individual Business Functions, they can easily be
  combined in a "higher level" test script in order
  to accommodate complex test scenarios.
• Data input/output and expected results is stored
  as easily maintainable text records. The users
  expected results are used for verification, which
  is a requirement for System Testing.
• Functions return "TRUE" or "FALSE" values to the
  calling script, rather than aborting, allowing
  for more effective error handling, and increasing
  the robustness of the test scripts. This, along
  with a well-designed recovery routine, enables
  unattended execution of test scripts.

9
Keyword/Test Plan Driven Method This method
uses the actual Test Case document developed by
the tester using a spreadsheet containing special
"Key-Words".  This method preserves most of the
advantages of the "Functional Decomposition"
method, while eliminating most of the
disadvantages.  In this method, the entire
process is data-driven, including functionality.
10

• How to Work With this
• The above table can be created as a template in a
  excel sheet.
• A controller script can be written to fetch the
  data as a list. This can be passed to an utility
  program and where in it identifies the keyword
  and commands the business functions to perform
  the intended action.
• The data can be given as comma-delimiter or else
  tab-delimiter as specific to the verification in
  the project.
• The controller script should be written such that
  it fetches all the required data until the
  end-of-file (EOF) is reached.
• Architecture
• The architecture seems to be same for both
  Functional Decomposition and as well as Test
  Plan/Key-Driven methods.
• Driver Script It calls the application specific
  controller script and passes it to the test
  cases.
• Test case scripts Reads and executes the data
  from the driver scripts and also calls utility
  scripts based on the keywords from Controller
  scripts.
• Utility Scripts Performs actions based on the
  parameters from Controller scripts by calling
  user-defined functions and returns back the
  control to controller scripts.

11

• User-Defined functions It contains general or
  application-specific functionalities. It can be
  called from any script to perform specific
  actions or tasks.
• Preparation is the key
• Preparation have to made by the organization
  before initializing automation process. If
  preparation has not been made adequately, then
  the ramp-up time for developing scripts will
  increase dramatically.
• An adequate test-environment must exists that
  accurately replicates the production environment.
  It can be a small-scale replica. But it should
  contain all the hardware, database and operating
  systems that development environment has.
• The test environments database must be restored
  to a known baseline, otherwise the tests
  performed against the database could not be
  repeated, since it is altered.
• Dedicated high speed machines should be set up to
  run the scripts. Its vague to run the scripts in
  the testers machine parallel to developing
  scripts.
• Detailed test cases should be converted into
  automation format. The tool cannot think and we
  should script everything to tell what to do.

12
Managing Resistance to Change

• The main fall of automation in lots of
  organizations is mainly because of the wrong
  choice in choosing automation tool for the
  applications. The main downfall happens is that
  the management chooses the tool without
  consulting the testers, who is responsible for
  automation and who have to cope up all these.
• There is a myth in between almost all testers
  that Tool replaces human. The testers should
  approach a testing tool such like another tool
  that makes their testing in a more efficient way
  by means of
• Relaxing them from being executing the same
  boring kind of test cases.
• It helps to save time by making them to create
  value added activities for the testing projects.
• The common pitfalls in introducing a new
  automation tool in the manual testing projects
  are
• It will take so much of the time for all testers
  to learn to use the tool.
• The tool may be difficult for the testers to use.

13

• This pitfalls can be removed by hiring a Test
  Tool Specialist or else the tool vendors will
  also offer some free Training Programs for
  learning more about the tool. A senior-level
  programmer who can learn easily the tool or else
  an expert in that particular tool can reduce
  those pitfalls.
• Mentoring the team with the above mentioned
  specialists will increase the efficiency in
  automating approach.

14
Staffing Requirements

• Testing tools normally have a standard approach
  for programming. The tester who uses automation
  testing tool is nothing but a programmer.
  Programming skills in any language with a good
  approach is needed for a good automation tester.
• He/She should have Logical, technical and as well
  as analytical skills. If these skills are there
  for an automation tester, they can master any
  tool.
• The automation tester is responsible for
• Developing standards and procedures for
  automation scripts/functions.
• Developing change management procedures for
  automation scripts/functions implementation.
• Developing the details and infra-structure for a
  data-driven method.
• Testing, managing and implementing the scripts
  done by testers.
• Running the automated scripts and provides
  results in the form of reports to the
  organization.

15
Conclusion

• Its a better way to switch rather than following
  the traditional functional approach.
• Select a tool that will allow you to implement
  all logics what you can implement with
  Data-Driven approach and as well as it can
  support long-term testing strategies.
• Before getting a tool for the project, make sure
  that the vendor can provide you support and as
  well as training.
• Its the right way to work as a team in planning
  together and selecting a right tool for the
  application.
• Lets see how to select a good tool for the
  application under test in upcoming presentations.