Automated Web Page Testing with Selenium IDE:

1
(No Transcript)
2
Starting from Scratch with RC, Python, and
Pysaunter

• Mary Ann May-Pumphrey
• Adobe EchoSign QA
• 02/28/12

3
From Scratch

• gt 1000 manual test cases in TestLink
• No Selenium automation
• Very little Selenium knowledge amongst the rest
  of the QA team members
• gt 20 of my time getting assigned to help execute
  manual tests

4
Why RC?

• Selenium-RC had several years worth of bug fixes
  when EchoSign decision was made in 06/11.
  Selenium-WebDriver was just about to be released
  for the first time.
• I had experience with RC, but not WebDriver, and
  producing tests asap was a priority.

5
Why Python?

• Selenium-1 supported C, Java, Perl, PHP, Python,
  Ruby.
• Selenium-2 quietly dropped Perl PHP.
• Java Python seem best-suited for long-term
  survival.

6
Why Python?

• Interpreted languages are simpler to write
  scripts in.
• Python is easier for manual testers to learn than
  Java, and we want our manual testers to do at
  least some automation.
• An IDE is less critical for Python scripts can
  be easily written debugged with a terminal/cmd
  window.

7
Why Pysaunter?

• Priority was on getting tests developed asap,
  rather than designing our own customized
  framework.
• Pysaunter implements Page Objects.

8
Why Pysaunter?

• Pysaunters creator--Adam Goucher--is a
  widely-known and respected Selenium contributor
  consultant.
• Framework supports both RC and WebDriver tests,
  and provides examples of both.

9
Page Object Model

• A test case comprises business logic,
  understandable by anyone who reads it.
• A page object file comprises locators, assertion
  methods, action methods, strings, etc., that are
  specific to a single page (or even part of a
  single page).

10
Page Object Model

• Selenium API calls should only appear in the page
  object files, not in the test scripts themselves.
• If everything is done correctly, in theory, the
  entire test suite could be switched to something
  other than Selenium without any modifications to
  the test scripts. (!?!)

11
pytest Flags

• Every EchoSign test method has a decorator
  containing pytest flags, ordered from most
  general to most specific, e.g.,
• _at_pytest.marks('regression','search','new_search',
• 'new_search_for_company_name',
• 'EA-627'))
• def test_new_search_for_company_name(self)

12
Executing the Tests

• To run just the smoke tests
• pysaunter.py m smoke v
• To run all the tests
• pysaunter.py m smoke or regression v

13
Structure of Test Suite

• conf
• conftest.py
• files
• logs
• modules
• pytest.ini
• scripts
• support

14
Structure of Test Suitescripts modules/pages
dirs

• Each file of test scripts, e.g.,
• scripts/AddToDocument.py,
• scripts/Upgrade.py, etc.
• Has a corresponding PO file
• modules/pages/AddToDocumentPage.py,
• modules/pages/UpgradePage.py, etc.

15
Example Test in scripts/Registration.py

• _at_pytest.marks('regression','negative','registratio
  n','register_without_any_input_values','EA-594')
• def test_register_without_any_input_values(self)
• r RegistrationPage()
• r.open_default_url()
• r.do_register_without_input_values()
• for msg in r.missing_messages.keys()
• self.assert_(r.is_missing_message_present(
  msg),"No or wrong error message present for "
  msg)

16
Relevant PO Code modules/pages/RegistrationPage.
py

• missing_messages
• "missing_email" "Please enter your email
  address",
• ...
• "missing_country" "Please specify your
  country",
• def do_register_without_input_values(self)
• self.se.click(locators'submit_button')
• def is_missing_message_present(self,key)
• return self.se.is_text_present(self.missing_me
  ssageskey)

17
Structure of Test Suitemodules/py

• Non-PO files
• imap.py, retrieve.py, extract.py all deal with
  fetching email
• helpers.py contains very general functions

18
Structure of Test Suitefiles dir

• EchoSign-specific, not part of Py.saunter
• Contains various files to be uploaded, archived,
  added to library, etc. by the test cases.

19
Structure of Test Suitelogs dir

• latest.xml dated .xml files contain execution
  results
• One dir/test with snapshots taken during run
• Demo

20
Integration with TestLink

• TestLink Keyword manual vs. automated allows
  manual testers to run queries to retrieve just
  manual tests for regression.
• Final pytest mark is always the TestLink id of
  corresponding manual test, i.e.,
• _at_pytest.marks('smoke','registration',
• 'register_free_account','EA-583')

21
Integration with Jenkins

• CI test runs on our internal build system are in
  our near-term plans.
• In the meantime, two artifacts
• echosign_selenium.tar.gz makes it very easy to
  install the latest checked-in version of test
  suiteno P4 client needed.
• list.html shows entire list of auomated tests to
  date.

22
Integration with FogBugz

• Any bug found by Selenium automation has two
  features
• Name of automated test somewhere in the bug
  report
• automation tag for metrics purposes

23
Best PracticesCoding Standards

• Adhering to very basic items from the Style Guide
  for Python Code
• Function variable names lowercase,
  underscores_for_readability
• Class names should be CapitalizeEachWord
• Indentation should be 4 spaces per levelno tabs

24
Best Practices skipif()

• skipif() decorators, for tests that only work in
  certain environments
• _at_pytest.mark.skipif('helpers.get_env()
  "preview"')
• _at_pytest.mark.skipif('helpers.get_env() !
  "preview"')

25
Best Practices skip()

• skip() decorators, for tests that arent quite
  finished, or have been found to have a problem
• _at_pytest.mark.skip()

26
Best Practices xfail()

• xfail() decorators, for tests that are expected
  to fail
• _at_pytest.mark.xfail(reason"FogBugz 13864")

27
Best Practices Error Handling

• Provide an error message for every assertion in
  case it fails.
• Put all these error messages in a dictionary at
  top of class.
• Use self-documenting dictionary keys.

28
Best Practices Error Handling

• error_strings
• "is_archived_count_not" "The Archived count
  is not ",
• .
• .
• . "manage_page_not_displaying_only
  _archived_folder" "The Manage page is not
  displaying the Archived folder OR is showing more
  folders also.,

29
Best Practices Error-Handling

• self.assert_
• (
• m.is_only_folder_displayed("widget"),
• self.error_strings'manage_page_not_displaying_onl
  y_widget_folder'
• )

30
Best Practices Code Reviews

• Code reviews of test code required for check-in

31
Best Practices Testing the Test

• Temporarily put a pdb.set_trace() just before the
  assertion in a new/passing test.
• Run the test.
• When the pdb prompt appears, delete a file or an
  email or do ltwhatevergt will best simulate a
  failure situation.
• Ensure that the test fails.

32
Best Practices Important Tests vs. More Tests

• Try to balance development of smoke tests
  (important but hard) with development of
  regression tests (less important but easier).

33
Whats Worked Well

• Pysaunter creator Adam Goucher has been very
  supportive, both in terms of bug fixes feature
  requests.
• The Python RC API has presented virtually no
  issues.

34
Whats Worked Well

• Pysaunters image-capturing feature is very
  useful. Much faster than video and usually just
  as effective.
• Pysaunter is Just Enough Framework for our
  current needs.
• Non-Selenium colleagues have been able to add new
  tests soon after getting started.

35
Concerns

• Lack of a Pysaunter community gt no peer support
  or best-practice discussion
• Pysaunters future unclear
• RCs longevity unclear, as is LOE for migration
  to WebDriver
• Selenium/Python job-seekers seem fewer in number
  than Selenium/Java job-seekers

36
Resources

• Instructions on how to install RC/Python/Pysaunter
  from my blog
• PushToTest-Sponsored Webinar Selenium Python
  Page Objects
• The Quick Python Book, 2nd ed.

37
Resources

• PushToTest-Sponsored Webinar Selenium--You Are
  Doing It Wrong
• Python Module of the Week
• The Element-34 Blog

38
QA?

• Now!
• or
• maryann_at_adobe.com