CSC407 Web Testing

1
CSC407Web Testing
Greg Wilson gvwilson_at_cs.utoronto.ca Fall 2006
2
Testing

• Remember testing?
• Its ironic distributed applications are harder
  to debug, but we test them less
• Standard web applications are not harder to test
  than other programs
• Provided you design them for testability in the
  first place
• The things you do to make them testable are
  things you should be doing anyway

3
Legacy Code

• Legacy code (n.) source code that relates to a
  no-longer supported computer system
• Often used as an insult, meaning a mess youd
  wish on your worst enemy
• Feathers definition legacy code is code that
  doesnt have unit tests
• Without comprehensive unit tests, you can never
  be sure that your changes havent broken something

4
The Problem

• How do you test this?

PostgreSQL
HTTP request
Apache
Python CGI
Filesystem
HTML
5
Naïve Strategy Input and Output
PostgreSQL
HTTP request
Apache
Python CGI
Filesystem
HTML

• Accept the system as given
• Send HTTP, parse HTML
• Faithful
• Lots of work
• And remember, screen scraping is fragile

6
Record and Playback

• Do regression testing to make sure that we
  havent broken anything that used to work
• So
• Record a working system (HTTP in, text out)
• Replay the HTTP, and diff the result
• WinRunner is the gold standard for desktop apps
• MaxQ (maxq.tigris.org)
• Allows non-programmers to create and run tests
• But you have to have a nearly-working system
• and its still screen-scraping

7
How About Changing the Problem?

• Dont look at HTML directly
• Instead, insert markers and look for those
• Much less likely to change over time
• ltpgt User ID
• ltem classtest roleuidgtGVWlt/emgt
• lt/pgt
• Going to use CSS to style the page anyway
• Not checking everything
• but giving yourself more time to check the
  things that require human eyeballs

8
Changing the Problem
PostgreSQL
HTTP request
Apache
Python CGI
Filesystem
XML
9
Faking the Server

• How about getting rid of the web server?
• Run the CGI ourselves
• Tell it to produce XML (or plain text) rather
  than HTML
• Makes debugging easier
• But testing even less faithful

PostgreSQL
FakeCGI
Python CGI
Test Code
Filesystem
10
Faking the Server

• "All" we need is our own cgi.FieldStorage
• And os.environ
• And sys.stdin and sys.stdout
• Design for testability
• Code to interfaces, not implementations
• if testing
• import test.FieldStorage as FieldStorage
• import test.environ as environ
• else
• import cgi.FieldStorage as FieldStorage
• import os.environ as environ

11
More Practical Issues

• Database is persistent storage
• Thats its whole purpose
• But tests are supposed to be independent
• In theory, create a new database for each
• In practice, just wipe and re-fill the tables
• If tests are slow, programmers wont run them

12
Practical Issues

• If tests are slow covers a lot of sins
• How long does it take developers to set up?
• Install the software the first time?
• Reinstall/reconfigure for testing?
• Will it interrupt real service?
• Will it impact other developers?
• SelectAccess could take two full days to install
  and configure
• Which was eight days less than the competition

13
Practical Issues

• Solution change what youre testing
• SQLite and HSQLDB are much smaller than
  PostgreSQL
• And both can store database in memory
• Useless for real work, but great for testing
• Design for testing
• Abstract away the details of external packages
• A good idea anyway
• Run developer tests using replacements
• Still do final tests with real system, of course

14
Mock Objects

• If no replacement is available, build a mock
  object
• Behaves like a limited version of the real thing
• E.g., when asked for contents of a file, hands
  back the path, reversed
• Only works if you have a clean interface between
  your application and the subsystem
• But you should anyway

15
A More Testable System
SQLite-M
Test Code
FakeCGI
Python CGI
MockFS

• Much simpler to (re)install and (re)initialize
• And much less likely to mess up production system
• As faithful as you make it
• Weeding out the easy problems gives developers
  more time to look at the hard ones

16
Putting It All Together

• Goal is to produce something that can
• Be run from the command line, so that its output
  can be compared to saved output (testing via
  Makefile)
• Be called inside unittest (Pythons equivalent of
  JUnit) as part of a larger test suite
• Would like to avoid creating temporary files
• May need to pass command-line parameters to the
  thing being tested

17
A Week of Hard Work

• can sometimes save 5 minutes of Google
• HttpUnit
• Java library that acts like a browser
• Handles cookies, redirects, etc.
• HtmlUnit
• Models the returned page, rather than the HTTP
  transaction
• Compatible with JUnit
• There are lots of others
• For example, check out Selenium

18
Testing Summary

• When testing a large system, ask
• Where and how will I inject my tests?
• When and how will I check the results?
• Every interface is a potential cut point
• I/O, class creation, method calls
• The more you replace
• The easier testing is
• But the less faithful it becomes
• An economic decision

19
Exercise 2

• Design testability into your group signup system
• Well write unit tests is not enough
• What extra classes will be created (if any)?
• How will they be configured into the system?
• Etc.