Testing

1
Testing

• Verification Does the program conform to its
  specification?
• How do you verify a large program?
• Infinite Matrix of possibilities.
• Can check a program conforms given a single
  input.
• If have an input of all integers cant possibly
  test all possible inputs.

2
Example Fonts

• There are TrueType and Postscript fonts.
• Each font can be 20 different sizes.
• Each font can represent approximately 70
  characters
• There are hundreds of TrueType and Postscript
  fonts.
• Verifying the display characteristics of each
  font and size would take a tremendous amount of
  time

3
How can you Gain Confidence that Program meets
specification?

• Group similar objects together
• Test a few similar objects extensively
• Take a small sample of the rest

4
Fonts Solution

• Rigorously test 1 or 2 TrueType and 1 or 2
  Postscript fonts with all sizes
• Assume that every TrueType font is rather
  similar.
• Test a small sample like 10 letters at 1 font
  size for the others.

5
Suggestions for Assignment 4

• Skim, Do not read html specification
• Search the web quickly for information related to
  this topic. Dont replicate work when it already
  exists
• Answer the easy questions first
• Partition tags into related groups.
• Only test hot spots
• Do not forget the big picture. (You cannot afford
  to spend a lot of time on small details)

6
Tools for testing

• Expect
• tool primarily for automating interactive
  command line applications such as telnet, ftp,
  passwd, fsck, rlogin, tip, etc. Expect is also
  useful for testing this kind of application.
• Basic use consists of sending a request (just
  like a user would do) and expecting a response.
  
• Expect is very powerful and easy to use
• "Exploring Expect"(PubO'Reilly, ISBN
  1-56592-090-2)

7
Tools for testing II

• DejaGnu
• DejaGnu is a framework for testing other
  programs. Its purpose is to provide a single
  front end for all tests.
• The DejaGnu framework allows for a large number
  of small test cases to be written and
  incorporated into larger Test Suites.
• Handles statistics gathering on tests.
• Flexible enough to write tests for any type of
  program
• DejaGnu's output is designed to be parsed by
  other filtering script, and it is also human
  readable.

8
Testing GUIs

• Testing GUIs is difficult!!!
• Simulating random mouse clicks require
  specialized software
• Many different kinds of users for GUI systems
• Application should be able to survive
  unknowledgeable users
• A solution is to have a wrapper for every menu
  item/button and invoke the functions according to
  a typical usage scenario
• A good test is the Monkey test. How long can
  the program go without crashing with a Monkey
  hitting random buttons.

9
Scripting

• Tcl/Tk, Perl, Python, Lisp variants, JPython,
  TclBlend
• Scripting is an excellent tool for testing.
• Script languages are interpreted you can peak
  and poke with functions in real time
• Scripts are very fast to write and can flexibly
  exercise program functions
• Powerful regular expression packages make
  determining results of tests easier
• Loosely typed languages
• Great for testing in Embedded environments