JUnit

1
JUnit

• A unit test framework for Java
• Authors Erich Gamma, Kent Beck
• Objective
• If tests are simple to create and execute, then
  programmers will be more inclined to create and
  execute tests.
• Web site junit.org

2
Introduction

• What do we need to do automated testing?
• Test script
• Actions to send to system under test (SUT).
• Responses expected from SUT.
• How to determine whether a test was successful or
  not?
• Test execution system
• Mechanism to read test scripts, and connect test
  case to SUT.
• Keeps track of test results.

3
Test case verdicts

• A verdict is the declared result of executing a
  single test.
• Pass the test case achieved its intended
  purpose, and the software under test performed as
  expected.
• Fail the test case achieved its intended
  purpose, but the software under test did not
  perform as expected.
• Error the test case did not achieve its
  intended purpose.
• Potential reasons
• An unexpected event occurred during the test
  case.
• The test case could not be set up properly

4
A note on JUnit versions...

• The current version is 4.4, available from July.
  2007
• To use JUnit 4.x, you must use Java version 5 or
  6
• JUnit 4, introduced April 2006, is a significant
  (i.e. not compatible) change from prior versions.
• JUnit 4 is used in this presentation.
• Much of the JUnit documentation and examples
  currently available are for JUnit 3, which is
  somewhat different.
• JUnit 3 can be used with earlier versions of Java
  (1.4 and earlier).
• Eclipse and NetBeans give the option of using
  JUnit 3.8.1 or JUnit 4.x, which are packaged in
  separate libraries.

5
What is a JUnit Test?

• A test script is just a collection of Java
  methods.
• General idea is to create a few Java objects, do
  something interesting with them, and then
  determine if the objects have the correct
  properties.
• What is added? Assertions.
• A package of methods that checks for various
  properties
• equality of objects
• identical object references
• null / non-null object references
• equality of arrays
• The assertions are used to determine the test
  case verdict.

6
When is JUnit appropriate?

• As the name implies
• for unit testing of small amounts of code
• On its own, it is not intended for complex
  testing, system testing, etc.
• Many tools have built on JUnit to extend its use
  to these contexts.
• In the test-driven development methodology, a
  JUnit test should be written first (before any
  code), and executed.
• Then, implementation code should be written that
  would be the minimum code required to get the
  test to pass and no extra functionality.
• Once the code is written, re-execute the test and
  it should pass.
• Every time new code is added, re-execute all
  tests again to be sure nothing gets broken.

7
A JUnit 4 Test Case

• / Test of setName() method, of class Value
  /
• _at_Test
• public void createAndSetName()
• Value v1 new Value( )
• v1.setName( "Y" )
• String expected "Y"
• String actual v1.getName( )
• Assert.assertEquals( expected, actual )

8
A JUnit 4 Test Case

• / Test of setName() method, of class Value
  /
• _at_Test
• public void createAndSetName()
• Value v1 new Value( )
• v1.setName( "Y" )
• String expected "Y"
• String actual v1.getName( )
• Assert.assertEquals( expected, actual )

Identifies this Java method as a test case, for
the test runner
9
A JUnit 4 Test Case

• / Test of setName() method, of class Value
  /
• _at_Test
• public void createAndSetName()
• Value v1 new Value( )
• v1.setName( "Y" )
• String expected "Y"
• String actual v1.getName( )
• Assert.assertEquals( expected, actual )

Objective confirm that setName saves the
specified name in the Value object
10
A JUnit 4 Test Case

• / Test of setName() method, of class Value
  /
• _at_Test
• public void createAndSetName()
• Value v1 new Value( )
• v1.setName( "Y" )
• String expected "Y"
• String actual v1.getName( )
• Assert.assertEquals( expected, actual )

Check to see that the Value object really did
store the name
11
A JUnit 4 Test Case

• / Test of setName() method, of class Value
  /
• _at_Test
• public void createAndSetName()
• Value v1 new Value( )
• v1.setName( "Y" )
• String expected "Y"
• String actual v1.getName( )
• Assert.assertEquals( expected, actual )

We want expected and actual to be equal. If they
arent, then the test case should fail.
12
Assertions

• Assertions are defined in the JUnit class Assert
• If an assertion is true, the method continues
  executing.
• If any assertion is false, the method stops
  executing at that point, and the result for the
  test case will be fail.
• If any other exception is thrown during the
  method, the result for the test case will be
  error.
• If no assertions were violated for the entire
  method, the test case will pass.
• All assertion methods are static methods in the
  class org.junit.Assert

13
Assertion methods (1)

• Boolean conditions are true or false
• assertTrue(condition)
• assertFalse(condition)
• Objects are null or non-null
• assertNull(object)
• assertNotNull(object)
• Objects are identical (i.e. two references to the
  same object), or not identical.
• assertSame(expected, actual)
• true if expected actual
• assertNotSame(expected, actual)

14
Assertion methods (2)

• Equality of objects
• assertEquals(expected, actual)
• Valid if expected.equals( actual )
• Equality of arrays
• assertArrayEquals(expected, actual)
• The parameters are either arrays of primitive
  types, or Object.
• There is also an unconditional failure assertion
  fail() that always results in a fail verdict.

15
Assertion method parameters

• In any assertion method with two parameters, the
  first parameter is the expected value, and the
  second parameter should be the actual value.
• This does not affect the comparison, but this
  ordering is assumed for creating the failure
  message to the user.
• Any assertion method can have an additional
  String parameter as the first parameter. The
  string will be included in the failure message if
  the assertion fails.
• Examples
• fail( message )
• assertEquals( message, expected, actual)

16
Equality assertions Objects

• For general objects
• assertEquals(a,b) relies on the equals() method
  of the class under test.
• The effect is to evaluate a.equals( b ).
• It is up to the class under test to determine a
  suitable equality relation. JUnit uses whatever
  is available.
• Any class under test that does not override the
  equals() method from class Object will get the
  default equals() behaviour that is, object
  identity .

17
Equality assertions Primitive types

• If a and b are of a primitive type such as int,
  boolean, etc., then the following is done for
  assertEquals(a,b)
• JUnit3, and JUnit 4.4 a b is evaluated.
• JUnit 4.0 through 4.3 a and b are converted to
  their equivalent object type (Integer, Boolean,
  etc.), and then a.equals( b ) is evaluated.
• Using the object equivalents resulted in some
  undesirable side effects for numeric comparisons
  between values with different types, such as an
  int and a long. In JUnit 4.4, the direct
  equality comparison was restored.
• int a 1
• long b 1L
• assertEquals( a, b ) // should this be true?

18
Array Equality Assertions

• Two arrays can be compared for equality with
  assertArrayEquals( Object expected, Object
  actual).
• Equality for arrays means
• They have the same length
• Each element of the array is equal, which means
• For each valid value for i, check as appropriate
• assertEquals(expectedi,actuali)
• or if the elements are themselves arrays
• assertArrayEquals(expectedi,actuali)
• Two null arrays are considered equal.

19
Floating point assertions

• When comparing floating point types (double or
  float), there is an additional required parameter
  delta.
• The assertion evaluates
• Math.abs( expected actual ) lt delta
• to avoid problems with round-off errors with
  floating point comparisons.
• Example
• assertEquals( aDouble, anotherDouble, 0.0001 )

20
Organization of JUnit tests

• Each _at_Test method represents a single test case
  that can independently have a verdict (pass,
  error, fail).
• Methods with no annotation are not considered
  test cases, and can be used as helper methods.
• Normally, all the tests for one Java class are
  grouped together into a separate class.
• Naming convention
• Class to be tested Value
• Class containing tests ValueTest

21
Running JUnit Tests (1)

• The JUnit framework does not provide a graphical
  test runner. Instead, it provides an API that
  can be used by applications to run test cases and
  a textual runner than can be used from a command
  line.
• Eclipse and NetBeans each provide a graphical
  test runner that is integrated into their
  respective environments.

22
Running JUnit tests (2)

• With the runner provided by JUnit
• When a class is selected for execution, all the
  test case methods in the class will be run.
• The order in which the methods in the class are
  called (i.e. the order of test case execution)
  isnot predictable.
• Test runners provided by IDEs may allow the user
  to select particular methods, or to set the order
  of execution.
• It is good practice to write tests with are
  independent of execution order, and that are
  without dependencies on the state of any previous
  test(s).

23
Running JUnit tests (from Eclipse)
24
Test fixtures

• A test fixture is the context in which a test
  case runs.
• Typically, test fixtures include
• Objects or resources that are available for use
  by any test case.
• Activities required to make these objects
  available and/or resource allocation and
  de-allocation setup and teardown.

25
Setup and Teardown

• For a collection of tests for a particular class,
  there are often some repeated tasks that must be
  done prior to each test case.
• Examples create some interesting objects to
  work with, open a network connection, etc.
• Likewise, at the end of each test case, there may
  be repeated tasks to clean up after test
  execution.
• Ensures resources are released, test system is in
  known state for next test case, etc.
• Since a test case failure ends execution of a
  test method at that point, code to clean up
  cannot be at the end of the method.

26
Setup and Teardown

• Setup
• Use the _at_Before annotation on a method containing
  code to run before each test case.
• Teardown (regardless of the verdict)
• Use the _at_After annotation on a method containing
  code to run after each test case.
• These methods will run even if exceptions are
  thrown in the test case or an assertion fails.
• It is allowed to have any number of these
  annotations.
• All methods annotated with _at_Before will be run
  before each test case, but they may be run in any
  order.

27
Example Using a file as a text fixture

• public class OutputTest
• private File output
• _at_Before public void createOutputFile()
• output new File(...)
• _at_After public void deleteOutputFile()
• output.delete()
• _at_Test public void test1WithFile()
• // code for test case objective

28
Method execution order

• createOutputFile()
• test1WithFile()
• deleteOutputFile()
• createOutputFile()
• test2WithFile()
• deleteOutputFile()
• Assumption test1WithFile runs before
  test2WithFile which is not guaranteed.

29
Once-only setup

• It is also possible to run a method once only for
  the entire test class, before any of the tests
  are executed, and prior to any _at_Before method(s).
• Useful for starting servers, opening
  communications, etc. that are time-consuming to
  close and re-open for each test.
• Indicate with _at_BeforeClass annotation (can only
  be used on one method, which must be static)
• _at_BeforeClass public static void anyNameHere()
• // class setup code here

30
Once-only tear down

• A corresponding once-only cleanup method is also
  available. It is run after all test case methods
  in the class have been executed, and after any
  _at_After methods
• Useful for stopping servers, closing
  communication links, etc.
• Indicate with _at_AfterClass annotation (can only be
  used on one method, which must be static)
• _at_AfterClass public static void anyNameHere()
• // class cleanup code here

31
Exception testing (1)

• Add parameter to _at_Test annotation, indicating
  that a particular class of exception is expected
  to occur during the test.
• _at_Test(expectedExceptedTypeOfException.class)
• public void testException()
• exceptionCausingMethod()
• If no exception is thrown, or an unexpected
  exception occurs, the test will fail.
• That is, reaching the end of the method with no
  exception will cause a test case failure.
• Testing contents of the exception message, or
  limiting the scope of where the exception is
  expected requires using the approach on the next
  slide.

32
Exception testing (2)

• Catch exception, and use fail( ) if not thrown
• public void testException()
• try
• exceptionCausingMethod()
• // If this point is reached, the expected
• // exception was not thrown.
• Assert.fail("Exception should have
  occurred")
• catch ( ExceptedTypeOfException exc )
• String expected "A suitable error
  message"
• String actual exc.getMessage()
• Assert.assertEquals( expected, actual )

33
Timeouts

• The maximum time allowed for a test to run may be
  specified.
• Time units milliseconds
• _at_Test( timeout 100 )
• public void timeConsumingTest()
• // do the test
• If the test takes longer than 100 ms to execute,
  it will be terminated and reported as a failure.

34
Ignored Tests

• The _at_Ignore annotation is available to mark a
  test that should be skipped. The test will not
  pass or fail, but will be reported as ignored.
• _at_Ignore
• _at_Test
• public void testThatIsNotReadyYet()
• // this test wont be run

35
Parameterized Tests

• In many cases, the structure of a test is the
  same in terms of code and only the data
  changes.
• JUnit provides a facility to run one _at_Test method
  multiple times with different sets of data.
• Classes to be run in this manner need to use a
  different test runner.
• The _at_RunWith annotation is attached to the class
  header to identify a class to use as a test
  runner
• _at_RunWith( Parameterized.class )
• public class ParameterizedBitTest

36
Parameterized Tests

• Suppose that class BitTest contains JUnit test
  methods.
• A new instance of BitTest is created for each
  test normally the default constructor is used to
  create the BitTest objects.
• In a parameterized test class, a constructor with
  the set of parameters needed for each test run is
  needed.
• Instance variables can be set to keep the
  parameter values for the test method to use.
• To specify the set of parameters, the _at_Parameters
  annotation is used on a special method to return
  a collection of parameters
• Type CollectionltObjectgt (any implementation
  of this interface can be used)
• Each element of the collection is a new set of
  parameters
• The Object array contains the set of values for
  each test run
• Element 0 becomes the value passed to the first
  constructor parameter.

37
Parameterized Test Example (1)

• _at_RunWith( Parameterized.class )
• public class BitTest
• // Values to be used when test method is run
• private Bit firstBit
• private Bit secondBit
• private Bit expectedBit
• public BitTest( int first, int second, int
  expected )
• this.firstBit new Bit( first )
• this.secondBit new Bit( second )
• this.expectedBit new Bit( expected )

38
Parameterized Test Example (2)

• _at_Parameters
• public static ListltIntegergt data( )
• ListltIntegergt params new
  LinkedListltIntegergt( )
• params.add( new Integer 0, 0, 0 )
• params.add( new Integer 0, 1, 0 )
• params.add( new Integer 1, 0, 0 )
• params.add( new Integer 1, 1, 1 )
• return params
• _at_Test
• public void testAnd( )
• // firstBit, secondBit, and expectedBit
  already have values
• Bit actualBit firstBit.and( secondBit )
• Assert.assertEquals( expectedBit, actualBit
  )

39
Running a parameterized test
40
JUnit 3

• At this point, migration is still underway from
  JUnit 3 to JUnit 4
• Eclipse 3.3 and NetBeans 6.0 each have both
  versions available.
• The Eclipse Test and Performance Tools platform
  is not yet fully JUnit 4 compatible ?.
• Within the JUnit archive, the following packages
  are used so that the two versions can co-exist.
• JUnit 3 junit.framework.
• JUnit 4 org.junit.
• Be careful when importing JUnit classes that you
  get the correct package for the version that you
  want.

41
Whats different when using JUnit 3 (1)

• No _at_Test annotation is available
• Annotations were not available before Java 5
• Instead, tests are identified as follows
• The class containing tests must inherit from
  TestCase, a class supplied by JUnit.
• The name of the class containing tests must have
  Test as part of the name.
• Each method must begin with test and have no
  parameters (e.g.
• e.g. testCreateAndSetName()

42
Whats different when using JUnit 3 (2)

• Test fixtures implemented by overriding the
  empty methods inherited from TestCase
• Test setup override method setUp()
• Test cleanup override method tearDown()
• There are no equivalents to _at_BeforeClass or
  _at_AfterClass
• Test suites (deprecated in JUnit 4) A class
  could define a collection of classes representing
  a test suite, to provide a test organization
  facility.
• Some tools still require the use of test suites,
  so specialized test runners are provided to
• convert JUnit 4 test classes to a JUnit3
  compatible suite
• run a suite of test classes in JUnit 4

43
The same test case in JUnit 3

• / Test of setName() method, of class Value
  /
• class ValueTest extends TestCase
• public void testCreateAndSetName()
• Value v1 new Value( )
• v1.setName( "Y" )
• String expected "Y"
• String actual v1.getName( )
• Assert.assertEquals( expected, actual
  )

44
Under the hood

• In its original form, JUnit was conceived as a
  stand-alone tool, and had a very simple
  architecture.
• It is now viewed as a framework on which to build
  test execution into an IDE or other tools, and
  has been rewritten to be an event-driven tool to
  which listeners can register but is more
  complex to understand.
• The source code for JUnit can be obtained from
  junit.org.

45
Key points for the tool

• Locate and identify test cases.
• Manage the execution.
• Run a single test.
• Report results.

46
Finding the tests

• Extensive use is made of Javas reflection
  capabilities, which includes classes for
• Class
• Method
• Field (that is, attributes)
• Constructor
• Annotation

47
Class Class

• An object of type Class can be queried to
  discover
• its name
• the sets of constructors, methods, and fields
• any classes contained inside the class
• the class in which this class is enclosed (if
  applicable)
• interfaces implemented
• superclass
• enum constants
• package containing the class
• protection (public, private, etc.) for the class
• The newInstance() method can be used to create a
  new object of the class.

48
Class Method

• A Method object can be queried for
• its name
• protection (public, etc.)
• return type
• parameters, and their types
• annotations (such as _at_Test, _at_Before, etc.)
• declared thrown exceptions
• The method invoke( Object obj, Object a) can be
  called on a Method object, so that
• aMethod.invoke( obj, a ) is equivalent to
  obj.aMethod(a)

49
JUnits use of reflection

• JUnit can use the reflection information to
• Get annotations on the class that specify a test
  runner
• Find the methods in the class to check for
  _at_Test annotations
• Create instances of the test class at test
  execution time
• Run a method annotated with _at_Test, using invoke()

50
JUnits test execution process (1)

• For each test to run (that is, a Method object
  with an _at_Test annotation has been selected),
  JUnit will do the following.
• Notify listeners that a test is starting.
• Start a try-catch-finally block
• Create an instance of the test class (using
  newInstance() on the class object)
• A separate instance of the class will be created
  for every test to be run.
• Use invoke() to run each of the methods annotated
  with _at_Before
• continued next page...

51
JUnits test execution process (2)

• Use invoke() to run the selected _at_Test method
• Catch all exceptions to this point. If any have
  occurred, record their causes and notify
  listeners.
• In a finally block, use invoke() to run each of
  methods annotated with _at_After.
• Exceptions need to be caught here so that all
  _at_After methods get a chance to run.
• Notify listeners that the test has ended.

52
How assertions work

• JUnits assertion methods perform the comparison
  requested. If the assertion is true, the method
  returns. If the assertion is false, a specified
  exception is thrown.
• The set of exception classes thrown by assertion
  methods (AssertionError, ComparisonFailure,
  ArrayComparisonFailure) is known. If one of
  these exceptions is thrown, it is interpreted as
  a test case failure. Any other exception is
  interpreted as a test case error.

53
How assertions work

• Code for the assertEquals method for two objects
  (slightly modified for clarity)
• static public void assertEquals( Object expected,
  Object actual )
• if ( expected null actual null )
• return
• if (expected ! null expected.equals(
  actual ) )
• return
• // code to determine a failure message
• throw new AssertionError( message )