Verification and validation

1
Verification and validation

• Verification
• Are we building the system correctly?
• Eliminate bugs in the system
• Validation
• Are we building the correct system?
• Should already be handled by a thorough
  requirements analysis based on use cases

2
Testing

• Purpose of testing
• To find the as many bugs in the system as
  possible
• What is the difficulty?
• Developers are not keen to find their own bugs
• What the industry does?
• Use a team of testers whose main job is to find
  fault in the system
• typical tester developer ratio 1 3
• take up at least 30 of the development cost

3
Test levels

• Unit testing
• To test one and only one unit, usually a class
• Integration testing
• Usually use cases based
• Integration and unit tests can be done together
• System testing
• Testing the entire system, typically from an
  end-user view

4
Unit testing

• Specification test (black box test)
• Verify the units externally observable behavior
• Given certain input and a particular state, see
  if the output return is as expected
• Partition the range into equivalent classes to
  reduce the possible combinations

5
Equivalent partitioning

• An equivalent set is a set of conditions for
  which an object is supposed to behave similarly,
  so as to reduce the number of test cases
• Example
• partition the state of a bank account into
• empty, positive and negative balance
• partition the state of a stack into
• empty, half-full, full
• Partition the expected input value (0 to 100)
• To value at boundary, outside boundary and normal
• e.g. -10, -1, 0, 1, 40, 60, 99, 100, 110

6
Unit testing - structure test (white box test)

• Structure test
• every statement has to be executed at least once
• Test
• the most interesting paths
• the least-know paths
• The high risk paths

while ...
If altb
7
Integration testing

• To test different units working together
• Each time test only one use case
• The sequence diagram is a good source to specify
  the test case
• For a user case, we may have the following tests
• basic course tests
• alternative course tests
• tests of user documentation

8
Integration testing

• Example to test ATM withdraw use case
• Test case Withdraw 300 from Account 123456
• Input
• User selects Withdraw option, key in the password
  number (888) and withdraw 300 from account
  123456
• initial balance 1000
• Result
• Dispense 300
• Print the new balance 700 of account 123456
• Conditions
• no other use cases are allowed to access the
  accounts during this test case

9
Integration testing

• The test case using in the example can verify
  only one scenario
• a matrix format can be used to represent similar
  test cases that differ only in a single input or
  result
• test cases A test cases B
• Input /result input/result
• 300/700 200/2040
• 600/4000 . . .
• . . . . . .

10
System testing

• Test entire system as a whole
• several use cases are executed in parallel
• the test can be divided into
• operation tests
• full-scale tests
• negative tests
• tests based on the requirements specification
• tests of the user documentation

11
Testing techniques

• Regression test
• Run the test every time you change the system
• Main purpose is to verify old functionality still
  work
• Important test, must be automated (e.g. JUnit)
• Operation test
• Test the system in normal operation over a long
  period
• Use the system in the intended manner
• Only normal mistakes are made
• Measure mean-Time-To-Failure (MTTF)

12
Testing techniques

• Full-scale test
• run the system on its maximum scale
• System is used by many users
• Performance test or capacity test
• Measure the system performance under different
  loads
• Overload test
• Goes one step further than the full-scale test to
  see how the system behaves when it is overloaded
• Should not expect normal performance but at least
  the system should not go down and catastrophe not
  to occur

13
Testing techniques

• Negative tests
• Stress test that try to use the system in ways it
  is not designed for, so as to reveal system
  weaknesses
• e.g. incorrect network configuration,
  insufficient hardware capacity, impossible work
  load
• Tests based on requirements
• Tests based on requirement specifications
• Testing of the user documentation
• to check the consistency between manuals and
  system behavior

14
Testing techniques

• Ergonomic test
• Test the man-machine interface
• Is the interface consistent with the use cases?
• Are the menu logical and readable for
  non-computer professionals?
• Can users understand the failure messages?

15
Testing techniques

• Acceptance tests
• Alpha testing
• final check by the customers
• test under real environment for longer time than
  when the system was developed
• Beta testing
• Product is not targeted to general rather than
  specific customers
• Product is tested by specially selected customers

16
Testing techniques

• Installation tests
• Verify that the system can be installed on the
  customer platform and the system operates
  correctly
• Configuration tests
• Verify that the system works correctly in
  different configurations, such as different
  network configurations

17
C attribute

• A C attribute is something that is inside a
  square brackets, such as serializable
• Two ways of using the attribute
• Simple way
• Use the attribute as a tag (a label)
• The tag can be identified by reflection and can
  do useful work, e.g. testing
• Advance way
• Aspect programming, will be discussed

18
Attribute uses as a simple tag

• Serializable //this class can be serialized
• public class Dummy
• int x
• NonSerialized
• int tmp //no need to serialized this part
• The alternative is to add serializable code to
  the class, but this pollutes the class with
  non-core responsibility
• The serializable responsibility is factored
  (taken) out of the class

19
NUnit (an example of using attribute as tag)

• Main philosophy
• Prove the quality of your software by testing
• Relentless testing (XP)
• Test all methods using different scenario
• Repeat of all your test cases whenever you change
  your code
• Regression test
• How to do this efficiently?

20
NUnit

• Based on the simple testing framework JUnit
  proposed by Beck and Gamma
• The cornerstone of Extreme Programming (XP)
• Download NUnit V2.1 from www.nunit.org
• Study QuickStart.doc

21
Example of writing test cases

• The class to be tested by NUnit
• public class Account
• double balance0
• public void Deposit(double amount)
• balance amount
• public double Balance
• get return balance

22
JUnit by Beck and Gamma
Test Run()
JUnit Framework
TestSuite
TestCase
TestRunner testSuite.Run()

Your code
YourTestCase
23
Note

• TestRunner has-a TestSuite
• Your write the YourTestCases, add to TestSuite
• testSuite.Run() invokes all testcases
• class Test, TestSuite,
• and TestCase are Composite
• Test is a Command
• TestCase is a Template

Test Run()
TestCase Run() SetUp() RunTest()
TearDown()
YourTestCase
24
Test-driven development (TDD)

• Write the test cases before you write the code !
• Work on one test at a time
• Keep the test small
• TDD Golden Rule
• Never write code unless you have a test that
  requires it
• Ref http//www.parlezuml.com/tutorials/tdd_nunit/
  index_files/frame.htm

25
How to improve an already very good testing tool

• NUnit already makes regression testing very easy
• But still need to learn the NUnit framework
• The current version of NUnit is very easy to use
  owing to attribute programming

26
Example of writing test cases

• using NUnit.Framework
• TestFixture
• public class AccountTest
• Test //the test case
• public void Deposite()
• Account acc new Account()
• float balanceBefore acc.Balance
• acc.Deposit(10.0F)
• float balanceAfter acc.Balance
• Assert.AreEqual(10.0F, balanceAfter-balanceBefor
  e)

27
Assertions

• Assertions are central to unit testing
• NUnit provides a rich set of assertions as static
  methods of the Assert class
• Comparison test
• Assert.AreEqual( 1.0, sum)
• Condition test
• Assert.IsTrue(bool condition)
• Assert.IsNull(object anObject)
• Utility methods
• Assert.Fail(string message)

28

• Other common attributes
• SetUp
• Method marked with SetUp will be run before
  every test
• TearDown
• Method marked with TearDown will be run after
  every test
• Compile the program
• Start NUnit GUI
• Select the .dll file
• Run

29
Reflection

• NUnit simply loads the assembly
• Find the attributes all classes/methods by the
  reflection mechanism, and runs the methods
• Use of Reflection
• Viewing metadata, perform type discovery
• Dynamic invocation
• to invoke properties and methods on objects
  dynamically instantiated based on type discovery.
  
• refhttp//www.ondotnet.com/pub/a/dotnet/excerpt/p
  rog_csharp_ch18/index.html?page1

30
Create your own attribute TestFixture

• //custom attribute, target includes class
  method
• AttributeUsage(AttributeTargets.Class
• AttributeTargets.Method AllowMultiple
  true)
• public class TestFixtureAttribute
  System.Attribute
• string str
• TestFixtureAttribute(string str)
  //constructor
• this.str str
• public string Message() //define a property
• get return str

31
Reflection and attribute

• Load the assembly
• Assembly a Assembly.Load(AssemblyName)
• GetTypes() return an array of Type objects
• Type types a.GetTypes( )
• Check whether the type has attribute
  TestFixture
• Object obj type.GetCustomAttributes()0
• if (obj is TestFixtureAttribute)
• //found the attribute instance, do something
• Console.WriteLine(0, obj.Message)

32
AOP (Aspect Oriented Programming)

• Limitation of OOP
• Principle of divide-and-conquer
• Decompose complex system into simpler units
• Each unit has a clear responsibility
• Question
• What if some responsibilities are not confined to
  any particular object but are instead scattered
  through out the system?

33
Classic example - logging facility

• How to trace the flow of operations?
• The traditional way
• public class Foo
• protected Logbook log
• public bar()
• log.enter(bar() entered)
• \\business logic of bar()
• log.enter(bar() quitted)

34
Classic example - logging facility

• The problems
• Messy, need to provide the code for every
  objects, bad code reuse
• Objects are overloaded with non-core
  responsibilities
• The problem
• Some responsibilities cannot be cleanly
  encapsulated in an object or method
• e.g. security, transaction, performance
  evaluation
• The solution
• Aspect-Oriented Programming (AOP)

35
Concerns

• Concern is a particular goal of a system
• A typical system has
• Core concerns
• E.g. processing payments in bank applications
• System-level concerns
• Logging, transaction, security, . . .
• System-level concerns tend to crosscut (share) by
  many modules
• Such concerns are known as crosscutting concerns

36

• OOP
• Each object should have clear responsibility
• Good at addressing core concerns
• BUT cant handle crosscutting concerns well
• Aspect-Oriented Programming (AOP)
• Solve the problem of crosscutting concerns by a
  pattern called Interception

37
AOP solution via C attributes

• Logbook
• public class Foo ContextBoundObject
• protected Logbook log
• public bar()
• log.enter(bar() entered)
• \\business logic of bar()
• log.enter(bar() quitted)

38
What is changed?

• Add an attribute Logbook
• This attribute is associated with a component
  that carries out the logging operation
• Foo is now a subclass of ContextBoundObject
• Foo now consists of its core business logic only,
  no more crosscutting concerns

39
What happened? The Interception pattern

• C compiler inserted a transparent proxy which
  intercepted all calls to Foo

Transparent Proxy
Message Sink
client
Foo
Logbook
Inserted by compiler
40
To handle more crosscutting concerns

• Logbook, Security
• public class Foo ContextBoundObject

Transparent Proxy
Message Sink A
Message Sink B
client
Foo
Logbook
Security
Inserted by compiler
41

• Logging, security, etc are aspects of a system
• Each aspect addresses a crosscutting concern
• By combining aspects together, one can build a
  highly configurable application rapidly
• Core concerns are addressed by objects
• Crosscutting concerns are addressed by aspects
• C
• Interception supported by compiler
• Java
• Tools like AspectJ inserts code to source file

42
AOP basics

• Identify the concerns in a system
• Separate the concerns into core and crosscutting
  (usually system-level) concerns
• Implement each concern separately
• e.g. concerns in a credit card application
• Business logic (core concern)
• Logging (crosscutting concern)
• Authentication (crosscutting concern)
• Integrate all the concerns by an aspect
  integrator (also known as aspect weaver)

43
The prism analogy
Security, persistence, transaction
Business logic
logging
Weaver
Concern Identifier
Requirements
system
Aspectual Decomposition
Aspectual Recomposition
44
Configure a new system using aspects
Logging
Business logic
Security
Transaction
Implementation modules
45
Noun, verb, and adjective

• If procedure is verb,
• and class is noun,
• then aspect is the adjectives that describes the
  noun
• Reference
• See Decouple Components by Injecting Custom
  Services into Your Objects Interception Chain
  (http//msdn.microsoft.com/msdnmag/issues/03/03/Co
  ntextsinNET)
• For overview, articles by Ramnivas
  (http//www.javaworld.com/javaworld/jw-01-2002/jw-
  0118-aspect.html)