Integration of Software Testing into the Development Process

1
Integration of Software Testing into the
Development Process

• Sriram Sankar
• Google, Inc.

2
Presentation Outline

• Warming up
• Developer testing with JUnit
• QA and release process
• Production monitoring
• Guice

3

• Another engineer checked in a change that broke
  some functionality you had built

4

• Huge growing code base very difficult to make
  significant changes without breaking other
  existing functionality

5

• Customer reports problem with software that was
  not present in earlier versions

6

• Unexpected production problem in web application

7

• Web application runs out of memory

8

• Other scenarios?

9
Our focus Web Applications

• Runs on company machines
• Usually runs on many machines
• More rapid release cycles
• Easy to watch/analyze running production servers
• Typically contains many moving parts (on
  different machines)

10
Developer Testing with JUnit
11
JUnit

• Unit testing framework although it can be used
  for other forms of testing also
• Permits tests to be written along with setup and
  teardown functions
• Regression tests can be run easily with clear
  indication of failures
• Proper methodologies need to be followed to
  develop useful test suites

12
Example Brokerage Accout

• public class Account public Account(int
  accountId) initialize database
  connections public void buy(String stk, int
  amt) update database public boolean
  sell(String stk, int amt) check balance,
  update database public int balance(String
  stk) get balance from database

13
Our Tests

• Buy shares
• Sell shares with adequate balance
• Sell shares without adequate balance

14
Tests Using JUnit

• import org.junit.import static
  org.junit.Assert.public class AccountTest
  Account acct _at_Before public void
  initializeAccount() acct new
  Account(1) acct.buy(GOOG, 20) acct.buy(YH
  OO, 50) acct.buy(MSFT, 10) _at_Test
  public void testBuy() acct.buy(GOOG,
  100) assertTrue(acct.balance(GOOG)
  120) _at_Test public void testSaleWithBalance()
  acct.sell(YHOO, 30) assertTrue(acct.balan
  ce(YHOO) 20) _at_Test public void
  testSaleWithoutBalance() acct.sell(MSFT,
  30) assertTrue(acct.balance(MSFT)
  10)

15
Running the Tests

• import org.junit.runner.class Main public
  static void main(String args)
  JUnitCore.main(AccountTest)
• Running this will print
• JUnit version 4.3.1Time 4.225OK (3 tests)

16
Problem

• The tests have to access the database
• Why is this a problem?
• What are other similar problems?

17
Solution Mocks

• We abstract the functionality we wish to simplify
  into an interface in this case the database
  access code.
• We provide a standard implementation that does
  access the database.
• And we provide a mock implementation that mimics
  the database adequately for the purpose of
  testing.

18
Abstracting Database Access

• public interface DbAccess String get(String
  key) void put(String key, String value)

19

• class Account private int accountId private
  DbAccess db public Account(int accountId)
  this.accountId accountId db new
  RealImpl() public void buy(String stock, int
  quantity) String key accountId ","
  stock db.put(key, "" (balance(stock)
  quantity)) public boolean sell(String stock,
  int quantity) String key accountId ","
  stock int bal balance(stock) if (bal lt
  quantity) return false else db.put(key, ""
  (bal - quantity)) return true public int
  balance(String stock) String key accountId
  "," stock String value db.get(key) if
  (value null) return 0 else return
  Integer.parseInt(value)

20
Actual DbAccess Implementation

• public class RealImpl implements DbAccess
  public RealImpl() initialize
  database public String get(String key)
  get data from database public void
  put(String key, String value) store data in
  database

21
Mock DbAccess Implementation

• public class MockImpl implements DbAccess
  private MapltString,Stringgt store new
  HashMapltString, Stringgt() public String
  get(String key) return store.get(key) pub
  lic void put(String key, String value)
  store.put(key, value)

22
To run the unit tests

• Replace
• db new RealImpl()
• With
• db new MockImpl()
• in Account.java
• This is still a problem!Well come back to this
  later.

23
Automating Running of Unit Tests

• Have a framework into which unit tests can be
  added so that all unit tests can be run from a
  single command.
• Have the ability to determine which change caused
  the failure when there is a failure.
• Have the ability of running only those tests that
  are relevant to a particular change.
• Other suggestions?

24
Other Best Practices

• Require at least one unit test for each new class
• Require a unit test for each new bug that fails
  in the presence of the bug
• Require that all tests pass before checking in a
  changelist
• Time set aside to write tests
• Organize tests into categories smoke,
  non-smoke, etc.
• Other suggestions?

25
Unit Testing vs. Functional Testing

• Strictly speaking, a unit test tests a single
  unit (such as a class)
• A functional (or system) test tests the entire
  application
• There are many intermediate possibilities such
  as testing at the process level (mocking out all
  network interactions)

26
What does unit testing buy us?
27
What does unit testing not buy us?
28
Developing Mocks

• Mocks can be as simple as noops in which case
  they can be used for integration testing
• At the other extreme, mocks can be very detailed
  and allows for full functional testing
• But building mocks can be a pain

29
Test Driven Development (TDD)

• Write tests first, then write code to be tested
• Writing the tests makes you think like a user
  before you put on your implementer hat
• The tests serve as a (partial) specification that
  guides development, as well as a concrete measure
  of progress
• The resulting testable code tends to be better
  factored, loosely coupled, and more readable and
  maintainable
• It guarantees that unit testing doesn't get
  conveniently forgotten at the end of the project

30
EasyMock

• A tool that builds mock objects based on your
  specifications
• It constrains how the mock is used during
  testing, causing the test to fail if the
  constraints fail
• Falls somewhere in-between a noop mock and a
  fully functional mock

31
EasyMock Usage

• Creating mock objectsmock EasyMock.createMock(
  DbAccess.class)
• Setting constraints on EasyMock
  objectsEasyMock.expect(mock.get(1,GOOG)). a
  ndReturn(20)
• Making EasyMock ready for testingEasyMock.replay
  (mock)
• Verifying that the EasyMock object was used
  properlyEasyMock.verify(mock)

32
AgitarOne

• Product from company Agitar that fully automates
  the process of writing unit tests
• Generated tests (and their infrastructure) will
  pass with current code base
• Product has to be purchased!

33
Exercise

• Rewrite the tests on Slide 14 (Tests using
  JUnit) to use EasyMock

34
Beyond Developer Testing (QA) and Release Process
35
Automation Still Cannot Replace Manual QA

• At this point, no amount of developer test
  automation is going to fully replace manual
  testing by a qualified test engineer
• Trying out weird corner cases, physically
  disconnecting the network cable, etc. are still
  not automatable to our satisfaction
• Eventually, a test engineer has to certify a
  build as ready for production release
• Lots of automation available for this step (e.g.,
  UI testing frameworks)

36
Production Monitoring
37
Web Applications In Production

• Unfortunately, applications are never perfect and
  there will always be problems encountered in
  production settings
• Hence they need to be monitored 24/7
• How can we do this with as much automation as
  possible?

38
Sources of Information

• Logging logs can be written with various kinds
  of annotations, exception traces can be logged
• Special insider access to application (special
  URLs, ports, etc.) that allows us to diagnose the
  health of a running application
• Internal users of application
• External users (customers) of application

39
Logging

• Perform frequent analysis of logs from production
  servers
• Focus on the severe logs and use the location
  in the source file from where the logging
  happened to determine who to contact. Can be
  done fully automatically with integration with a
  source control system
• Same approach can be followed for exceptions
  the stack trace provides linkages to the source
  which can be used to identify the person to
  contact

40
Special Insider Access

• Special insider access can be used to determine
  various aspects of the health of the web
  applications memory used, average time take to
  process requests, number and latency of database
  operations, etc.

41
Paging

• Detection of problems that persist can cause
  notifications through a pager
• There is always a pager carrier on duty who knows
  how to contact the person best suited to address
  the problem on hand

42
Guice
43
Problem from an earlier slide

• Replace
• db new RealImpl()
• With
• db new MockImpl()
• In
• public Account(int accountId)
  this.accountId accountId db new
  RealImpl()

44
Solution

• Change Accounts constructor as follows
• public Account(int accountId, DbAccess db)
  this.accountId accountId this.db
  db
• The real application will create an Account
  object as follows
• new Account(accountId, new RealImpl())
• The JUnit test will create an Account object as
  follows
• new Account(accountId, new MockImpl())

45
Guice

• A dependency injection framework push instead
  of pull model
• Scales our solution in previous slide to large
  systems
• Allows decoupling of large complex systems into
  simpler modules
• Makes it easy to work on individual modules
  without the need to import other modules
• Works very well with unit testing, functional
  testing, etc.

46
Hello World 1

• import com.google.inject.public class Hello
  public static void main(String args)
  Injector injector Guice.createInjector()
  Greeter greeter injector.getInstance(Greeter.
  class) greeter.sayHello() public class
  Greeter public void sayHello()
  System.out.println(Hello World)

47
Notes on Hello World 1

• The simplest possible Guice example
• Instead of creating a Greeter object ourselves,
  we let Guice create one for us
• Guice uses one of its many rules to do this
  namely, if we ask for an object of a class type
  and we have not provided any additional details,
  it uses the default constructor to create an
  object

48
Hello World 2

• import com.google.inject.public class Greeter
  private final Displayer displayer _at_Inject
  public Greeter(Displayer d) displayer
  d public void sayHello() displayer.displa
  y(Hello World) public class Displayer
  public void display(String s)
  System.out.println(s)

49
Notes on Hello World 2

• Class Hello is the same as before, so not
  repeated
• We have a constructor annotated with _at_Inject
  which means this is what Guice should use when
  instantiating an object of this type (it does not
  have to use its rule any more)
• We now have a class Displayer that Greeter
  depends on and is a parameter of the Greeter
  constructor
• Guice knows it has to create a Displayer object
  before it can create a Greeter object for which
  it uses the rule to use the default constructor

50
Hello World 3

• import com.google.inject.public class Greeter
  private final Displayer displayer private
  final Person person _at_Inject public
  Greeter(Displayer d, Person p) displayer d
  person p public void sayHello()
  displayer.display(Hello
  person) public class Displayer public
  void display(String s) System.out.println(s)
  public class Person public String
  toString() return Paddy

51
Notes on Hello World 3

• This example extends the previous example to
  introduce another dependency Person
• This shows that Guice will walk through the
  entire dependency graph and build objects as
  needed

52
Hello World 4

• Modifies Hello World 2 to use an interface for
  Displayer. This does not work. Why?
• public interface Displayer void
  display(String s)public class StdoutDisplayer
  implements Displayer public void
  display(String s) System.out.println(s)

53
Bindings and Modules

• Bindings are mappings between interfaces and
  their desired implementation
• Modules are collections of bindings
• You start an application with a list of modules
• When Guice cannot figure out what to do, it looks
  for help in the appropriate bindings in the
  loaded modules

54
Binding and Module for Hello World 4

• import com.google.inject.class DisplayModule
  implements Module public void configure(Binder
  binder) binder.bind(Displayer.class) .to(St
  doutDisplayer.class)
• The Injector creation statement is replaced with
• Injector injector Guice.createInjector( ne
  w DisplayModule))

55
So How is Guice useful for Testing?

• For every group of related classes, we create two
  modules one that instantiates the real objects
  and the other that instantiates the mock objects
• To unit test a portion of an application, create
  an injector with the real objects of the portion
  to be tested and the mocks for all other portions
• This approach is extremely flexible and allows
  creating all kinds of useful configurations for
  testing without having to make any changes to the
  underlying code

56
Exercise

• Modify the brokerage account example and tests
  to use Guice