Tutorial lab 2: Code instrumentation

1
Tutorial / lab 2 Code instrumentation

• Goals of this session
• Create some tests for a Java class.
• Measure the code coverage.

2
Code Instrumentation Tools

• Tools that modify production code for various
  purposes during code execution
• Debuggers
• Add information to allow for break points or
  step-by-step execution.
• Coverage measurement
• Report on the number of times various code
  elements have been executed.
• Profilers
• Measure amount of time spent in various parts of
  the code.

3
Coverage tools

• Program is typically compiled with special
  options, to add extra source or object code.
• Additional data structures, such as a flow graph,
  may also be created.
• Program is run, possibly via test cases
• During execution, information is accumulated and
  written to an output file.
• Post-processing phase
• User report is generated from output file.

4
How to measure coverage?

• Instrument the source code before compilation
• Instrument the virtual machine or object code.
• The Apache Byte Code Engineering Library (BCEL)
  is used by several Java coverage tools
• http//jakarta.apache.org/bcel
• Use a customized virtual machine.

5
Source code instrumentation

• Create a copy of the source code, instrument the
  copy, and compile the instrumented copy to .class
  files.
• This is the approach used by the Clover code
  coverage tool.
• Typical method
• Set up an array of counters.
• Insert source code at points of interest that
  increment specific counters.
• Coverage tool calls main() method
• After main method returns, dump values of
  counters.

6
Source code before instrumentation

• public final void add( final MatrixRowltTgt
  newValues )
• if ( getNumColumns( ) 0 )
• contents newValues.copy( ).getContents(
  )
• else
• contents.addAll( newValues.copy(
  ).getContents( ) )
• A counter is inserted for each method, statement,
  and branch.
• Statement counters are before each statement, in
  case the statement throws an exception.
• Branch counters need to identify true and false
  branches taken.

7
Same code after instrumentation

• public final void add( final MatrixRowltTgt
  newValues )
• CLOVER.M348
• CLOVER.S1814
• if ( ( getNumColumns( )0
  CLOVER.CT338!0 true )
• ( CLOVER.CF338 0 false ) )
• CLOVER.S1815
• contents newValues.copy( ).getContents(
  )
• else
• CLOVER.S1816
• contents.addAll( newValues.copy(
  ).getContents( ) )

statement counters
branch counters
method counter
8
Counting branches

• Count number of times that a branch condition
  evaluates to true and false.
• if ( ( aBoolean CT338!0 true )
• ( CF338 0 false ) )
• If aBoolean is true, then the is also
  evaluated, which will increment the CT counter.
  We want the expressions overall evaluation to be
  true.
• If aBoolean is false, then execution switches to
  the right side of the clause. Increment the
  CF counter, but then ensure that the entire
  expression still evaluates to false to take the
  else branch.
• Short-circuit evaluation means that as soon as
  the expression result can be determined,
  evaluation stops. This avoids
• Incrementing the true counter if aBoolean is
  false.
• Incrementing the false counter if aBoolean is
  true.

9
Analysis

• Advantages
• Results are directly mapped to source code.
• If a statement throws an exception, it will be
  counted as having been executed.
• Disadvantages
• Requires keeping a second version of source code,
  and ensuring that original source is not
  over-written.
• Instrumented code must not affect original code
  execution.
• Management of .class files from compiling
  alternate source code.
• Memory usage for statement coverage alone,
  every statement requires one integer as a counter.

10
Byte code instrumentation

• Can be done in two ways
• Instrument object files in advance to produce a
  alternate version.
• Example create instrumented .class files for
  Java.
• Instrument object files as they are used.
• Example create customized .class file loader
  that inserts the instrumentation at run time.
• This is the approach used by the Emma code
  coverage tool
• This is an open-source tool see
  http//emma.sourceforge.net
• Eclipse plug-in version http//www.eclemma.org

11
Analysis

• Pre-instrumentation
• Advantages
• source code is not modified.
• can be done as a batch process before execution
• Disadvantages
• management of two .class file versions.
• mapping of source code statements to/from object
  code
• counting source code statements can be difficult
• object code may need extra information such as
  special compiler options