JIFL: JIT Instrumentation Framework for Linux

1
JIFL JIT Instrumentation Framework for Linux
Marek Olszewski Adam Czajkowski Keir
Mierle University of Toronto
2
Instrumenting Operating Systems

• Operating systems are growing in complexity
• Becoming harder to understand
• Kernel instrumentation is a well know method of
  combating this problem
• Used for debugging, profiling, monitoring,
  coverage testing, security auditing...
• Dynamic instrumentation is especially useful
• No recompilation no reboot
• Good for debugging systemic problems
• Feasible in production settings

3
Dynamic Instrumentation

• All dynamic instrumentation tools for operating
  systems are probe based
• Overwrite existing code with jump/trap
  instructions
• Efficient on fixed length architectures
• Slow on variable length architectures
• Must use trap instruction (and hash table lookup)
• JIT-based instrumentation can be more efficient
• Proven itself for user space (Pin, Valgrind)
• Probe-based instrumentation is seldom used in
  user-space

4
Probe-based Instrumentation
Instrumentation Code
Trap Handler
OS Code

1. Look up which instrumentation to call
2. Call instrumentation
3. Emulate overwritten instruction

Instrumentation Code
5
JIT Instrumentation

• JIT instrumentation rewrites the OS code with
  calls to instrumentation code
• Creates a duplicate instrumented copy of the OS
• Called the code-cache
• Since instrumentation is dynamic, it is not
  feasible to rewrite the entire operating system
  up-front
• Instrumentation is performed just-in-time, basic
  block by basic block, right before each new basic
  block is executed.
• The resulting code is fast
• No hash table lookup required
• Though there is some cost in executing in the
  code-cache

6
JIT Instrumentation
Instrumentation Code
Duplicate Copy of OS
OS Code
Instrumentation Code
7
Software Architecture
8
Using JIFL

• Use JIFLs API and build system to easily write
  loadable instrumentation plugins
• Start by implementing four key functions
• plugin_init
• plugin_exit
• plugin_start
• plugin_stop
• Load/Unload plugin with Linuxs insmod/rmmod
  tools
• Start/Stop instrumentation with the JIFL starter
  tool

9
Example Plugin
include "jifl.h" syscall_t syscall long long
count // Executed for every instrumented basic
block static void inc_count(long long
counter_ptr, long size) counter_ptr
size // Called for every newly discovered
basic block static void bb_inst(bb_t bb, void
arg) long bb_size bb_size(bb)
bb_insertcall(bb, inc_count, ARG_VOID_PTR,
count, ARG_INT32, bb_size, ARG_END) // Start
instrumentation of clone system call static void
plugin_start() syscall_init(syscall,
__NR_clone) Syscall_add_bb_instrumentation(s
yscall, bb_inst, NULL) syscall_start_instrume
nting(syscall) // Stop instrumentation of
clone system call static void plugin_stop()
syscall_stop_instrumenting(syscall)
printk("Clone system call executed lld
instructions\n", count)
10
Performance Evaluation
Apache Web Server Throughput
11
Conclusions

• JIT instrumentation viable for operating systems
• Fine grained instrumentation now possible for
  kernel space on variable length architectures
• Intels x86
• AMDs AMD64
• Great performance
• Though it comes with a fixed cost