jFuzz

1
jFuzz Java based Whitebox Fuzzing

• David Harvison
• Adam Kiezun

2
Summary

• Problem
• Generating interesting test inputs for file
  reading programs takes time.
• Approach
• Create a smart fuzzer to generate inputs that
  cause programs to crash.
• Results
• jFuzz generates a many input files and creates a
  base for others to expand upon.

3
Problem

• Bugs in a program may cause crashes for specific
  input files.
• A compiler with buggy code, a media player with a
  corrupt file, etc.
• Creating input files by hand takes time.
• Some of the files may exercise the same code.
• Want a way to automatically generate input files
  that crash the program.

4
Idea

• Generate inputs that cause crashes by generating
  many inputs that exercise unique execution paths.

5
Program Example

• if (car0 '-')
• neg true
• else
• neg false
• cnt
• if (car1 gt '0' car1 lt '5')
• val car1 - '0'
• else
• val car1

• File reading code.

6
Program Example

• File reading code.
• Want to generate files which exercise different
  program paths.

• if (car0 '-')
• neg true
• else
• neg false
• cnt
• if (car1 gt '0' car1 lt '5')
• val car1 - '0'
• else
• val car1

7
Program Example

• File reading code.
• Want to generate files which exercise different
  program paths.

if (car0 '-') neg true else
neg false cnt if (car1 gt '0'
car1 lt '5') val car1 - '0' else
val car1
car0 '-' car1 '5'
8
Program Example

• File reading code.
• Want to generate files which exercise different
  program paths.

if (car0 '-') neg true else
neg false cnt if (car1 gt '0'
car1 lt '5') val car1 - '0' else
val car1
car0 '-' car1 '7'
9
Program Example

• File reading code.
• Want to generate files which exercise different
  program paths.

if (car0 '-') neg true else
neg false cnt if (car1 gt '0'
car1 lt '5') val car1 - '0' else
val car1
car0 '' car1 '3'
10
Program Example

• File reading code.
• Want to generate files which exercise different
  program paths.

if (car0 '-') neg true else
neg false cnt if (car1 gt '0'
car1 lt '5') val car1 - '0' else
val car1
car0 '' car1 '9'
11
Related Tools

• Cute, EXE, SAGE, catchconv, Apollo
• Smart fuzzers - programs that generate
  interesting new inputs for programs.
• Not for Java.
• JCute
• Smart fuzzer for Java.
• Reinstruments code Requires source files.
• Has problems with the JDK.

12
Overall Idea

• Input
• A compiled (into bytecode) Java program.
• A valid input file.
• Output
• New input files which exercise unique control
  paths.
• Run the subject program in a modified JVM.
• A logic predicate, the Path Condition, is formed
  as the program executes.
• Describes control flow of execution.
• New inputs are created by manipulating the path
  condition.

13
Example
public void top(char input) int cnt 0
if (input0 b) cnt if (input1
a) cnt if (input2 d) cnt if
(input3 !) cnt if (cnt gt 3)
crash()
good
14
Example
public void top(char input) int cnt 0
if (input0 b) cnt if (input1
a) cnt if (input2 d) cnt if
(input3 !) cnt if (cnt gt 3)
crash()
I0 ! 'b'
I1 ! 'a'
I2 ! 'd'
I3 ! '!'
path condition
good

• Negate constraints in path condition.
• Solve the new path condition to create new inputs.

15
Example
public void top(char input) int cnt 0
if (input0 b) cnt if (input1
a) cnt if (input2 d) cnt if
(input3 !) cnt if (cnt gt 3)
crash()
I0 ! 'b'
I1 ! 'a'
I2 ! 'd'
I3 '!'
goo!
good
16
Example
public void top(char input) int cnt 0
if (input0 b) cnt if (input1
a) cnt if (input2 d) cnt if
(input3 !) cnt if (cnt gt 3)
crash()
I0 ! 'b'
I1 ! 'a'
I2 'd'
godd
good
17
Example
bood
public void top(char input) int cnt 0
if (input0 b) cnt if (input1
a) cnt if (input2 d) cnt if
(input3 !) cnt if (cnt gt 3)
crash()
gaod
godd
goo!
good
All paths are explored systematically.
18
Tool Used

• NASA Java PathFinder
• Dynamic analysis framework for Java implemented
  as a JVM.
• Allows backtracking including saving and
  restoring the whole state of the VM.
• Can execute all thread interleavings.
• Can execute a program on all possible inputs.

19
Attributes

• Additional state-stored information.
• Associated with runtime values.
• JPF propagates attributes across assignment,
  method calls, etc.
• Allows us to keep track of how the variables
  relate to the input using symbolic expressions.

20
Concrete v. Concolic
Normal
With Attributes
exp0
exp1
3
4
3
4


7
Sum (exp0, exp1)
7
Concrete
Concolic execution is both concrete and symbolic.
Symbolic
21
Concrete v. Concolic

• public class IADD extends Instruction
• public Instruction execute (...
  ThreadInfo th)
• 1 int v1 th.pop()
• 2 int v2 th.pop()
• 3 th.push(v1 v2, ...)
• 4 return getNext(th)

• public class IADD extends ...bytecode.IADD
• public Instruction execute (...
  ThreadInfo th)
• 1 int v1 th.pop()
• 2 int v2 th.pop()
• 3 th.push(v1 v2, ...)
• 4 StackFrame sf th.getTopFrame()
• 5 IntExpr sym_v1 sf.getOperandAttr()
• 6 IntExpr sym_v2 sf.getOperandAttr()
• 7 if (sym_v1 null sym_v2 null)
• return getNext(th)
• 7 IntExpr result sym_v1._plus(sym_v2)
• 8 sf.setOperandAttr(result)
• 9 return getNext(th)

Concrete
Symbolic
22
jFuzz Architecture
Subject and Input

• Runs JPF many times on the subject program and
  input files.
• Each run
• Collects the Path Condition (PC).
• Negates each constraint, reduces, and solves.
• Uses new PCs to generate new input files.
• Keeps track of inputs which caused exceptions to
  be thrown.

jFuzz
JPF
Subject and Original Input
PC
Negated PC
Solver
New Input
Inputs which cause crashes
23
Creating New Inputs

• For a given execution some parts of the input may
  not be read.

24
Creating New Inputs

• For a given execution some parts of the input may
  not be read.

25
Creating New Inputs

• For a given execution some parts of the input may
  not be read.
• When the path condition is solved, only the read
  parts will have new values.

26
Creating New Inputs

• For a given execution some parts of the input may
  not be read.
• When the path condition is solved, only the read
  parts will have new values.
• The changes are written over the original input,
  preserving the unused parts.

27
Reducing the Path Condition
Subject and Input

• Path Conditions can be very long.
• Not all constraints are effected by negating the
  PC.
• Constraints not effected can be removed from the
  PC.

jFuzz
JPF
Subject and Original Input
PC
Negated PC
PC Minimizer
Solver
New Input
Inputs which cause crashes
28
Example Reduction

• Path Condition
• 1 a b lt 10
• 2 b gt 6
• 3 c lt 15
• 4 a lt 3
• 5 c d gt 7
• 6 e ! 1
• 7 c e 5
• 8 a 2

29
Example Reduction

• Start with fuzzing the last constraint.

• Path Condition
• 1 a b lt 10
• 2 b gt 6
• 3 c lt 15
• 4 a lt 3
• 5 c d gt 7
• 6 e ! 1
• 7 c e 5
• 8 a ! 2

30
Example Reduction

• Start with fuzzing the last constraint.
• Select all constraints which contain variables in
  that constraint.

• Path Condition
• 1 a b lt 10
• 2 b gt 6
• 3 c lt 15
• 4 a lt 3
• 5 c d gt 7
• 6 e ! 1
• 7 c e 5
• 8 a ! 2

31
Example Reduction

• Start with fuzzing the last constraint.
• Select all constraints which contain variables in
  that constraint.
• If one of the constraints contains multiple
  variables, select all constraints which contain
  those variables.

• Path Condition
• 1 a b lt 10
• 2 b gt 6
• 3 c lt 15
• 4 a lt 3
• 5 c d gt 7
• 6 e ! 1
• 7 c e 5
• 8 a ! 2

32
Example Reduction

• Start with fuzzing the last constraint.
• Select all constraints which contain variables in
  that constraint.
• If one of the constraints contains multiple
  variables, select all constraints which contain
  those variables.
• All other constraints can be removed.

• Path Condition
• 1 a b lt 10
• 2 b gt 6
• 4 a lt 3
• 8 a ! 2

33
Example Reduction

• Start with fuzzing the last constraint.
• Select all constraints which contain variables in
  that constraint.
• If one of the constraints contains multiple
  variables, select all constraints which contain
  those variables.
• All other constraints can be removed.
• Variables not in the new PC are left unchanged.

• Path Condition
• 1 a b lt 10
• 2 b gt 6
• 3 a lt 3
• 4 a ! 2

34
Reducing the Path Condition

• Reductions are performed for every constraint
  that is negated.
• jFuzz uses a UnionFind data structure to find
  which variables are connected to each other.
• In our case study, the average reduction was from
  around 250 constraints to about 5 constraints.

35
Case Study
test1.dimacs c test 3 single clauses and 2 c
binary clauses p cnf 4 5 1 0 2 0 3 0 -2 4 0 -3 4 0

• Subject Sat4J
• SAT solver written in Java.
• Takes inputs in dimacs files.
• 10 kloc.
• Goals
• Create inputs that crash Sat4J.
• Create a set of good inputs.

36
Results

• After 30 minutes of execution
• 12,000 input files were created.
• 70 crashes where found.
• The crashes are actually normal for SAT4J
• 38 Invalid DIMACS files.
• 27 Contradictions.
• 4 Assertion Errors.
• A Java compiler would be more compelling.
• Any crash is due to a bug in the compiler.
• Much larger program.

37
Performance

• Sat4J was run 100 times in each VM.
• The times are average runtime.
• Simplifying the Path Condition reduces the
  solving time by 30.

38
Conclusions

• jFuzz is the first concolic tester for Java which
  will work for any bytecode.
• This opens the door for more advanced fuzzing
  techniques, such as grammar based fuzzing.

39
(No Transcript)
40
Fuzzing the Input
Path Condition 1 a b lt 10 2 b gt 6 3 c lt
15 4 a lt 3 5 c d gt 7 6 e ! 1 7 c e
5 8 a ! 2
a
b
c
d
e
41
Fuzzing the Input
Path Condition 1 a b lt 10 2 b gt 6 3 a lt
3 4 a ! 2
a
b
c
d
e

• The reduced PC does not effect all parts of the
  input.
• The solver will only return values for variables
  in the PC.

42
Fuzzing the Input
Path Condition 1 a b lt 10 2 b gt 6 3 a lt
3 4 a ! 2
a'
b'
c
d
e

• The new values are written over the original
  input.
• The new input only differs from the original in
  the values in the PC.