Model Checking One Million Lines of C Code

1
Model Checking One Million Lines of C Code

• Hao Chen
• Drew Dean (SRI International)
• David Wagner
• with
• David Schultz, Geoff Morrison, Ben Schwarz
• Jacob West, Jeremy Lin

2
Outline

• Overview of MOPS
• Engineering effortsMake MOPS work on lots of
  real world applications
• ExperienceSecurity properties and findings

3
MOPS (MOdel checking Programs for Security
properties)

• A static analysis tool that checks source
  programs for temporal safety properties
• Main features
• Pushdown model checking
• Inter-procedural analysis
• Control flow centric

4
Algorithm Pushdown Model Checking

• ? set of security operations
• e.g. ? seteuid(!0), seteuid(0), execl()
• B?? sequences of security operations that
  violate the property
• B is a regular language
• The user describes B by an FSA
• T?? sequences of security operationsfrom
  feasible traces in the program
• T is a context-free language
• MOPS automatically builds T from the program
• Question Is B ? T ??

B
T
5
The MOPS Process
Safety Property
FSA
Program OK
Program
CFG
Model Checker
Parser
Error Traces (they may violate the property)
FSA Finite State Automaton CFG Control Flow
Graph
6
Integrate MOPS into the Build Process

• Goal build a CFG for each target executable in
  the source package
• Low-level programs
• mops_cc1 parse each foo.c into foo.cfg
• mops_ld link all .cfg together
• How to use these programs to build CFGs for each
  package conveniently?

7
First Attempt Edit Makefile

• Feasible for packages with simple Makefiles
• OpenSSH
• Problems
• Some Makefiles are automatically generated they
  will be overwritten when the package is rebuilt
• Too many Makefiles to edit some packages
  generate one Makefile in each sub-directory
• Some Makefiles are very complicated

8
Second Attempt Interpose on the Build Process to
Generate CFGs

• Set the env variable GCC_EXEC_PREFIXto let gcc
  use
• mops_cc1 for cc1
• mops_ld for ld
• Therefore
• Each source file foo.c is parsed into a CFG file
  foo.o
• Each executable file a.out contains a linked CFG
• Note that machine code is not generated

9
Second Attempt Advantages and Problems

• Advantage works even if the build process
• Moves files into other directories (mv)
• Renames files (mv)
• Creates libraries (ar)
• Problem
• Sometimes machine code is needed
• By autoconf to test the functionality of the
  compiler
• By the build process to generate C header files
• But we let gcc create CFGs instead of machine code

10
Third AttemptBuild Both Machine Code and CFGs

• Solution
• Build both machine code and CFGs. For each
  foo.c, build both foo.o and foo.cfg
• Problem
• Machine code and its corresponding CFG may be
  separated if the build process
• Moves and renames files (mv)
• Creates and uses libraries (ar)
• We need to bind machine code with its CFG

11
Final Attempt

• Place both machine code and its CFG in the same
  object file
• The ELF object format allows user-defined
  sections
• For each source file foo.c
• gccs cc1 generates machine code foo.o
• mops_cc1 generates CFG foo.cfg
• Run objcopy to place foo.cfg into foo.o

12
Now, as Easy as This

• mops m property.mfsa -- gcc foo.c bar.c
• mops b rpm m property.mfsa -- rh9/.src.rpm

13
Experiment Checking Security Properties on Large
Packages

• Properties
• Drop privilege before making unsafe system calls
• Avoid race condition in file system access
• Create root-jail safely
• Create temporary files safely
• Avoid stderr vulnerability
• Programs
• 7 programs, total one millions LOC

14
Property Drop Privilege before Executing
Untrusted Programs

• Setuid-root programs need to drop root privilege
  before executing untrusted programs
• exec...(), system(), popen()
• Otherwise, the untrusted program will run with
  the root privilege

15
Property Avoid Race Condition in File System
Access

• It is suspicious to make two system calls in a
  program path that use the same filename
• E.g.
• access(f) followed by open(f)
• stat(f) followed by create(f)
• Vulnerability
• Symbolic links

16
Property Create root jail securely

• Rule chroot() must be followed by chdir()
  immediately
• A vulnerable program
• The adversary sends in ../../etc/password from
  the network

// cwd/var/ftp chroot(/var/ftp/pub) filename
read_from_network() fd open(filename,
O_RDONLY)
17
Property Avoid Race Condition in Creating
Temporary Files

• victim.cfilename mktemp(template)fd
  open(filename, )
• But an adversary can create a file with the same
  name between the two statements
• Then, victim.c will
• Open the adversarys file, or
• Fail to create the temporary file (with the
  O_EXCL flag)

18
Rules for Avoiding Race Condition in Creating
Temporary Files

• Never use the unsafe functionsmktemp(),
  tmpnam(), tempname(), tmpfile()
• Never reuse the parameter f in mkstemp(f) in any
  other function call, such as stat(), chmod(), etc
• Because the same name may refer to a different
  file (tmpwatch)

19
Property Avoid stderr Vulnerability
int main() // fd 0 and 1 are open, but fd 2
is closed f open(/etc/passwd, O_RDWR)
// now /etc/passwd is opened to fd 2
fprintf(stderr, s, user_input) // oops,
have written to /etc/passwd
20
Rule Avoid stderr Vulnerability

• Property
• The first three file descriptors should not be
  opened by any file except /dev/null and /dev/zero

21
Property stderr vulnerability
Package LOC Running Time Error Traces Error Traces
Package LOC Running Time Real Bugs Total
Sendmail 221K 13m32s 0 2
Postfix 97K 58.2s 0 2
OpenSSH 58K 3m1s 3 7
Apache 240K 53.0s 2 2
BIND 280K 3m13s 1 1
At 5.1K 7.7s 2 2
Cron 3.7K 5.0s 2 2
22
Property create temporary files safely
Package LOC Running Time Error Traces Error Traces
Package LOC Running Time Real Bugs Total
Sendmail 221K 43.9s 0 0
Postfix 97K 28.8s 0 0
OpenSSH 58K 49.0s 0 1
Apache 240K 47.4s 1 1
BIND 280K 67.3s 1 1
At 5.1K 5.3s 0 0
Cron 3.7K 4.4s 0 0
23
Experiment Build CFGs for All RPM Packages in
Redhat 9.0 Linux

• Number of packages 840
• Time 3 days 14 hours on 1.5G Pentium
• Preliminary results
• Successful 514 packages
• Failed 326 packages
• Reasons for failure
• Lack of prerequisite RPM packages(the major
  reason)
• Bugs/limitations in MOPSs parser
• The package was not written in C

24
Summary

• Engineering efforts in making MOPS work on lots
  of real world applications
• Experience in checking security properties on
  large applications