Deriving%20formal%20specifications%20(almost)%20automatically

1
Deriving formal specifications (almost)
automatically

• Glenn Ammons and Ras Bodik and James R. Larus

2
Three pillars of formal verification

• Model checkers and other verifiers
• well automated (SLAM, Spin, type checkers, Vault)
• Program abstractors
• getting there (SLAM, Englers metacompiler)
• Formal specifications
• Written by hand
• Our goal bring automation to writing formal
  specifications

3
Deriving specs is feasible

• Well-debugged software exists
• Good code obeys the rules, but doesnt state them
  clearly
• Common behavior is good behavior
• Because testing exposes common behavior
• Programmers exist
• But they dont want to write specs!

4
Rules describe good behavior
A rule is a nondeterministic finite automaton
T XNextEvent
XSetSelectionOwner(T)
S
F
XSetSelectionOwner(T)
XGetSelectionOwner
XGetSelectionOwner
5
Rules are derived from traces, with user guidance
XtAppNextEvent() event(type 5, window 22,
time 315) XtDispatchEven
t(type 5, window 22, time
315) XtFree(NULL) XtFree(NULL) XtMalloc(size
8) 0x10 XmuInternStrings(names 0x20, count
2, atoms_return
0x10) XtOwnSelection(widget 0x30, selection
1, time 315)
And so on the more traces the better
6
Overview
Traces
Programs or traces (buggy?)
Seeds
Abstraction prescription
Rule learner
Program abstractor
Rules
Abstract programs or traces
Matcher
Bugs!
7
Case study selections in X11

• The rule SetSelectionOwner must be passed a
  timestamp from an Xevent
• 25 programs from the X11 distribution and the
  contrib directories (all used selections)
• Verification done over traces (not statically)
• Found two bugs in 29 static uses
• Found three benign violations

8
To do

• Static checking typestates
• Better simplifier
• Better user interaction
• What else can we learn?
• Protocols like socket/bind/accept/close
• Operations on data structures

9
Power
What else can we do with this stuff? Compare with
Ernst
10
Detailed figure?
11
Running example
12
Testing vs. verification

• Examines the complete programs
• Examines some inputs
• For better coverage, write more test cases

• Examines only some aspects of programs
• Examines all inputs
• For better coverage, write more specs

The practice sees writing test cases as easier
than writing formal models and specifications, so
testing dominates.