Performance of OSHL on Problems Requiring Definition Expansion

1
Performance of OSHL on Problems Requiring
Definition Expansion

• Swaha Miller
• David A. Plaisted
• UNC Chapel Hill

2
How do humans prove theorems?

• Semantics
• Case analysis
• Sequential search through space of possible
  structures
• Focus on the theorem

3
Systematic methods can now routinely solve
verification problems with thousands or tens of
thousands of variables, while local search
methods can solve hard random 3SAT problems with
millions of variables. (from a conference
announcement)
4
DPLL Example
p,r,?p,?q,r,p,?r
pT
pF
T,r,?T,?q,r,T,?r
F,r,?F,?q,r,F,?r
SIMPLIFY
SIMPLIFY
?q,r
r,?r
SIMPLIFY

5
Hyper Linking
6

• Eliminating Duplication with the Hyper-Linking
  Strategy, Shie-Jue Lee and David A. Plaisted,
  Journal of Automated Reasoning 9 (1992) 25-42.

7
Later propositional strategies

• Billons disconnection calculus, derived from
  hyper-linking
• Disconnection calculus theorem prover (DCTP),
  derived from Billons work
• FDPLL (Model Evolution Calculus)

8
Performance of DCTP on TPTP, 2003

• DCTP 1.3 first in EPS and EPR (largely
  propositional)
• DCTP 10.2p third in FNE (first-order, no
  equality) solving same number as best provers
• DCTP 10.2p fourth in FOF and FEQ (all first-order
  formulae, and formulae with equality)
• DCTP 1.3 is a single strategy prover.

9
Strategy Selection in E
10
Strategy Selection

• Schulz, Stephan, E-A Brainiac Theorem Prover,
  Journal of AI Communications 15(2/3)111-126,
  2002.

11
Strategy Selection

• The Vampire kernel provides a fairly large number
  of features for strategy selection. The most
  important ones are
• Choice of the main saturation procedure (i)
  OTTER loop, with or without the Limited Resource
  Strategy, (ii) DISCOUNT loop.
• A variety of optional simplifications.
• Parameterised reduction orderings.
• A number of built-in literal selection functions
  and different modes of comparing literals.
• Age-weight ratio that specifies how strongly
  lighter clauses are preferred for inference
  selection.
• Set-of-support strategy.

12
Strategy Selection

• The automatic mode of Vampire 7.0 is derived from
  extensive experimental data obtained on problems
  from TPTP v2.6.0. Input problems are classified
  taking into account simple syntactic properties,
  such as being Horn or non-Horn, presence of
  equality, etc. Additionally, we take into account
  the presence of some important kinds of axioms,
  such as set theory axioms, associativity and
  commutativity. Every class of problems is
  assigned a fixed schedule consisting of a number
  of kernel strategies called one by one with
  different time limits.

13
DCTP Strategy Selection

• DCTP 1.31 has been implemented as a monolithic
  system in the Bigloo dialect of the Scheme
  language.
• DCTP 1.31 is a single strategy prover.
  Individual strategies are started by DCTP 10.21p
  using the schedule based resource allocation
  scheme known from the E-SETHEO system. Of course,
  different schedules have been precomputed for the
  syntactic problem classes. The problem classes
  are more or less identical with the sub-classes
  of the competition organisers.
• In CASC-J2 DCTP 10.21p performed substantially
  better.

14
Goal of OSHL

• First-order logic
• Clause form
• Propositional efficiency
• Semantics
• Requires ground decidability

15
Structure of OSHL

• Goal sensitivity if semantics chosen properly
• Choose initial semantics to satisfy axioms
• Use of natural semantics
• For group theory problems, can specify a group
• Sequential search through possible
  interpretations
• Thus similar to Davis and Putnams method
• Propositional Efficiency
• Constructs a semantic tree

16
Ordered Semantic Hyperlinking (Oshl)

• Reduce first-order logic problem to propositional
  problem
• Imports propositional efficiency into first-order
  logic
• The algorithm
• Imposes an ordering on clauses
• Progresses by generating ground instances Di of
  input clauses and refining interpretations

17
Semantics

• Trivial semantics
• Positive Choose I0 to falsify all atoms, first
  D is all positive. Forward chaining.
• Negative Choose I0 to satisfy all atoms, first
  D is all negative. Backward chaining.
• Natural semantics I0 chosen by user

18
Semantics Ordering

• ltt a well founded ordering on atoms, extended to
  literals
• Extend ltt to interpretations as follows
• I and J agree on L if they interpret L the same
• Suppose I0 is given
• I ltt J if I and J are not identical, A is the
  minimal atom on which they disagree, and I agrees
  with I0 on A

19
Rules of OSHL Start with empty sequence (C1,C2,
, Cn), D minimal ground instance of an input
clause that contradicts I, I minimal model of
sequence (C1,C2, , Cn,D) (C1,C2, , Cn, D), Cn
out of order (C1,C2, , Cn-1,D) (C1,C2, ,
Cn,D), max resolution possible (C1,C2, ,
Cn-1,res(Cn,D,L)) Proof if empty clause derived
20
-
Propositional Example (?p I0 p) () (-p1,
-p2, -p3) I0-p3 (-p1, -p2, -p3, -p4, -p5,
-p6) I0 -p3,-p6 (, , -p7) I0
-p3,-p6,-p7 (, , -p7, p3, p7) (,
-p4, -p5, -p6, p3) (-p1, -p2,
-p3,p3) (-p1, -p2 ) I0 -p2
21
U Rules

• Choose clauses instances to match existing
  literals. Look for a contradiction.
• Basic clauses and U clauses
• Basic clauses are used in three rules given
• Sequence can also have U clauses on the end
• U clauses have a selected literal
• In basic clauses the max. lit. is selected
• In U clauses other literals can be selected.
• Significant performance enhancement.

22
UR Resolution Example

• Given the sequence (s(a), p(b) , t(a), q(b))
• and the clause ?p(X), ?q(X), r(X)
• create the sequence
• (s(a), p(b), t(a), q(b), ?p(b), ?q(b),
  r(b) )

X ? b
23
Filtering Example

• Given the sequence (s(a), p(b), t(a), q(b))
• and the clause ?p(X), ?q(X)
• create the sequence
• (s(a), p(b), t(a), q(b), ?p(b), ?q(b) )

X ? b
24
Case Analysis Example

• Given the sequence (s(a), p(b), t(a), q(b))
• and the clause ?q(X), r(X), s(X)
• create the sequence
• (s(a), p(b), t(a), q(b), ?q(b), r(b),
  s(b) )

X ? b
25
Example Proof Using U Rules

• All positive semantics
• Clauses
• A1. ?X?Y, ?Y?X, XY
• A2. ?Z?X, ?X?Y, Z?Y
• A3. g(X,Y)?X, X?Y
• A4. ?g(X,Y)?Y, X?Y
• A5. ?Z?X, Z?X ? Y
• A6. ?Z?Y, Z?X ? Y
• A7. ?Z?X ? Y, Z?X, Z?Y
• T. ?A ? B B ? A

26
Example Proof Using U Rules

• 1. ?A ? B B ? A (T)
• 2. ?A ? B ? B ? A, ?B ? A ? A ? B, A ? B B ?
  A (Case Analysis, A1)
• 3. ?g(A ? B, B ? A) ? B ? A, A ? B ? B ? A (UR
  resolution, A4)
• 4. g(A ? B, B ? A) ? B ? A, ?g() ? B (UR
  resolution, A5)
• 5. g(A ? B, B ? A) ? B ? A, ?g() ? A (UR
  resolution, A6)
• 6. g() ? B, g() ? A, ?g() ? A ? B (UR
  resolution, A7)
• 7. A ? B ? B ? A, g() ? A ? B (Filtering, A3)

27
Example Proof Using U Rules

• 1. ?A ? B B ? A
• 2. ?A ? B ? B ? A, ?B ? A ? A ? B, A ? B B ?
  A (Case Analysis)
• 3. ?g(A ? B, B ? A) ? B ? A, A ? B ? B ? A (UR
  resolution)
• 4. g(A ? B, B ? A) ? B ? A, ?g() ? B (UR
  resolution)
• 5. g(A ? B, B ? A) ? B ? A, ?g() ? A (UR
  resolution)
• 8. g() ? B, g() ? A, A ? B ? B ? A,
  (Resolution of 6. and 7.)

28
Example Proof Using U Rules

• 1. ?A ? B B ? A
• 2. ?A ? B ? B ? A, ?B ? A ? A ? B, A ? B B ?
  A (Case Analysis)
• 3. ?g(A ? B, B ? A) ? B ? A, A ? B ? B ? A (UR
  resolution)
• 4. g(A ? B, B ? A) ? B ? A, ?g() ? B (UR
  resolution)
• 9. g(A ? B, B ? A) ? B ? A, g() ? B, A ? B ? B
  ? A (Resolution of 8. and 5.)

29
Example Proof Using U Rules

• 1. ?A ? B B ? A
• 2. ?A ? B ? B ? A, ?B ? A ? A ? B, A ? B B ?
  A (Case Analysis)
• 3. ?g(A ? B, B ? A) ? B ? A, A ? B ? B ? A (UR
  resolution)
• 10. g(A ? B, B ? A) ? B ? A (Resolution of 9.
  and 4.)

30
Example Proof Using U Rules

• 1. ?A ? B B ? A
• 2. ?A ? B ? B ? A, ?B ? A ? A ? B, A ? B B ?
  A (Case Analysis)
• 11. A ? B ? B ? A (Resolution of 10. and 3.)

31
Example Proof Using U Rules

• 1. ?A ? B B ? A
• 12. ?B ? A ? A ? B, A ? B B ? A (Resolution
  of 11 and 2)

Now the other half of the proof will be done.
Note that there is only one ascending sequence of
clauses constructed by OSHL and we are only
indicating part of it.
32
Implementation Results

• Slower implementation speed of OSHL
• Uniform strategy versus strategy selection
• The choice of Otter
• Influence of U rules on an earlier version
• None 233 proofs in 30 seconds on TPTP problems
• Using them 900 proofs in 30 seconds
• All results for trivial semantics

33
Heuristics

• Delta size of propositional instances
• Relevance distance
• Favor propositional terms in the theorem

34
Problems Involving Definitions

• Consider the problem
• S1 ? S2 ? ? Sn Sn ? Sn-1 ? ?
  S1
• ? is associated to the left
• For increasing values of n, gives us a set of
  progressively harder problems
• Requires expanding definitions of set union, set
  equality, and subset relations
• Tested on OSHL-U, Otter and 3 other leading
  provers
• Vampire
• E-Setheo
• DCTP

35
Problems Involving Definitions
n OSHL-U OSHL-U Otter Otter Vampire Vampire E-Setheo DCTP
n time (s) Gen time (s) Gen time (s) Gen time (s) time (s)
2 0.175 41 600 100 K 0.00 103 0.0 0.01
3 0.678 85 600 66 K 70.1 3 M 0.3 300
4 2.107 141 600 47 K 300 25 M 0.3 300
5 5.317 207 600 46 K 300 25 M 2.6 300
6 12.02 283 600 60 K 300 25 M 300 300
7 38.97 525 600 56 K 300 25 M 300 300
8 77.94 663 600 56 K 300 25 M 300 300
36
Problems Involving Definitions
n OSHL-U OSHL-U Otter Otter Vampire Vampire E-Setheo DCTP
n time (s) Gen time (s) Gen time (s) Gen time (s) time (s)
2 0.175 41 600 100 K 0.00 103 0.0 0.01
3 0.678 85 600 66 K 70.1 3 M 0.3 300
4 2.107 141 600 47 K 300 25 M 0.3 300
5 5.317 207 600 46 K 300 25 M 2.6 300
6 12.02 283 600 60 K 300 25 M 300 300
7 38.97 525 600 56 K 300 25 M 300 300
8 77.94 663 600 56 K 300 25 M 300 300
37
Problems Involving Definitions
n OSHL-U OSHL-U Otter Otter Vampire Vampire E-Setheo DCTP
n time (s) Gen time (s) Gen time (s) Gen time (s) time (s)
2 0.175 41 600 100 K 0.00 103 0.0 0.01
3 0.678 85 600 66 K 70.1 3 M 0.3 300
4 2.107 141 600 47 K 300 25 M 0.3 300
5 5.317 207 600 46 K 300 25 M 2.6 300
6 12.02 283 600 60 K 300 25 M 300 300
7 38.97 525 600 56 K 300 25 M 300 300
8 77.94 663 600 56 K 300 25 M 300 300
38
Problems Involving Definitions

• 9 other sets of problems tested
• S1 ? S2 ? ? Sn Sn ? Sn-1 ? ? S1, left side
  left associated, right side right associated
• S1 ? S2 ? ? Sn S1 ? S2 ? ? Sn ? S1 ? S2 ?
  ? Sn, both sides associated to the left
• S1 ? S2 ? ? Sn S1 ? S2 ? ? Sn ? S1 ? S2 ?
  ? Sn, left side left associated, right side right
  associated
• S1 ? S2 ? ? Sn S1 ? S2 ? ? Sn, left side
  left associated, right side right associated
• 5 similar sets of problems involving ?

• Results were similar for all 10 sets of problems
  tested

39
Implementation Results

• OSHL has no special data structures.
• Implemented in OCaML
• No special equality methods
• Semantics was implemented but frequently only
  trivial semantics was used.
• Thus significant performance improvements are
  possible.