Parallel External Directed Model Checking with Linear I/O

1
Parallel External Directed Model Checking with
Linear I/O

• Shahid Jabbar
• Stefan Edelkamp
• Computer Science Department
• University of Dortmund, Dortmund, Germany

2
Model Checking

• Given
• A model of a system.
• A specification property
• Model Checking Problem Does the system satisfy
  the property ?
• Method An exhaustive exploration of the state
  space to search for a state that does not satisfy
  the property.
• Problem How to cope with large state spaces that
  do not fit into the main memory?

3
Directed Model Checking (Edelkamp, Leue,
Lluch-Lafuente, 2004)

• A guided search in the state space.
• Usually by some heuristic estimate.
• Only promising states are explored.
• Under-certain conditions proved to be complete.
• Shorter error trails
• Better for human comprehension
• Problem The inevitable demands of the model ..
  Space, space and space.

4
Possible Solution

• Use Virtual Memory.
• Assume a bigger address space divided into pages.
  
• Saved on the hard disk but are moved back to the
  main memory whenever they are called Page
  Faults.
• Pages are mapped to physical locations within the
  main memory and the desired content is returned
  from the main memory location.

5
Problem with the Virtual Memory
Virtual Address Space
0x000000
Memory Page
0xFFFFFF
6
External Memory Model (Aggarwal and Vitter)
If the input size is very large, running time
depends on the I/Os rather than on the number of
instructions.
M
Scan(N) O(N / B) Sort(N) O(N/B log M/B N/B)
Input of size N and N gtgt M
7
External BFS (Munagala Ranade)
I Remove Duplicates by sorting the nodes
according to the indices and doing an scan and
compaction phase.
II Subtract layers t and t1 from t2.
8
A Algorithm a.k.a Goal-directed Dijkstra

• A heuristic estimate is used to guide the search.
  
• E.g. Straight line distance from the current node
  to the goal in case of a graph with a geometric
  layout.
• Reweighing w(u,v) w(u,v) h(u) h(v)
• Problems
• A needs to store all the states during
  exploration.
• A generates large amount of duplicates that can
  be removed using an internal hash table only if
  it can fit in the main memory.
• A do not exhibit any locality of expansion. For
  large state spaces, standard virtual memory
  management can result in excessive page faults.

9
Bringing Locality
h

• Implicit, unweighted, undirected graphs
• Consistent
• heuristic
• estimates.
• gt ?h -1,0,1

0 1 2 3 4 5 6






0
1
2
3
4
5
A Bucket !!
g
10
External A Edelkamp, Jabbar, and Schroedl,
2004

• Buckets represent temporal locality cache
  efficient order of expansion.
• If we store the states in the same bucket
  together we can exploit the spatial locality.
• Munagala and Ranades BFS and Korfs delayed
  duplicate detection for implicit graphs.

External A
11
External Search For Model Checking Jabbar and
Edelkamp VMCAI 05

• Uses Harddisk to store the state space divided
  in the form of Buckets.
• Implemented on top of SPIN model checker.
• Promising Largest exploration so far took 20
  GB much larger than even the address limits of
  most computers.
• Pause and Resume support Can add more
  harddisks.
• Problems
• Slow duplicate detection phase
• Internal Processing Time gtgt External I/O time

12
Solution

• Distribute the internal working on multiple
  processors.

13
Distributed Directed Model CheckingObservations

• Since each state in a Bucket is independent of
  the other they can be expanded in a parallel
  fashion.
• Duplicates removal can be distributed on
  different processors.
• Bulk (Streamed) transfers between processors are
  much better than single transfers.

14
Distributed Queue
P0
ltg, h, start byte, sizegt
lt15,34, 20, 100gt
TOP
P1
lt15,34, 0, 100gt
lt15,34, 40, 100gt
P2
lt15,34, 60, 100gt
15
Delayed Duplicate Detection

• Each state can appear several times in a bucket.
• A bucket has to be searched completely for the
  duplicates.

Single Files
GOAL
Sorted buffers
P0
P1
P2
P3
Problem Concurrent Writes !!!!
16
Multiple Processors - Multiple Disks variant
P1
P3
P4
P2
Sorted buffers w.r.t the hash val
Sorted Files
Divide w.r.t the hash ranges
Sorted buffers from every processor
Sorted File
17
I/O Complexity

• External memory algorithms are evaluated on the
  number of I/Os.
• Expansion
• Linear in I/O gt O(Scan(V))
• Delayed Duplicate Detection
• Phase I Given that enough file pointers are
  provided by the operating system
• O(scan(E))
• Else O(sort(E))
• Phase 2 Subtracting previous levels k
  .O(Scan(E)) where k is bounded by the size of
  the largest cycle in the combined automata.

18
Comparison with other approaches

• Delayed transfer.
• Bulk transfer is much better than individual
  transfers over a network.
• External Memory provides the space for large
  state spaces.

19
Deadlock Detection in CORBA-GIOP 3-2 (Space
Consumption 2.1 Gigabytes)
Time taken on Proc. 1 Time taken on Proc. 2 Time taken on Proc. 3 Speed-up
1 Processor 25m 59s
1 Processor 18m 20s
2 Processors 17m 30s 17m 29s 1.48
2 Processors 9m 49s 9m 44s 1.89
3 Processors 15m 55s 16m 6s 15m 58s 1.64
3 Processors 7m 32s 7m 28s 7m 22s 2.44
Real-time
Multiple Processors Machine
CPU-time
20
Deadlock Detection in CORBA-GIOP 4-1 (Space
Consumption 5.2 Gigabytes)
Time taken on Proc. 1 Time taken on Proc. 2 Time taken on Proc. 3 Speed-up
1 Processor 73m 10s
1 Processor 52m 50s
2 Processors 41m 42s 41m 38s 1.75
2 Processors 25m 56s 25m 49s 2.04
3 Processors 37m 24s 34m 27s 37m 20s 2.12
3 Processors 18m 8s 18m 11s 18m 20s 2.91
Real-time
Multiple Processors Machine
CPU-time
21
Deadlock Detection in CORBA-GIOP 4-2 (Space
Consumption 20 Gigabytes)
Time taken on Proc. 1 Time taken on Proc. 2 Time taken on Proc. 3 Speed-up
1 Processor 269m 9s
1 Processor 186m 12s
2 Processors 165m 25s 165m 25s 1.62
2 Processors 91m 10s 90m 32s 2.04
3 Processors 151m 6s 151m 3s 151m 5s 1.78
3 Processors 63m 12s 63m 35s 63m 59s 2.93
Real-time
Multiple Processors Machine
CPU-time
22
Deadlock Detection in Optical Telegraph (Space
Consumption 4.3 Gigabytes)
Time on Proc. 1 Time on Proc. 2 Time on Proc. 3 Speed-up
1 Processor 55m 53s
1 Processor 43m 26s
2 Processors 31m 43s 31m 36s 1.76
2 Processors 22m 46s 22m 58s 1.89
3 Processors 23m 32s 23m 17s 23m 10s 2.41
3 Processors 15m 20s 14m 24s 14m 25s 3.01
Real-time
Multiple Processors Machine
CPU-time
23
Deadlock Detection in Optical Telegraph (Space
Consumption 4.3 Gigabytes)
Time taken on Proc. 1 Time taken on Proc. 2 Speed-up
1 Processor 76m 33s
1 Processor 26m 37s
2 Processors 54m 20s 54m 6s 1.41
2 Processors 14m 11s 14m 12s 1.87
Real-time
Workstations connected via NFS
CPU-time
24
Deadlock Detection in CORBA-GIOP 4-1 (Space
Consumption 5.2 Gigabytes)
Time taken on Proc. 1 Time taken on Proc. 2 Speed-up
1 Processor 100m 27s
1 Processor 31m 6s
2 Processors 76m 38s 76m 39s 1.3
2 Processors 15m 52s 15m 31s 1.96
Real-time
Workstations connected via NFS
CPU-time
25
Summary

• State space explosion problem can be circumvented
  by Directed External Model Checking
• Time turns out to be a bottle-neck.
• not for the External I/O but for Expansion
• Internal work is divided on multiple processors.
• Delayed transfer of state sets ? low network
  cost.
• Implemented on top of IO-HSF-SPIN SPIN model
  checker with external heuristic search.
• Significant speed-up.

26
External Directed LTL Model Checking under
review

• Schuppan and Biere approach gt liveness as
  reachability.
• Liveness requires searching for an acceptance
  cycle
• A path to a previously seen state that also
  visits an accepting state.
• Save a tuple of states.
• Two new heuristics to accelerate the search.

27
External Directed LTL Model Checking under
review
0 1 2 3 4
Same states in both parts
Arrives at the final state
Arrives again at the same final state
Already seen final state
Current state